Variance

The expected value page summarized the center of a distribution. Suppose D records the dependency length of a selected syntactic relation. Two probability models may have the same mean dependency length while differing in how tightly their mass is concentrated around that mean.

Consider these constructed PMFs.

dependency length d	mass under D_1	mass under D_2
1	0	.50
2	1	0
3	0	.50

Both variables have mean 2:

\mathbb{E}[D_1]=2

and

\mathbb{E}[D_2]=1(.50)+3(.50)=2.

The means do not capture the fact that D_1 always equals 2 while D_2 is always one word away from 2.

The variance summarizes expected squared distance from the mean:

\operatorname{Var}(X) \equiv\mathbb{E}\!\left[(X-\mathbb{E}[X])^2\right].

Computing deviations before averaging

For D_2, subtract the mean from each support value.

d	d-2	(d-2)^2	mass	weighted squared deviation
1	-1	1	.50	.50
3	1	1	.50	.50

Add the final column:

\operatorname{Var}(D_2)=.50+.50=1.

For D_1, the only value equals its mean. Its squared deviation is zero, so

\operatorname{Var}(D_1)=0.

The two variables have equal means but different variances.

Why square the deviations?

If we averaged the signed deviations from the mean, the result would be zero:

(-1)(.50)+(1)(.50)=0.

This cancellation occurs for any variable with a finite mean. Squaring makes deviations on both sides contribute positively.

Squaring also gives greater weight to larger departures. A deviation of 2 contributes 2^2=4, while a deviation of 1 contributes 1^2=1. Variance is thus sensitive to probability placed far from the mean.

Computing variance in base R

Code

dependency_length <- c(1, 3)
dependency_mass <- c(.50, .50)

mean_length <- sum(dependency_length * dependency_mass)
variance_length <- sum((dependency_length - mean_length)^2 * dependency_mass)

c(mean = mean_length, variance = variance_length)

The results are 2 and 1. The code follows the definition directly: find the mean, compute squared deviations, weight them, and add.

Keeping the units visible

If dependency length is measured in words, then each deviation is measured in words. Squaring produces units of words^2. The variance of D_2 is thus 1\text{ word}^2.

Squared units make variance useful for algebra but less direct as a descriptive distance. Standard deviation takes a square root to return to the original measurement units.

Variance also requires the expected squared deviation to be finite. A variable can have a finite mean but place enough probability far from that mean for its variance to be infinite.

Signed deviations cancel

The mean signed deviation cannot summarize spread. The positive and negative deviations cancel, producing zero even for the dispersed variable D_2.

Variance repairs the cancellation by squaring before averaging. This choice defines a particular spread summary; it should not be confused with an average absolute distance.

Check your understanding

Compute the variance of a variable that takes 0 and 4 with equal probability.
Why is the average signed deviation from the mean always zero when the expectation exists?
What units does variance have when the variable is measured in milliseconds?
Can a variable have variance zero while taking two different values with positive probability? Justify your answer from the definition.

Variance measures spread in squared units. The next page expresses the same spread scale in the units of the original variable.

--- title: "Variance" --- The [expected value page](expected-values.qmd) summarized the center of a distribution. Suppose $D$ records the dependency length of a selected syntactic relation. Two probability models may have the same mean dependency length while differing in how tightly their mass is concentrated around that mean. Consider these constructed PMFs. | dependency length $d$ | mass under $D_1$ | mass under $D_2$ | |---:|---:|---:| | 1 | $0$ | $.50$ | | 2 | $1$ | $0$ | | 3 | $0$ | $.50$ | Both variables have mean 2: $$ \mathbb{E}[D_1]=2 $$ and $$ \mathbb{E}[D_2]=1(.50)+3(.50)=2. $$ The means do not capture the fact that $D_1$ always equals 2 while $D_2$ is always one word away from 2. The [**variance**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) summarizes expected squared distance from the mean: $$ \operatorname{Var}(X) \equiv\mathbb{E}\!\left[(X-\mathbb{E}[X])^2\right]. $$ ## Computing deviations before averaging For $D_2$, subtract the mean from each support value. | $d$ | $d-2$ | $(d-2)^2$ | mass | weighted squared deviation | |---:|---:|---:|---:|---:| | 1 | $-1$ | $1$ | $.50$ | $.50$ | | 3 | $1$ | $1$ | $.50$ | $.50$ | Add the final column: $$ \operatorname{Var}(D_2)=.50+.50=1. $$ For $D_1$, the only value equals its mean. Its squared deviation is zero, so $$ \operatorname{Var}(D_1)=0. $$ The two variables have equal means but different variances. ## Why square the deviations? If we averaged the signed deviations from the mean, the result would be zero: $$ (-1)(.50)+(1)(.50)=0. $$ This cancellation occurs for any variable with a finite mean. Squaring makes deviations on both sides contribute positively. Squaring also gives greater weight to larger departures. A deviation of 2 contributes $2^2=4$, while a deviation of 1 contributes $1^2=1$. Variance is thus sensitive to probability placed far from the mean. ## Computing variance in base R ```{r} #| eval: false dependency_length <- c(1, 3) dependency_mass <- c(.50, .50) mean_length <- sum(dependency_length * dependency_mass) variance_length <- sum((dependency_length - mean_length)^2 * dependency_mass) c(mean = mean_length, variance = variance_length) ``` The results are 2 and 1. The code follows the definition directly: find the mean, compute squared deviations, weight them, and add. ## Keeping the units visible If dependency length is measured in words, then each deviation is measured in words. Squaring produces units of words$^2$. The variance of $D_2$ is thus $1\text{ word}^2$. Squared units make variance useful for algebra but less direct as a descriptive distance. [Standard deviation](standard-deviation.qmd) takes a square root to return to the original measurement units. Variance also requires the expected squared deviation to be finite. A variable can have a finite mean but place enough probability far from that mean for its variance to be infinite. ## Signed deviations cancel The mean signed deviation cannot summarize spread. The positive and negative deviations cancel, producing zero even for the dispersed variable $D_2$. Variance repairs the cancellation by squaring before averaging. This choice defines a particular spread summary; it should not be confused with an average absolute distance. ## Check your understanding 1. Compute the variance of a variable that takes 0 and 4 with equal probability. 2. Why is the average signed deviation from the mean always zero when the expectation exists? 3. What units does variance have when the variable is measured in milliseconds? 4. Can a variable have variance zero while taking two different values with positive probability? Justify your answer from the definition. Variance measures spread in squared units. The [next page](standard-deviation.qmd) expresses the same spread scale in the units of the original variable.