Linearity of expectation

The preceding page showed that a transformation is ordinarily applied before its expectation is taken. A linear combination has an additional simplification. Suppose a corpus sample contains three clause positions, and I_i records whether position i contains a resumptive pronoun.

T\equiv I_1+I_2+I_3.

We want the expected total count. Deriving every possible value of T would require tracking all combinations of the three indicators. Linearity of expectation lets us work term by term instead.

Beginning with one indicator

An indicator I_i takes value 1 when the target event occurs and 0 otherwise. Define its target probability by \pi_i\equiv p_{I_i}(1). Then

\begin{aligned} \mathbb{E}[I_i] &=0\times p_{I_i}(0) +1\times p_{I_i}(1)\\ &=\pi_i. \end{aligned}

The expected value of an indicator is the probability of the event it marks.

Suppose the three target probabilities are

\pi_1=.20, \qquad \pi_2=.50, \qquad \pi_3=.80.

Then the expected total is

\begin{aligned} \mathbb{E}[T] &=\mathbb{E}[I_1+I_2+I_3]\\ &=\mathbb{E}[I_1]+\mathbb{E}[I_2]+\mathbb{E}[I_3]\\ &=.20+.50+.80\\ &=1.50. \end{aligned}

The value 1.50 need not be an observed count. It is the probability weighted average count under repeated realizations of the three positions.

Stating the general rule

For random variables X_1,\ldots,X_n satisfying \mathbb E[|X_i|]<\infty and constants a_1,\ldots,a_n,

\mathbb{E}\!\left[\sum_{i=1}^n a_iX_i\right] =\sum_{i=1}^n a_i\mathbb{E}[X_i].

For two variables and an added constant, this gives

\mathbb{E}[aX+bY+c] =a\mathbb{E}[X]+b\mathbb{E}[Y]+c.

The expectation operation passes through sums and constant multiples.

Independence is not required

The three clause positions may be dependent. A speaker who uses a resumptive pronoun in one position might be more likely to use another later in the same discourse. Linearity of expectation still gives

\mathbb{E}[T]=\pi_1+\pi_2+\pi_3.

Dependence affects how probability is arranged across the possible total counts. It does not change the expected sum when the three marginal event probabilities remain fixed.

A smaller hand check makes this point visible. Suppose exactly one of two variants is selected, so I_1+I_2=1 on every outcome. If

p_{I_1}(1)=.70

and

p_{I_2}(1)=.30,

then

\mathbb{E}[I_1+I_2]=.70+.30=1.

The indicators are constrained by one another, but linearity still holds.

Computing the expected total in base R

Code

resumptive_probability <- c(.20, .50, .80)

expected_total <- sum(resumptive_probability)
expected_total

The result is 1.5. The calculation can use unequal probabilities for different linguistic contexts.

The linear rule should not be extended to nonlinear operations. In general,

\mathbb{E}[XY] \neq\mathbb{E}[X]\mathbb{E}[Y].

That product equality requires additional conditions. Linearity concerns sums and constant multiples only.

Check your understanding

If four token positions have target probabilities .10, .25, .40, and .60, what is the expected total count?
Why does the expected count calculation not require independent positions?
Show from the two possible indicator values that \mathbb{E}[I]=p_I(1).
Which operation in \mathbb{E}[XY] prevents direct use of linearity?

Linearity simplifies expectations that exist. The next page asks what must be checked before treating an expectation as a finite summary.

--- title: "Linearity of expectation" --- The [preceding page](expectation-of-functions.qmd) showed that a transformation is ordinarily applied before its expectation is taken. A linear combination has an additional simplification. Suppose a corpus sample contains three clause positions, and $I_i$ records whether position $i$ contains a resumptive pronoun. $$ T\equiv I_1+I_2+I_3. $$ We want the expected total count. Deriving every possible value of $T$ would require tracking all combinations of the three indicators. [**Linearity of expectation**](https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_%28Siegrist%29/04%3A_Expected_Value/4.01%3A_Definitions_and_Basic_Properties) lets us work term by term instead. ## Beginning with one indicator An indicator $I_i$ takes value 1 when the target event occurs and 0 otherwise. Define its target probability by $\pi_i\equiv p_{I_i}(1)$. Then $$ \begin{aligned} \mathbb{E}[I_i] &=0\times p_{I_i}(0) +1\times p_{I_i}(1)\\ &=\pi_i. \end{aligned} $$ The expected value of an indicator is the probability of the event it marks. Suppose the three target probabilities are $$ \pi_1=.20, \qquad \pi_2=.50, \qquad \pi_3=.80. $$ Then the expected total is $$ \begin{aligned} \mathbb{E}[T] &=\mathbb{E}[I_1+I_2+I_3]\\ &=\mathbb{E}[I_1]+\mathbb{E}[I_2]+\mathbb{E}[I_3]\\ &=.20+.50+.80\\ &=1.50. \end{aligned} $$ The value 1.50 need not be an observed count. It is the probability weighted average count under repeated realizations of the three positions. ## Stating the general rule For random variables $X_1,\ldots,X_n$ satisfying $\mathbb E[|X_i|]<\infty$ and constants $a_1,\ldots,a_n$, $$ \mathbb{E}\!\left[\sum_{i=1}^n a_iX_i\right] =\sum_{i=1}^n a_i\mathbb{E}[X_i]. $$ For two variables and an added constant, this gives $$ \mathbb{E}[aX+bY+c] =a\mathbb{E}[X]+b\mathbb{E}[Y]+c. $$ The expectation operation passes through sums and constant multiples. ## Independence is not required The three clause positions may be dependent. A speaker who uses a resumptive pronoun in one position might be more likely to use another later in the same discourse. Linearity of expectation still gives $$ \mathbb{E}[T]=\pi_1+\pi_2+\pi_3. $$ Dependence affects how probability is arranged across the possible total counts. It does not change the expected sum when the three marginal event probabilities remain fixed. A smaller hand check makes this point visible. Suppose exactly one of two variants is selected, so $I_1+I_2=1$ on every outcome. If $$ p_{I_1}(1)=.70 $$ and $$ p_{I_2}(1)=.30, $$ then $$ \mathbb{E}[I_1+I_2]=.70+.30=1. $$ The indicators are constrained by one another, but linearity still holds. ## Computing the expected total in base R ```{r} #| eval: false resumptive_probability <- c(.20, .50, .80) expected_total <- sum(resumptive_probability) expected_total ``` The result is `1.5`. The calculation can use unequal probabilities for different linguistic contexts. The linear rule should not be extended to nonlinear operations. In general, $$ \mathbb{E}[XY] \neq\mathbb{E}[X]\mathbb{E}[Y]. $$ That product equality requires additional conditions. Linearity concerns sums and constant multiples only. ## Check your understanding 1. If four token positions have target probabilities $.10$, $.25$, $.40$, and $.60$, what is the expected total count? 2. Why does the expected count calculation not require independent positions? 3. Show from the two possible indicator values that $\mathbb{E}[I]=p_I(1)$. 4. Which operation in $\mathbb{E}[XY]$ prevents direct use of linearity? Linearity simplifies expectations that exist. The [next page](when-expectations-fail.qmd) asks what must be checked before treating an expectation as a finite summary.