Expected values

The earlier pages introduced probability mass functions and probability density functions. We can now ask how to summarize the center of either distribution with one number. Suppose R records the number of self repairs in a sampled dialogue turn.

The table below is a constructed annotation distribution. A repair count depends on the transcription and disfluency scheme, as illustrated by the Switchboard disfluency annotation guidelines; it is not a direct measure of a speaker’s underlying planning process.

repair count r	p_R(r)
0	.50
1	.30
2	.15
3	.05

We want one number that summarizes the center of this probability distribution. Averaging the four support values would give (0+1+2+3)/4=1.5, but that calculation gives equal weight to values with unequal probabilities.

The expected value weights each value by its probability. It is also called the mean of the distribution when the weighted average exists.

Weighting and adding by hand

Multiply each support value by its mass.

r	p_R(r)	r p_R(r)
0	.50	0
1	.30	.30
2	.15	.30
3	.05	.15

Add the weighted contributions:

0+.30+.30+.15=.75.

In notation,

\mathbb{E}[R] \equiv\sum_{r\in\operatorname{cod}(R)} r\,p_R(r) =.75.

The symbol \mathbb{E} denotes the expectation operation. It takes the full probability distribution of R and returns its probability weighted average.

Interpreting a value outside the support

The support contains only whole repair counts, so no dialogue turn has exactly .75 repairs. The expected value need not be a possible observed value.

Let R_1,R_2,\ldots be independent variables with this PMF. Their sample mean has expectation .75 for every sample size. The later treatment of the law of large numbers explains the additional conditions under which sample averages stabilize near this value. For current purposes, the important distinction is that the expectation is a property of the distribution, not the outcome of one turn.

Computing the expectation in base R

Code

repair_count <- 0:3
repair_mass <- c(.50, .30, .15, .05)

sum(repair_count * repair_mass)

The result is .75. The multiplication is elementwise, so each support value is paired with its own probability before the products are summed.

The continuous form

For an absolutely continuous variable X with density f_X, suppose

\int_{-\infty}^{\infty}|x|f_X(x)\,\mathrm{d}x<\infty.

Its expected value is

\mathbb{E}[X] \equiv\int_{-\infty}^{\infty}x f_X(x)\,\mathrm{d}x.

The factor x weights the probability density at each value. Absolute integrability guarantees that the expectation is finite.

The discrete sum and continuous integral implement the same idea: values receiving more modeled probability contribute more to the center.

Distribution-family calculations

If K\sim\operatorname{Geometric}(\pi) counts failures before the first success, then

\begin{aligned} \mathbb E[K] &=\sum_{k=0}^{\infty}k(1-\pi)^k\pi\\ &=\pi(1-\pi) \sum_{k=1}^{\infty}k(1-\pi)^{k-1}\\ &=\frac{1-\pi}{\pi}. \end{aligned}

If X\sim\operatorname{Beta}(\alpha,\beta), then

\begin{aligned} \mathbb E[X] &=\int_0^1 x\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)} \,\mathrm dx\\ &=\frac{B(\alpha+1,\beta)}{B(\alpha,\beta)}\\ &=\frac{\alpha}{\alpha+\beta}. \end{aligned}

These equations are derived properties of the corresponding PMF and PDF. They are not definitions of the geometric or beta families.

Weighting each support value by its probability

The expectation is not the arithmetic average of the distinct support values. For the repair example, that calculation gives 1.5 rather than .75 because it treats the rare value 3 as if it were as common as 0.

Identify the probability attached to each value. The expectation averages outcomes under the distribution, not labels in the support set.

Check your understanding

Recompute \mathbb{E}[R] after moving .10 of mass from R=0 to R=2.
Why can .75 be the expected repair count when no turn has .75 repairs?
What quantity provides the weights in a discrete expectation?
Explain the shared logic of the discrete sum and the continuous integral.

Expected value averages the original values. The next page asks how to average a transformed property of those values.

--- title: "Expected values" --- The earlier pages introduced [probability mass functions](probability-mass-functions.qmd) and [probability density functions](probability-density-functions.qmd). We can now ask how to summarize the center of either distribution with one number. Suppose $R$ records the number of self repairs in a sampled dialogue turn. The table below is a constructed annotation distribution. A repair count depends on the transcription and disfluency scheme, as illustrated by the [Switchboard disfluency annotation guidelines](https://web.stanford.edu/~bresnan/128_2003-4/Docs/dflguide.pdf); it is not a direct measure of a speaker's underlying planning process. | repair count $r$ | $p_R(r)$ | |---:|---:| | 0 | $.50$ | | 1 | $.30$ | | 2 | $.15$ | | 3 | $.05$ | We want one number that summarizes the center of this probability distribution. Averaging the four support values would give $(0+1+2+3)/4=1.5$, but that calculation gives equal weight to values with unequal probabilities. The [**expected value**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) weights each value by its probability. It is also called the mean of the distribution when the weighted average exists. ## Weighting and adding by hand Multiply each support value by its mass. | $r$ | $p_R(r)$ | $r p_R(r)$ | |---:|---:|---:| | 0 | $.50$ | $0$ | | 1 | $.30$ | $.30$ | | 2 | $.15$ | $.30$ | | 3 | $.05$ | $.15$ | Add the weighted contributions: $$ 0+.30+.30+.15=.75. $$ In notation, $$ \mathbb{E}[R] \equiv\sum_{r\in\operatorname{cod}(R)} r\,p_R(r) =.75. $$ The symbol $\mathbb{E}$ denotes the expectation operation. It takes the full probability distribution of $R$ and returns its probability weighted average. ## Interpreting a value outside the support The support contains only whole repair counts, so no dialogue turn has exactly $.75$ repairs. The expected value need not be a possible observed value. Let $R_1,R_2,\ldots$ be independent variables with this PMF. Their sample mean has expectation $.75$ for every sample size. The [later treatment of the law of large numbers](../statistical-inference/monte-carlo-integration.qmd#quantifying-monte-carlo-error) explains the additional conditions under which sample averages stabilize near this value. For current purposes, the important distinction is that the expectation is a property of the distribution, not the outcome of one turn. ## Computing the expectation in base R ```{r} #| eval: false repair_count <- 0:3 repair_mass <- c(.50, .30, .15, .05) sum(repair_count * repair_mass) ``` The result is `.75`. The multiplication is elementwise, so each support value is paired with its own probability before the products are summed. ## The continuous form For an absolutely continuous variable $X$ with density $f_X$, suppose $$ \int_{-\infty}^{\infty}|x|f_X(x)\,\mathrm{d}x<\infty. $$ Its expected value is $$ \mathbb{E}[X] \equiv\int_{-\infty}^{\infty}x f_X(x)\,\mathrm{d}x. $$ The factor $x$ weights the probability density at each value. Absolute integrability guarantees that the expectation is finite. The discrete sum and continuous integral implement the same idea: values receiving more modeled probability contribute more to the center. ## Distribution-family calculations If $K\sim\operatorname{Geometric}(\pi)$ counts failures before the first success, then $$ \begin{aligned} \mathbb E[K] &=\sum_{k=0}^{\infty}k(1-\pi)^k\pi\\ &=\pi(1-\pi) \sum_{k=1}^{\infty}k(1-\pi)^{k-1}\\ &=\frac{1-\pi}{\pi}. \end{aligned} $$ If $X\sim\operatorname{Beta}(\alpha,\beta)$, then $$ \begin{aligned} \mathbb E[X] &=\int_0^1 x\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)} \,\mathrm dx\\ &=\frac{B(\alpha+1,\beta)}{B(\alpha,\beta)}\\ &=\frac{\alpha}{\alpha+\beta}. \end{aligned} $$ These equations are derived properties of the corresponding PMF and PDF. They are not definitions of the geometric or beta families. ## Weighting each support value by its probability The expectation is not the arithmetic average of the distinct support values. For the repair example, that calculation gives 1.5 rather than $.75$ because it treats the rare value 3 as if it were as common as 0. Identify the probability attached to each value. The expectation averages outcomes under the distribution, not labels in the support set. ## Check your understanding 1. Recompute $\mathbb{E}[R]$ after moving $.10$ of mass from $R=0$ to $R=2$. 2. Why can $.75$ be the expected repair count when no turn has $.75$ repairs? 3. What quantity provides the weights in a discrete expectation? 4. Explain the shared logic of the discrete sum and the continuous integral. Expected value averages the original values. The [next page](expectation-of-functions.qmd) asks how to average a transformed property of those values.