When an expectation does not exist

The expected value page defined the mean of a distribution only when the required sum or integral exists. Suppose L records utterance length in words. Every observed utterance has a finite length, and every finite sample has an ordinary arithmetic mean. It does not follow that every probability model over possible utterance lengths has a finite expected value.

An expectation exists as a finite summary only when its probability weighted contributions converge to a finite number.

For a discrete nonnegative variable,

\mathbb{E}[L]=\sum_{l=0}^{\infty}l\,p_L(l).

The masses may sum to one while the value weighted masses fail to have a finite sum.

Constructing a heavy tailed PMF

Consider the following toy model for positive utterance lengths:

p_L(l)\equiv\frac{6}{\pi^2l^2}, \qquad l=1,2,3,\ldots

The masses are nonnegative. They also sum to one because

\sum_{l=1}^{\infty}\frac{1}{l^2}=\frac{\pi^2}{6}.

Thus this assignment is a valid PMF over positive integer lengths.

Now compute the expectation:

\begin{aligned} \mathbb{E}[L] &=\sum_{l=1}^{\infty}l\frac{6}{\pi^2l^2}\\ &=\frac{6}{\pi^2}\sum_{l=1}^{\infty}\frac{1}{l}. \end{aligned}

The final series does not converge. The model assigns progressively less probability to long utterances, but length grows quickly enough that the weighted contributions have no finite sum.

Inspecting partial contributions

If we add contributions only through a finite cutoff K, we obtain

m_K=\frac{6}{\pi^2}\sum_{l=1}^{K}\frac{1}{l}.

These partial contributions keep increasing:

cutoff K	m_K
10	about 1.78
100	about 3.15
1,000	about 4.55

No finite cutoff reveals a value at which the contributions settle. The increase becomes slow, but it does not stop.

Code

partial_mean <- function(cutoff) {
  (6 / pi^2) * sum(1 / seq_len(cutoff))
}

sapply(c(10, 100, 1000), partial_mean)

The code evaluates finite partial sums. It illustrates divergence; it does not turn the infinite expectation into a finite mean.

A useful existence condition

For a variable that may take positive and negative values, a standard finite mean condition is

\mathbb{E}[|X|]<\infty.

This condition requires the expected absolute magnitude to be finite. It prevents positive and negative infinite contributions from being treated as if they cancel to a meaningful center.

Algebraic manipulations of expectation assume that the relevant expectations exist. Before splitting an expectation into terms or reporting a mean from a model, the weighted sum or integral should be checked.

What a finite sample can and cannot show

Every finite sample from the utterance-length model contains finitely many finite values, so its arithmetic mean is finite. A larger sample may contain an unusually long utterance that moves the sample mean substantially. But even a sample without such an utterance tells us only about the values observed in that sample. The model expectation depends on the probability assigned across the complete support.

Other summaries may remain defined. A threshold with half of the probability below it can exist even when the probability weighted mean does not.

Check your understanding

Why do the masses p_L(l) define a valid PMF even though \mathbb{E}[L] is not finite?
Which series appears after multiplying l by p_L(l)?
What does the condition \mathbb{E}[|X|]<\infty guarantee?
Why can every finite sample have a finite mean while the model expectation fails to exist?

The next spread summary also depends on a finite expectation of a transformed value. The next page introduces variance and keeps its existence requirement visible.

--- title: "When an expectation does not exist" --- The [expected value page](expected-values.qmd) defined the mean of a distribution only when the required sum or integral exists. Suppose $L$ records utterance length in words. Every observed utterance has a finite length, and every finite sample has an ordinary arithmetic mean. It does not follow that every probability model over possible utterance lengths has a finite expected value. An expectation exists as a finite summary only when its probability weighted contributions converge to a finite number. For a discrete nonnegative variable, $$ \mathbb{E}[L]=\sum_{l=0}^{\infty}l\,p_L(l). $$ The masses may sum to one while the value weighted masses fail to have a finite sum. ## Constructing a heavy tailed PMF Consider the following toy model for positive utterance lengths: $$ p_L(l)\equiv\frac{6}{\pi^2l^2}, \qquad l=1,2,3,\ldots $$ The masses are nonnegative. They also sum to one because $$ \sum_{l=1}^{\infty}\frac{1}{l^2}=\frac{\pi^2}{6}. $$ Thus this assignment is a valid PMF over positive integer lengths. Now compute the expectation: $$ \begin{aligned} \mathbb{E}[L] &=\sum_{l=1}^{\infty}l\frac{6}{\pi^2l^2}\\ &=\frac{6}{\pi^2}\sum_{l=1}^{\infty}\frac{1}{l}. \end{aligned} $$ The final series does not converge. The model assigns progressively less probability to long utterances, but length grows quickly enough that the weighted contributions have no finite sum. ## Inspecting partial contributions If we add contributions only through a finite cutoff $K$, we obtain $$ m_K=\frac{6}{\pi^2}\sum_{l=1}^{K}\frac{1}{l}. $$ These partial contributions keep increasing: | cutoff $K$ | $m_K$ | |---:|---:| | 10 | about $1.78$ | | 100 | about $3.15$ | | 1,000 | about $4.55$ | No finite cutoff reveals a value at which the contributions settle. The increase becomes slow, but it does not stop. ```{r} #| eval: false partial_mean <- function(cutoff) { (6 / pi^2) * sum(1 / seq_len(cutoff)) } sapply(c(10, 100, 1000), partial_mean) ``` The code evaluates finite partial sums. It illustrates divergence; it does not turn the infinite expectation into a finite mean. ## A useful existence condition For a variable that may take positive and negative values, a standard finite mean condition is $$ \mathbb{E}[|X|]<\infty. $$ This condition requires the expected absolute magnitude to be finite. It prevents positive and negative infinite contributions from being treated as if they cancel to a meaningful center. Algebraic manipulations of expectation assume that the relevant expectations exist. Before splitting an expectation into terms or reporting a mean from a model, the weighted sum or integral should be checked. ## What a finite sample can and cannot show Every finite sample from the utterance-length model contains finitely many finite values, so its arithmetic mean is finite. A larger sample may contain an unusually long utterance that moves the sample mean substantially. But even a sample without such an utterance tells us only about the values observed in that sample. The model expectation depends on the probability assigned across the complete support. Other summaries may remain defined. A threshold with half of the probability below it can exist even when the probability weighted mean does not. ## Check your understanding 1. Why do the masses $p_L(l)$ define a valid PMF even though $\mathbb{E}[L]$ is not finite? 2. Which series appears after multiplying $l$ by $p_L(l)$? 3. What does the condition $\mathbb{E}[|X|]<\infty$ guarantee? 4. Why can every finite sample have a finite mean while the model expectation fails to exist? The next spread summary also depends on a finite expectation of a transformed value. The [next page](variance-and-covariance.qmd) introduces variance and keeps its existence requirement visible.