Expectations of functions

The preceding page defined expectation as a probability weighted average. We now apply the same operation after transforming each possible value. Suppose L records the length of a word in phonological units, with this constructed PMF.

length l	p_L(l)
1	.50
2	.30
3	.20

An account of lexical access might propose that processing cost grows with squared length rather than length itself. Define the transformation

g(l)\equiv l^2.

The quantity of interest is now the expected transformed value \mathbb{E}[g(L)].

Transforming first, then average

Apply g to each support value and retain its original probability.

l	g(l)=l^2	p_L(l)	g(l)p_L(l)
1	1	.50	.50
2	4	.30	1.20
3	9	.20	1.80

Adding the last column gives

\begin{aligned} \mathbb{E}[g(L)] &=1(.50)+4(.30)+9(.20)\\ &=3.50. \end{aligned}

For a discrete random variable, when \sum_x|g(x)|p_X(x)<\infty,

\mathbb{E}[g(X)] \equiv\sum_x g(x)p_X(x).

For an absolutely continuous random variable, when \int|g(x)|f_X(x)\,\mathrm{d}x<\infty,

\mathbb{E}[g(X)] \equiv\int_{-\infty}^{\infty}g(x)f_X(x)\,\mathrm{d}x.

We can use the distribution of X directly. We do not need to derive a separate distribution for g(X) before computing the expectation.

Comparing two orders of operation

The expected word length is

\mathbb{E}[L] =1(.50)+2(.30)+3(.20) =1.70.

Squaring this mean gives

g(\mathbb{E}[L])=1.70^2=2.89.

This differs from the expected squared length:

\mathbb{E}[g(L)]=3.50.

The two operations answer different questions. g(\mathbb{E}[L]) transforms the center of the original values, whereas \mathbb{E}[g(L)] transforms every possible value and then averages the transformed results. Thus, for a nonlinear function g,

g(\mathbb{E}[X])\neq\mathbb{E}[g(X)]

in general.

Checking the calculation in base R

Code

word_length <- 1:3
word_mass <- c(.50, .30, .20)
cost <- function(length) length^2

expected_cost <- sum(cost(word_length) * word_mass)
cost_of_expected_length <- sum(word_length * word_mass)^2

c(expected_cost, cost_of_expected_length)

The results are 3.5 and 2.89.

Interpreting the transformed expectation

The value 3.5 is expressed in squared length units because the transformation squares the input. Its scientific interpretation depends on the proposed cost function. If the theory does not motivate g(l)=l^2, the calculation remains mathematically valid but does not become a theoretically useful processing summary.

This is why the transformation should be stated before the expectation is computed. The function encodes the property being averaged.

Check your understanding

Using the word length PMF, compute \mathbb{E}[2L+1] from the definition.
Compute (2\mathbb{E}[L]+1) and compare the result with your first answer.
Explain why \mathbb{E}[L^2] differs from \mathbb{E}[L]^2 here.
What theoretical claim is encoded by choosing g(l)=l^2 as the processing cost?

--- title: "Expectations of functions" --- The [preceding page](expected-values.qmd) defined expectation as a probability weighted average. We now apply the same operation after transforming each possible value. Suppose $L$ records the length of a word in phonological units, with this constructed PMF. | length $l$ | $p_L(l)$ | |---:|---:| | 1 | $.50$ | | 2 | $.30$ | | 3 | $.20$ | An account of lexical access might propose that processing cost grows with squared length rather than length itself. Define the transformation $$ g(l)\equiv l^2. $$ The quantity of interest is now the expected transformed value $\mathbb{E}[g(L)]$. ## Transforming first, then average Apply $g$ to each support value and retain its original probability. | $l$ | $g(l)=l^2$ | $p_L(l)$ | $g(l)p_L(l)$ | |---:|---:|---:|---:| | 1 | 1 | $.50$ | $.50$ | | 2 | 4 | $.30$ | $1.20$ | | 3 | 9 | $.20$ | $1.80$ | Adding the last column gives $$ \begin{aligned} \mathbb{E}[g(L)] &=1(.50)+4(.30)+9(.20)\\ &=3.50. \end{aligned} $$ For a discrete random variable, when $\sum_x|g(x)|p_X(x)<\infty$, $$ \mathbb{E}[g(X)] \equiv\sum_x g(x)p_X(x). $$ For an absolutely continuous random variable, when $\int|g(x)|f_X(x)\,\mathrm{d}x<\infty$, $$ \mathbb{E}[g(X)] \equiv\int_{-\infty}^{\infty}g(x)f_X(x)\,\mathrm{d}x. $$ We can use the distribution of $X$ directly. We do not need to derive a separate distribution for $g(X)$ before computing the expectation. ## Comparing two orders of operation The expected word length is $$ \mathbb{E}[L] =1(.50)+2(.30)+3(.20) =1.70. $$ Squaring this mean gives $$ g(\mathbb{E}[L])=1.70^2=2.89. $$ This differs from the expected squared length: $$ \mathbb{E}[g(L)]=3.50. $$ The two operations answer different questions. $g(\mathbb{E}[L])$ transforms the center of the original values, whereas $\mathbb{E}[g(L)]$ transforms every possible value and then averages the transformed results. Thus, for a nonlinear function $g$, $$ g(\mathbb{E}[X])\neq\mathbb{E}[g(X)] $$ in general. ## Checking the calculation in base R ```{r} #| eval: false word_length <- 1:3 word_mass <- c(.50, .30, .20) cost <- function(length) length^2 expected_cost <- sum(cost(word_length) * word_mass) cost_of_expected_length <- sum(word_length * word_mass)^2 c(expected_cost, cost_of_expected_length) ``` The results are `3.5` and `2.89`. ## Interpreting the transformed expectation The value 3.5 is expressed in squared length units because the transformation squares the input. Its scientific interpretation depends on the proposed cost function. If the theory does not motivate $g(l)=l^2$, the calculation remains mathematically valid but does not become a theoretically useful processing summary. This is why the transformation should be stated before the expectation is computed. The function encodes the property being averaged. ## Check your understanding 1. Using the word length PMF, compute $\mathbb{E}[2L+1]$ from the definition. 2. Compute $(2\mathbb{E}[L]+1)$ and compare the result with your first answer. 3. Explain why $\mathbb{E}[L^2]$ differs from $\mathbb{E}[L]^2$ here. 4. What theoretical claim is encoded by choosing $g(l)=l^2$ as the processing cost?