Cumulative distribution functions

The earlier pages represented discrete probability with a probability mass function and continuous probability with a probability density function. Must we switch representations every time we move between the two kinds of variable? A cumulative distribution function gives us one threshold-based representation that applies to both. Suppose M records the number of morphemes in a sampled word token.

The numerical table below is constructed. A corpus application would first need a morphological segmentation and a decision about whether the sampled objects are word tokens, word types, or lexemes; the CDF summarizes the resulting counts but does not choose among those units.

A cumulative distribution function, abbreviated CDF, records the probability at or below every threshold:

F_X:\mathbb R\to[0,1], \qquad F_X(x)\equiv\mathbb{P}(X\leq x).

The function accumulates probability from the lower end of the support through the threshold x.

Constructing a discrete CDF by hand

Consider this constructed PMF.

morpheme count m	p_M(m)
1	.55
2	.30
3	.10
4	.05

At threshold 1, the CDF contains only the first mass:

F_M(1)=.55.

At threshold 2, it contains the first two masses:

F_M(2)=.55+.30=.85.

Continuing gives the full CDF table.

threshold m	F_M(m)
1	.55
2	.85
3	.95
4	1

For a discrete variable,

F_X(x)=\sum_{\substack{x'\in\operatorname{cod}(X)\\x'\leq x}}p_X(x').

The CDF increases in jumps when a support value has positive mass.

Accumulating a continuous density

For a continuous variable with density f_X, the same definition is represented by area:

F_X(x)=\int_{-\infty}^{x}f_X(x')\,\mathrm{d}x'.

The operation changes from summing masses to integrating density, but the interpretation remains \mathbb{P}(X\leq x).

Suppose a boundary duration variable B has

F_B(120)=.25

and

F_B(180)=.72.

Then the probability in the interval above 120 and at or below 180 is

\begin{aligned} \mathbb{P}(120<B\leq180) &=F_B(180)-F_B(120)\\ &=.72-.25\\ &=.47. \end{aligned}

The subtraction removes the probability already accumulated through 120.

Computing the discrete CDF in base R

Code

morpheme_value <- 1:4
morpheme_mass <- c(.55, .30, .10, .05)
morpheme_cdf <- cumsum(morpheme_mass)

data.frame(morpheme_value, morpheme_mass, morpheme_cdf)

The CDF must be nondecreasing and approach one at the upper end of the support. The cumsum() result lets us check both properties.

Reading the threshold direction

The definition always uses the lower tail:

F_X(x)\equiv\mathbb{P}(X\leq x).

An upper tail probability follows by complementation:

\mathbb{P}(X>x)=1-F_X(x).

For the morpheme count variable,

\mathbb{P}(M>2)=1-.85=.15,

which agrees with adding the masses at 3 and 4. Now return to the boundary duration example. The statement F_B(180)=.72 means that the event B\leq180 has probability .72. It does not give the probability above 180, which is 1-.72=.28.

Check your understanding

Use the morpheme PMF to compute F_M(3).
Compute \mathbb{P}(M>1) from the CDF and verify it from the PMF.
If F_B(200)=.81, what is \mathbb{P}(B>200)?
Explain why a discrete CDF has jumps while a continuous variable cannot add positive mass at one point.

The CDF describes probability through every threshold. The next page compresses a complete distribution into one probability weighted center.

--- title: "Cumulative distribution functions" --- The earlier pages represented discrete probability with a [probability mass function](probability-mass-functions.qmd) and continuous probability with a [probability density function](probability-density-functions.qmd). Must we switch representations every time we move between the two kinds of variable? A cumulative distribution function gives us one threshold-based representation that applies to both. Suppose $M$ records the number of morphemes in a sampled word token. The numerical table below is constructed. A corpus application would first need a morphological segmentation and a decision about whether the sampled objects are word tokens, word types, or lexemes; the CDF summarizes the resulting counts but does not choose among those units. A [**cumulative distribution function**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html), abbreviated [**CDF**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html), records the probability at or below every threshold: $$ F_X:\mathbb R\to[0,1], \qquad F_X(x)\equiv\mathbb{P}(X\leq x). $$ The function accumulates probability from the lower end of the support through the threshold $x$. ## Constructing a discrete CDF by hand Consider this constructed PMF. | morpheme count $m$ | $p_M(m)$ | |---:|---:| | 1 | $.55$ | | 2 | $.30$ | | 3 | $.10$ | | 4 | $.05$ | At threshold 1, the CDF contains only the first mass: $$ F_M(1)=.55. $$ At threshold 2, it contains the first two masses: $$ F_M(2)=.55+.30=.85. $$ Continuing gives the full CDF table. | threshold $m$ | $F_M(m)$ | |---:|---:| | 1 | $.55$ | | 2 | $.85$ | | 3 | $.95$ | | 4 | $1$ | For a discrete variable, $$ F_X(x)=\sum_{\substack{x'\in\operatorname{cod}(X)\\x'\leq x}}p_X(x'). $$ The CDF increases in jumps when a support value has positive mass. ## Accumulating a continuous density For a continuous variable with density $f_X$, the same definition is represented by area: $$ F_X(x)=\int_{-\infty}^{x}f_X(x')\,\mathrm{d}x'. $$ The operation changes from summing masses to integrating density, but the interpretation remains $\mathbb{P}(X\leq x)$. Suppose a boundary duration variable $B$ has $$ F_B(120)=.25 $$ and $$ F_B(180)=.72. $$ Then the probability in the interval above 120 and at or below 180 is $$ \begin{aligned} \mathbb{P}(120<B\leq180) &=F_B(180)-F_B(120)\\ &=.72-.25\\ &=.47. \end{aligned} $$ The subtraction removes the probability already accumulated through 120. ## Computing the discrete CDF in base R ```{r} #| eval: false morpheme_value <- 1:4 morpheme_mass <- c(.55, .30, .10, .05) morpheme_cdf <- cumsum(morpheme_mass) data.frame(morpheme_value, morpheme_mass, morpheme_cdf) ``` The CDF must be nondecreasing and approach one at the upper end of the support. The `cumsum()` result lets us check both properties. ## Reading the threshold direction The definition always uses the lower tail: $$ F_X(x)\equiv\mathbb{P}(X\leq x). $$ An upper tail probability follows by complementation: $$ \mathbb{P}(X>x)=1-F_X(x). $$ For the morpheme count variable, $$ \mathbb{P}(M>2)=1-.85=.15, $$ which agrees with adding the masses at 3 and 4. Now return to the boundary duration example. The statement $F_B(180)=.72$ means that the event $B\leq180$ has probability $.72$. It does not give the probability above 180, which is $1-.72=.28$. ## Check your understanding 1. Use the morpheme PMF to compute $F_M(3)$. 2. Compute $\mathbb{P}(M>1)$ from the CDF and verify it from the PMF. 3. If $F_B(200)=.81$, what is $\mathbb{P}(B>200)$? 4. Explain why a discrete CDF has jumps while a continuous variable cannot add positive mass at one point. The CDF describes probability through every threshold. The [next page](expected-values.qmd) compresses a complete distribution into one probability weighted center.