From outcomes to distributions

The outcome-to-value question

Module 1 assigned probability to events, that is, observable sets of complete outcomes.

But a linguistic question usually targets one attribute of each outcome:

• its orthographic form;
• its annotated morphosyntactic case; or
• a measured duration.

Preserve the relevant events without discarding the outcome

If an analysis concerns duration, then replacing each complete token by its duration should preserve the events needed for duration questions.

But it should not erase the token record that supports other questions.

Complete vowel-token outcomes

Suppose the random experiment selects one complete vowel token.

The outcome retains the word, speaker, waveform, vowel label, and duration.

Module 1: possibilities and outcomes

The duration mapping

Let $\Omega\equiv\{\omega_1,\omega_2,\omega_3\}$. Define

$$D(\omega)\equiv\text{duration of vowel token }\omega\text{ in milliseconds}.$$

Thus $D(\omega_1)=112$, $D(\omega_2)=83$, and $D(\omega_3)=126$.

Duration values define events

The event that duration exceeds 100 ms is

$$\{\omega\in\Omega\mid D(\omega)>100\} =\{\omega_1,\omega_3\}.$$

The event contains vowel-token outcomes, not the bare values 112 and 126.

One outcome supports two mappings

Define

$$D(\omega)\equiv\text{duration in ms}, \qquad H(\omega)\equiv \begin{cases} 1,&\text{if the vowel is high},\\ 0,&\text{otherwise}. \end{cases}$$

The same outcome may satisfy $D(\omega_1)=112$ and $H(\omega_1)=1$.

Checking the selection in R

tokens <- data.frame(
  outcome = c("omega1", "omega2", "omega3"),
  word = c("heed", "hid", "heed"),
  speaker = c("s01", "s01", "s02"),
  duration = c(112, 83, 126)
)
tokens$outcome[tokens$duration > 100]
# "omega1" "omega3"

The operation selects duration values but returns the corresponding outcome identifiers.

Retaining the complete outcomes

Keeping only 112, 83, and 126 removes speaker and word identity.

A later analysis could no longer ask whether durations repeat within speakers or differ across words.

Defining a random variable

Basically, a random variable is a rule that reads one value from each outcome.

Write the outcome-to-value mapping as

$$X:\Omega\rightarrow\mathcal X.$$

The domain $\Omega$ contains outcomes. The codomain $\mathcal X$ contains possible values. We add the formal measurability requirement after examining preimages.

Full definition in the notes · OpenStax: random variables

The declared codomain

For a random variable $X$, write its declared value space as

$$\operatorname{cod}(X).$$

Thus a sum over possible values uses $x\in\operatorname{cod}(X)$. The support may be smaller.

For a discrete variable, its support is the subset of codomain values with positive probability mass.

Work one mapping by hand

For each token $\omega\in\Omega$, define

$$D(\omega)\equiv\text{duration of token }\omega\text{ in milliseconds}.$$

Thus $D(\omega_1)=112$, $D(\omega_2)=83$, and $D(\omega_3)=126$.

The numbers are measured values in milliseconds. They are not the speech tokens themselves.

How does a value question become an event?

Suppose we ask whether a selected token lasts more than 100 ms.

Prediction: the answer should be a set of token outcomes, since Module 1 assigns probability to events rather than to ungrounded values.

Value statements denote preimage events

For a set of values $B\subseteq\mathcal X$, define the shorthand

$$X^{-1}(B) =\{\omega\in\Omega\mid X(\omega)\in B\}.$$

This set contains outcomes, not bare values.

The preimage $X^{-1}(B)$ gathers exactly the outcomes whose values land in $B$.

A threshold event

For the three-outcome illustration, let $\Omega\equiv\{\omega_1,\omega_2,\omega_3\}$. Then

$$\{\omega\in\Omega\mid D(\omega)>100\} =\{\omega_1,\omega_3\}.$$

This is the duration-threshold event in $\mathcal F$.

A value-space question becomes probabilistically meaningful by pointing back to an event in $\Omega$.

Which mappings are allowed?

We need every value set we might ask about to point back to an observable event.

This requirement is measurability.

The value space also needs a rulebook

Let $\mathcal G$ be the collection of value sets we permit in probability questions.

Like the event space $\mathcal F$ from Module 1, $\mathcal G$ must also permit:

• the complement of every permitted set; and
• the union of every finite or countably infinite list of permitted sets.

The structure $\langle\mathcal X,\mathcal G\rangle$ is a measurable space.

Measurability makes value probabilities well defined

The function $X$ is measurable when

$$B\in\mathcal G \quad\Longrightarrow\quad X^{-1}(B)\in\mathcal F,$$

Thus every permitted value question has an observable preimage event.

A random variable is a function $X:\langle\Omega,\mathcal F\rangle\to\langle\mathcal X,\mathcal G\rangle$ satisfying this condition.

Notes: measurable random variables · Module 1: sigma-algebras

A case-coded pronoun variable

Let each outcome be a contextualized pronoun token. Define $R(\omega)$ as its combined form-and-case label, with codomain

$$\begin{aligned} (&\textit{I}_{\mathrm{Nom}},\textit{me}_{\mathrm{Acc}}, \textit{you}_{\mathrm{Nom}},\textit{you}_{\mathrm{Acc}}, \textit{they}_{\mathrm{Nom}},\textit{them}_{\mathrm{Acc}},\\ &\textit{it}_{\mathrm{Nom}},\textit{it}_{\mathrm{Acc}}, \textit{she}_{\mathrm{Nom}},\textit{her}_{\mathrm{Acc}}, \textit{he}_{\mathrm{Nom}},\textit{him}_{\mathrm{Acc}}, \textit{we}_{\mathrm{Nom}},\textit{us}_{\mathrm{Acc}}). \end{aligned}$$

An integer may store a position in this ordered codomain, but that code has no linguistic meaning apart from the declared correspondence.

Preimages are events

For the first three labels in the declared order,

$$R^{-1}(\{\textit{I}_{\mathrm{Nom}},\textit{me}_{\mathrm{Acc}}, \textit{you}_{\mathrm{Nom}}\}) =\{\omega\in\Omega\mid R(\omega)\in \{\textit{I}_{\mathrm{Nom}},\textit{me}_{\mathrm{Acc}}, \textit{you}_{\mathrm{Nom}}\}\}.$$

More generally,

$$R^{-1}(B) \equiv\{\omega\in\Omega\mid R(\omega)\in B\}.$$

The measurability condition requires this preimage to belong to $\mathcal F$ for every $B\in\mathcal G$.

The set on the right contains token records, not three abstract pronoun spellings.

What moves from outcomes to values?

The probability space remains on $\Omega$. The random variable induces probabilities for measurable sets of values.

Prediction: the probability of a value set $B$ must equal the probability of its preimage event.

Distribution of a random variable

The distribution of $X$ assigns each $B\in\mathcal G$ the probability

$$\mathbb{P}(X\in B) \equiv \mathbb{P}\!\left(X^{-1}(B)\right).$$

As a function of $B$, this assignment is a probability measure on $\langle\mathcal X,\mathcal G\rangle$. We retain $\mathbb{P}$ for the probability measure on $\langle\Omega,\mathcal F\rangle$ and introduce no second measure symbol.

Notes: reading probability through preimages

Can the possible values be listed?

This question identifies discrete distributions. We ask separately whether a non-discrete distribution has a density.

Discrete random variables

A random variable $X$ is discrete when some finite or countably infinite set $\mathcal S$ satisfies $\mathbb{P}(X\in\mathcal S)=1$.

Annotated grammatical person has finite categorical support. A token count or an utterance-length count has finite or countably infinite numerical support.

Discrete does not mean numerical or unordered. It means that a listable set carries all probability.

Codomain and support

The codomain lists values the function may return. The support contains the values with positive probability:

$$\operatorname{supp}(X) \equiv \{x\in\operatorname{cod}(X)\mid \mathbb P(X=x)>0\}.$$

Finite and countably infinite support

$$\operatorname{supp}(P) =\{\mathrm{first},\mathrm{second},\mathrm{third}\}$$

is finite. An utterance-length support such as $\{0,1,2,\ldots\}$ is countably infinite. Both are discrete.

Values define events

$$L=5 \quad\Longleftrightarrow\quad \{\omega\in\Omega\mid L(\omega)=5\}.$$

$$L\geq5 \quad\Longleftrightarrow\quad \{\omega\in\Omega\mid L(\omega)\in\{5,6,7,\ldots\}\}.$$

An observed sample is not the support

For observed lengths

$$2,5,1,5,3,$$

the distinct observed values are $\{1,2,3,5\}$. The modeled support may still contain $0$, $4$, $6$, and larger integers.

Discrete does not mean unordered

Grammatical person is discrete without a numerical distance between labels. Utterance length is discrete and ordered, with interpretable differences.

Discreteness concerns whether the probability-carrying values are separate and listable.

Notes: discrete support

How much probability belongs to each value?

For a discrete variable, separate codomain values can receive probability mass.

The function that records those probabilities is the probability mass function (PMF).

Defining a probability mass function

For a discrete random variable $X$, its probability mass function (PMF) is defined by

$$p_X:\mathcal X\to[0,1], \qquad p_X(x) \equiv\mathbb{P}(X=x).$$

The subscript names the random variable; the argument is a possible value.

Its support is the subset of codomain values with positive mass:

$$\operatorname{supp}(X) \equiv \{x\in\operatorname{cod}(X):p_X(x)>0\}.$$

Notes: probability mass functions

Requirements for a PMF

Its PMF satisfies

$$p_X(x)\geq0 \qquad\text{and}\qquad \sum_{x\in\operatorname{cod}(X)}p_X(x)=1.$$

For any $A\in\mathcal G$,

$$\mathbb{P}(X\in A) =\sum_{x\in A\cap\operatorname{cod}(X)}p_X(x).$$

Work a PMF question

Suppose $S$ counts syllables in a sampled word token.

Because $\{\omega\in\Omega\mid S(\omega)=3\}$ and $\{\omega\in\Omega\mid S(\omega)=4\}$ are disjoint,

$$\mathbb{P}(S\geq3)=.17+.08=.25.$$

For a discrete variable, an event probability is the sum of the masses at the codomain values in that event. Zero-mass values contribute zero.

Interpreting zero mass

If $p_S(7)=0$, then

$$\mathbb P\!\left(\{\omega\in\Omega\mid S(\omega)=7\}\right)=0$$

under the stated model. This does not imply that seven-syllable words are impossible in every linguistic population.

Omitted mass changes the model

Dropping the row with mass $.08$ leaves total mass $.92$, which is not a PMF.

Renormalizing the remaining rows defines a different model, conditional on at most three syllables.

What if modeled values fill an interval?

A duration is recorded at finite precision, but that storage convention does not force the model to have discrete support.

Prediction: interval probabilities should come from area, while individual points receive zero mass.

Which real-valued sets may receive probability?

A Borel set belongs to the smallest collection of subsets of $\mathbb R$ that contains every open interval and is closed under complements and countable unions.

Thus ordinary intervals, threshold sets, and sets formed from them by these operations are Borel.

MIT probability notes: Borel sets

Absolutely continuous random variables

A real-valued random variable $X$ is absolutely continuous when there is a function $f_X:\mathbb R\to[0,\infty)$ such that, for every Borel set $A\subseteq\mathbb R$,

$$\mathbb{P}(X\in A)=\int_A f_X(x)\,\mathrm{d}x.$$

The function $f_X$ is a probability density function.

Notes: continuous random variables

An uncountable codomain is not enough

Recall from Module 1 that a formant-frequency estimate records the estimated center frequency of a vocal-tract resonance. Suppose $X$ returns the first two such estimates $\langle F_1,F_2\rangle$ from a vowel token. Let the idealized value space be $\mathbb R_+^2$ and define

$$X:\Omega\to\mathbb R_+^2, \qquad X(\omega)\equiv\langle F_1(\omega),F_2(\omega)\rangle.$$

What absolute continuity adds

The codomain $\mathbb R_+^2$ is uncountable. That fact alone does not make the distribution absolutely continuous.

Absolute continuity requires probabilities to be recovered by integrating a density over ordinary two-dimensional area.

The units are hertz; the pair is an acoustic measurement, not a phoneme or vowel category.

A point and an interval are different events

Under a continuous model,

$$\mathbb P\!\left(\{\omega\in\Omega\mid V(\omega)=18.417\}\right)=0,$$

while an interval around that point may receive positive probability:

$$\mathbb P\!\left(\{\omega\in\Omega\mid 18<V(\omega)<19\}\right)>0.$$

Measurement precision does not choose the support

A stored value of 18 ms may be a rounded observation of an interval-valued quantity, or it may be the discrete output of the measurement process.

The scientific target determines which representation is appropriate.

What makes a function a PDF?

The density of $X$ satisfies

$$f_X(x)\geq0 \qquad\text{and}\qquad \int_{-\infty}^{\infty}f_X(x)\,\mathrm{d}x=1.$$

Conversely, any measurable nonnegative function with integral one defines an absolutely continuous probability distribution by integration.

Notes: probability density functions

Work an interval probability

Suppose

$$f_D(d)\equiv \begin{cases} 1/100,&50\leq d\leq150,\\ 0,&\text{otherwise}. \end{cases}$$

Then

$$\mathbb{P}(80\leq D\leq100)=\int_{80}^{100}\frac{1}{100}\,\mathrm{d}u=.20.$$

This is a model for a measured token duration. It is not a claim that durations are actually uniform.

Density has reciprocal measurement units

If $R$ is measured in syllables per second, then $f_R(r)$ is measured in seconds per syllable.

Interval width times density is unitless probability.

A density height may exceed one

Height 2 across an interval of width $.5$ has total area

$$2(.5)=1.$$

Density height is not bounded by one. Probability is.

Points have no mass under a density

For an absolutely continuous random variable,

$$\mathbb{P}(X=x)=\int_x^x f_X(t)\,\mathrm{d}t=0.$$

Thus $f_X(x)>0$ does not imply $\mathbb{P}(X=x)>0$.

Density height is not point probability. Only an integral of density gives probability.

Can one representation cover both cases?

Both a PMF and a PDF answer probability questions, but they do so differently.

The cumulative distribution function (CDF) instead asks one shared threshold question: how much probability lies at or below $x$?

Defining a cumulative distribution function

Every real-valued random variable has a cumulative distribution function (CDF), defined by

$$F_X:\mathbb R\to[0,1], \qquad F_X(x) \equiv\mathbb{P}(X\leq x).$$

The same definition applies to discrete, continuous, and mixed distributions.

Notes: cumulative distribution functions

Compute the same object two ways

For a discrete $X$,

$$F_X(x)=\sum_{t\leq x}p_X(t).$$

For an absolutely continuous $X$,

$$F_X(x)=\int_{-\infty}^{x}f_X(t)\,\mathrm{d}t.$$

These are consequences of the CDF definition.

PMFs, PDFs, and CDFs are different representations of distributions, not interchangeable names.

Which single number marks a distribution’s balance point?

A plain average gives equal weight to observed rows. A distributional average weights every possible value by its probability.

This probability-weighted average is the expected value.

When is the average well defined?

A variable is integrable when its expected absolute value is finite:

$$\mathbb E[|X|]<\infty.$$

This requirement prevents undefined cancellation between infinite positive and negative contributions.

Expectations average with respect to probability

For an integrable random variable $X$, define

$$\mathbb E[X] \equiv \begin{cases} \sum_{x\in\operatorname{cod}(X)} x\,p_X(x),&X\text{ discrete},\\[2mm] \int_{-\infty}^{\infty}x f_X(x)\,\mathrm{d}x,&X\text{ absolutely continuous}. \end{cases}$$

Notes: expected values

Work a weighted average

Suppose $S$ counts syllables in a sampled word token and takes values $1$, $2$, and $4$ with masses $.25$, $.50$, and $.25$. Then

$$\begin{aligned} \mathbb E[S] &=1(.25)+2(.50)+4(.25)\\ &=2.25. \end{aligned}$$

An expectation need not be a possible syllable count for any single token.

$\mathbb E[S]=2.25$ describes the distribution, not an individual word token.

What if the target is not the original value?

Some questions concern a transformed value, such as squared distance from a reference point.

Prediction: transform each possible value first, then average those transformed values using the original probabilities.

Expectations of functions

When $g(X)$ is integrable, define

$$\mathbb E[g(X)] \equiv\sum_{x\in\operatorname{cod}(X)}g(x)p_X(x)$$

in the discrete case, with the analogous integral in the continuous case.

In general, $\mathbb E[g(X)]\neq g(\mathbb E[X])$.

Notes: expectations of functions

Transform first, then average

For $L\in\{1,2,4\}$ with masses $(.25,.50,.25)$ and $g(l)=l^2$,

$$\mathbb E[L^2] =1^2(.25)+2^2(.50)+4^2(.25) =6.25.$$

Averaging first gives a different value

$$\mathbb E[L]=2.25 \qquad\text{but}\qquad \mathbb E[L]^2=5.0625.$$

Thus $\mathbb E[L^2]\neq\mathbb E[L]^2$. The function encodes the property being averaged.

Which transformations can move through expectation?

For a linear transformation $g(x)=ax+b$, averaging then transforming gives the same result as transforming then averaging.

Let us derive that claim before using it.

Indicators make the rule concrete

If $I_i$ marks a target event and $\pi_i\equiv p_{I_i}(1)$, then

$$\mathbb E[I_i]=\pi_i.$$

For probabilities $.20$, $.50$, and $.80$,

$$\mathbb E[I_1+I_2+I_3]=1.50.$$

Deriving linearity for a finite discrete model

For constants $a$ and $b$,

$$\begin{aligned} \mathbb E[aX+b] &=\sum_{x\in\operatorname{cod}(X)}(ax+b)p_X(x)\\ &=a\sum_{x\in\operatorname{cod}(X)}xp_X(x) +b\sum_{x\in\operatorname{cod}(X)}p_X(x)\\ &=a\mathbb E[X]+b. \end{aligned}$$

Linearity for sums

If each $X_i$ is integrable,

$$\mathbb E\!\left[\sum_{i=1}^{n}a_iX_i\right] =\sum_{i=1}^{n}a_i\mathbb E[X_i].$$

This equality does not require independence. Dependence can change the distribution and variance of the sum.

Notes: linearity of expectation

Expectation adds across variables even when their values co-vary.

Independence is not required

The indicator values may be dependent. Linearity still gives

$$\mathbb E[I_1+I_2+I_3] =\mathbb E[I_1]+\mathbb E[I_2]+\mathbb E[I_3].$$

The corresponding product rule does require additional conditions:

$$\mathbb E[XY]\neq\mathbb E[X]\mathbb E[Y]$$

in general.

Does every distribution have a mean?

No. The positive and negative parts of the weighted integral must not both diverge.

Symmetry does not guarantee an expectation

For $X\sim\operatorname{Cauchy}(x_0,\gamma)$ with $\gamma>0$,

$$f_X(x) \equiv \frac{1}{\pi\gamma\left[1+\left(\frac{x-x_0}{\gamma}\right)^2\right]}.$$

Why symmetric cancellation does not repair the mean

The positive and negative parts of $\int_{-\infty}^{\infty}x f_X(x)\,\mathrm dx$ are both infinite. Thus $\mathbb E[X]$ is undefined.

Integrating over $[-M,M]$ gives zero for every $M$. The limit of these symmetric truncations is the symmetric principal value, not an expectation.

Notes: when expectations fail

Finite samples do not create a model expectation

Every finite sample of Cauchy draws has a finite arithmetic mean. An extreme draw can still move that mean by an arbitrarily large amount as the sample grows.

The model supplies no finite population mean to which the sample means must converge.

How far do values tend to lie from the mean?

Signed deviations cancel. Squaring them yields nonnegative distances and weights large deviations more heavily.

Their expectation is the variance.

Equal means can hide different spreads

Let $D_1$ equal 2 with probability one. Let $D_2$ equal 1 or 3, each with probability $.50$.

$$\mathbb E[D_1]=\mathbb E[D_2]=2, \qquad \operatorname{Var}(D_1)=0, \qquad \operatorname{Var}(D_2)=1.$$

What must exist before variance?

The second moment is $\mathbb E[X^2]$.

It is finite when

$$\mathbb E[X^2]<\infty.$$

This condition ensures that the squared deviations used by variance have a finite average.

Defining variance

For a random variable with a finite second moment, define

$$\operatorname{Var}(X) \equiv\mathbb E\!\left[(X-\mathbb E[X])^2\right].$$

Variance is an expected squared deviation from the mean.

Notes: variance

Derive an equivalent calculation

Writing $\mu\equiv\mathbb E[X]$,

$$\begin{aligned} \operatorname{Var}(X) &=\mathbb E[X^2-2\mu X+\mu^2]\\ &=\mathbb E[X^2]-2\mu\mathbb E[X]+\mu^2\\ &=\mathbb E[X^2]-2\mu^2+\mu^2\\ &=\mathbb E[X^2]-\mu^2\\ &=\mathbb E[X^2]-\mathbb E[X]^2. \end{aligned}$$

Standard deviation

Variance squares the measurement unit. Taking its square root returns to the original unit.

The standard deviation is defined by

$$\sigma_X \equiv\sqrt{\operatorname{Var}(X)}.$$

If $X$ is measured in milliseconds, $\operatorname{Var}(X)$ has units ms$^2$ and $\sigma_X$ has units ms.

Notes: standard deviation

Voice onset time on the original scale

If

$$\operatorname{Var}(V)=625\text{ ms}^2,$$

then

$$\sigma_V=\sqrt{625\text{ ms}^2}=25\text{ ms}.$$

Standard deviation is not a complete shape description

Standard deviation is the root mean squared distance from the mean, not the average absolute distance.

Equal means and standard deviations do not guarantee equal tails, symmetry, or modality.

Standard deviation does not determine coverage

The mean and standard deviation do not determine how much probability lies within $\mu\pm\sigma$.

Coverage claims require an additional distribution-shape assumption.

What else can centered powers retain?

Variance uses the second power. Higher centered powers can summarize other aspects of distribution shape, when the corresponding expectations exist.

Defining central moments

When it exists, the $k$th central moment is defined by

$$\mu_k \equiv\mathbb E\!\left[(X-\mathbb E[X])^k\right].$$

Thus $\mu_1=0$ and $\mu_2=\operatorname{Var}(X)$.

Notes: central moments

Higher central moments

For $J\in\{1,2,5\}$ with masses $(.25,.50,.25)$ and $\mathbb E[J]=2.5$,

$$\mu_3=3, \qquad \mu_4=11.0625.$$

Odd powers retain direction. Higher powers give more weight to values far from the mean.

Moments do not identify a distribution

Different distributions can share the same first several moments.

Thus moments summarize specified centered powers; they do not determine the observation process or the full distributional shape.

What does choosing a family assert?

A distribution family links a support to a particular probability form.

Thus family choice must follow the observation process and the research question, not merely the shape of a histogram.

Distribution notation

A parameter is a fixed value that selects one distribution from a family. Changing it may change location, spread, shape, or several of these properties.

The statement

$$X\sim\mathcal D(\theta)$$

means that the distribution of $X$ belongs to family $\mathcal D$ with parameter $\theta$. The symbol $\sim$ does not mean numerical equality.

It says that $X$ has a distribution in the named family under the stated parameter convention.

Question: which category does one token receive?

Suppose one contextualized pronoun token receives exactly one form-and-case label from a finite list.

The categorical distribution assigns one probability to each label.

The categorical distribution

For labels $c_1,\ldots,c_K$ and $\boldsymbol\pi\in[0,1]^K$ with $\sum_{k=1}^K\pi_k=1$,

$$R\sim\operatorname{Categorical}(\boldsymbol\pi)$$

means

$$p_R(c_k) \equiv\mathbb{P}(R=c_k) =\pi_k.$$

Notes: categorical distributions · Stan reference

Declare the category order

Order the pronoun labels as shown in the graph and define the probability vector

$$\boldsymbol\pi \equiv(.03,.09,.03,.12,.07,.28,.07,.05,.02,.02,.07,.08,.05,.02).$$

The $k$th entry is the probability of the $k$th form-and-case label. Reordering the labels without applying the same permutation to $\boldsymbol\pi$ changes the distribution.

The labels encode two annotated attributes. They are not fourteen distinct English spellings.

PMF for the pronoun categorical distribution

Question: does the token have Case=Acc?

The categorical record retains all fourteen labels. If the question concerns one binary contrast, we can map those labels to 1 and 0.

Prediction: different spellings may receive the same value, and a syncretic spelling may receive different values in different contexts.

Bernoulli distribution on pronoun case

Using $\pi=.27$,

$$p_X(1)=.27, \qquad p_X(0)=.73.$$

The parameter is the probability of annotated accusative case because the definition of $X$ assigns Case=Acc outcomes the value $1$.

Bernoulli parameters inherit their meaning from the event coded as 1.

Bernoulli mean and variance

For $X\sim\operatorname{Bernoulli}(\pi)$,

$$\mathbb E[X]=\pi, \qquad \operatorname{Var}(X)=\pi(1-\pi).$$

The mean equals the probability of the event coded as 1.

The coding determines the parameter

If the coding is reversed, the parameter becomes the probability of the complementary event.

Thus $\pi$ is not an intrinsic property of the spelling. It inherits its meaning from the declared outcome-to-value mapping.

Question: how many successes occur in fixed opportunities?

Suppose we predeclare $n$ token opportunities and count how many have Case=Acc.

Prediction: a binomial model needs a fixed $n$, independent indicators, and one common success probability $\pi$.

The binomial distribution

Let $Y_1,\ldots,Y_n$ be independent Bernoulli variables with common parameter $\pi$, and define $K\equiv\sum_{i=1}^nY_i$. Then

$$K\sim\operatorname{Binomial}(n,\pi),$$

with support $\{0,\ldots,n\}$.

Notes: binomial distributions

Why does the binomial coefficient appear?

The binomial coefficient ${n\choose k}$ counts the ways to choose $k$ positions from $n$. Each sequence with $k$ ones has probability $\pi^k(1-\pi)^{n-k}$. Thus

$$p_K(k) \equiv\mathbb{P}(K=k) ={n\choose k}\pi^k(1-\pi)^{n-k}.$$

${n\choose k}$ counts the distinct indicator sequences that produce the same total $k$.

A binomial probability by hand

For $n=6$, $\pi=.25$, and $k=2$,

$$p_K(2) ={6\choose2}(.25)^2(.75)^4 \approx.297.$$

The calculation concerns exactly two target outcomes among six declared opportunities.

Binomial center and spread

For $K\sim\operatorname{Binomial}(n,\pi)$,

$$\mathbb E[K]=n\pi, \qquad \operatorname{Var}(K)=n\pi(1-\pi).$$

The linguistic assumptions remain visible

The model requires a fixed number of opportunities, independent indicators, and one common success probability.

The count $K=2$ does not retain which two positions were successes, and the same proportion can arise from different values of $n$.

What changes without replacement?

If selecting one clause changes which clauses remain available, later draws are not independent Bernoulli trials.

The hypergeometric distribution represents a count from a fixed finite population sampled without replacement.

Sampling without replacement

Suppose an archive contains $N$ clauses, $M$ of which are passive. Select $n$ clauses without replacement. If $K$ counts passives, then

$$p_K(k) \equiv\mathbb{P}(K=k) =\frac{{M\choose k}{N-M\choose n-k}}{{N\choose n}}.$$

The support satisfies

$$\max(0,n-N+M)\leq k\leq\min(n,M).$$

The linguistic unit

One unit is a clause record, and passive voice is a declared annotation on that record.

Under English Universal Dependencies, Voice=Pass marks a verbal past participle used as the predicate of a passive clause.

Notes: hypergeometric distributions · UD English Voice · R manual

A hypergeometric probability by hand

For $N=30$, $M=9$, $n=5$, and $k=2$,

$$p_K(2) =\frac{{9\choose2}{21\choose3}}{{30\choose5}} \approx.336.$$

Mean and finite population correction

$$\mathbb E[K]=5\frac9{30}=1.5, \qquad \operatorname{Var}(K) =5(.3)(.7)\frac{25}{29}.$$

The factor $25/29$ reduces variance because sampling one clause changes what remains.

Without replacement versus independent draws

A binomial count from independent draws with probability $.30$ also has mean $1.5$, but variance $5(.3)(.7)$.

The hypergeometric variance is smaller because the draws are dependent after clauses are removed.

Question: how long do we wait for one target event?

Let each opportunity be a Bernoulli trial, and let $K$ count failures before the first success.

Before naming the family, we need probabilities over $0,1,2,\ldots$ that sum to one.

A geometric series defines a PMF

A geometric series multiplies each successive term by the same ratio. Begin with

$$\sum_{k=0}^{\infty}\frac{1}{2^{k+1}}=1.$$

Thus

$$p_K(k)\equiv\frac{1}{2^{k+1}}, \qquad k=0,1,2,\ldots$$

is a PMF with countably infinite support.

Define the geometric family

Let $K$ count failures before the first success in independent Bernoulli trials with common success probability $\pi$. Define

$$K\sim\operatorname{Geometric}(\pi)$$

by the PMF

$$p_K(k) \equiv\mathbb{P}(K=k) =(1-\pi)^k\pi, \qquad k=0,1,2,\ldots$$

This is the convention used by dgeom in R. The total-trial count is $T\equiv K+1$.

Notes: geometric distributions · R manual: geometric distribution

A geometric waiting probability

For $\pi=.20$,

$$p_K(3)=(.80)^3(.20)=.1024, \qquad \mathbb P(K\geq4)=(.80)^4=.4096.$$

The first quantity fixes the first target after three failures; the second requires four initial failures.

$\operatorname{Geometric}(.5)$

As $\pi$ decreases, the distribution spreads rightward

As $\pi$ increases, mass concentrates at zero

Every geometric PMF decreases from its first support point

The mode is a value with the greatest probability mass. A mode at the smallest possible value is a boundary mode.

For every $k\geq0$,

$$\frac{p_K(k+1)}{p_K(k)} =\frac{(1-\pi)^{k+1}\pi}{(1-\pi)^k\pi} =1-\pi <1.$$

Thus $p_K(k+1)<p_K(k)$. If a positive count is $L\equiv K+1$, every geometric model has its mode at $L=1$.

This is a model prediction. We can now compare it with an empirical distribution.

What family removes the boundary-mode restriction?

The negative binomial distribution counts failures before the $r$th success. The additional parameter $r$ permits an interior mode.

This is a family comparison, not yet a claim that dictionary symbols arise from literal Bernoulli trials.

The negative binomial generalizes the geometric

Let $K$ count failures before the $r$th success, where $r\in\{1,2,\ldots\}$. Define

$$K\sim\operatorname{NegBin}(r,\pi)$$

by

$$p_K(k) \equiv\mathbb{P}(K=k) ={k+r-1\choose k}(1-\pi)^k\pi^r, \qquad k=0,1,2,\ldots$$

When $r=1$, the binomial coefficient equals one and this PMF is exactly geometric.

Notes: negative-binomial distributions · R manual

Begin with $r=1$ and $\pi=.5$

Increasing $r$ permits an interior mode

With $\pi$ fixed, increasing $r$ increases

$$\mathbb E[K]=r\frac{1-\pi}{\pi}.$$

With $r=5$, smaller $\pi$ moves mass farther right

With $r=5$, larger $\pi$ concentrates mass near zero

A larger $r$ can offset a large $\pi$

Mean and variance under the negative binomial

Set $K\equiv L-1$ so that the observed support begins at zero. Under this negative-binomial convention,

$$\mathbb E[K]=r\frac{1-\pi}{\pi}, \qquad \operatorname{Var}(K)=r\frac{1-\pi}{\pi^2}.$$

What does dispersion mean for a count?

For a count distribution, dispersion compares variance with the mean.

• Overdispersion: $\operatorname{Var}(K)>\mathbb E[K]$.
• Equidispersion: $\operatorname{Var}(K)=\mathbb E[K]$.
• Underdispersion: $\operatorname{Var}(K)<\mathbb E[K]$.

What dispersion does the family predict?

Writing $\mu\equiv\mathbb E[K]$ gives

$$\operatorname{Var}(K)=\mu+\frac{\mu^2}{r}>\mu$$

for every finite $r$.

Thus every finite negative-binomial model is overdispersed relative to its mean.

A negative-binomial probability by hand

For $r=3$, $\pi=.20$, and $k=4$,

$$p_K(4) ={6\choose4}(.80)^4(.20)^3 =.049152.$$

The final draw is the third target; the four failures occupy the first six positions.

Model probabilities or observed proportions?

A model PMF assigns probabilities to possible values.

An empirical distribution assigns each observed value its relative frequency.

CMUdict pronunciation-entry length

Use every noncomment pronunciation entry in CMU Pronouncing Dictionary 0.7b. For entry $\omega$, define

$$L(\omega)\equiv\text{number of ARPAbet-based pronunciation symbols in }\omega.$$

The unit is a pronunciation entry, not a corpus token or necessarily a unique spelling type.

CMUdict file-format documentation · Notes: the worked example

Pronunciation-entry lengths have an interior mode

The geometric family misses the observed mode

The empirical relative-frequency function has its mode at six symbols. A fitted geometric PMF must have its mode at one symbol.

How does likelihood compare parameter values?

Hold the observed data fixed. The likelihood is the joint PMF evaluated at observed discrete data, or the joint density evaluated at observed continuous data, treated as a function of the parameters.

A maximum-likelihood estimate is a parameter value that maximizes this function, when such a value exists. A larger likelihood means a better fit to these observations within the compared family.

Compare the prediction with CMUdict

Prediction: a finite $r$ cannot match underdispersed data, whose variance is below their mean.

For the pronunciation-entry lengths, the mean of $K$ is 5.381 and its variance is 4.566.

For each $r$, estimate $\pi$ and evaluate the likelihood of all observed entry lengths.

Negative-binomial fit

The likelihood increases toward $r\to\infty$, the Poisson boundary derived next.

The extra parameter fixes the geometric mode restriction, but every finite negative-binomial model remains overdispersed relative to its mean.

Empirical, geometric, and Poisson-limit CDFs

The negative-binomial generalization removes the geometric family’s boundary-mode restriction. For these dictionary entries, the maximum-likelihood fit approaches its Poisson limit rather than a finite $r$.

An added parameter can repair one mismatch while leaving another. Model criticism must name both.

Which family appears at the dispersion boundary?

The fitted negative-binomial sequence approaches a Poisson distribution while keeping its mean fixed.

Let us state that limit before interpreting a Poisson count.

The Poisson limiting case

Fix $\lambda>0$ and let

$$K_r\sim\operatorname{NegBin}\!\left(r,\frac{r}{r+\lambda}\right).$$

Then, for every $k=0,1,2,\ldots$,

$$\lim_{r\to\infty}\mathbb P(K_r=k) =e^{-\lambda}\frac{\lambda^k}{k!}.$$

Thus the negative-binomial family approaches $\operatorname{Poisson}(\lambda)$ as $r\to\infty$ while its mean remains $\lambda$.

Question: a count per what opportunity?

A count has no interpretable rate without an exposure: tokens, clauses, speakers, minutes, or another declared unit.

Poisson counts require a declared exposure

If events occur at rate $\lambda>0$ per exposure unit, a Poisson model for exposure $t>0$ specifies

$$Y_t\sim\operatorname{Poisson}(\lambda t),$$

with

$$p_{Y_t}(y) \equiv\mathbb{P}(Y_t=y) =e^{-\lambda t}\frac{(\lambda t)^y}{y!}, \qquad y=0,1,2,\ldots$$

Notes: Poisson distributions · R manual

Poisson probabilities by hand

For $\lambda t=2$,

$$p_{Y_t}(0)=e^{-2}\approx.135, \qquad p_{Y_t}(2)=\frac{e^{-2}2^2}{2!}\approx.271.$$

The first probability concerns no filled pauses; the second concerns exactly two in the declared exposure.

A Poisson distribution is not a complete process claim

The count model implies

$$\mathbb E[Y_t]=\operatorname{Var}(Y_t)=\lambda t.$$

A Poisson process with a constant rate additionally specifies $N(0)=0$, independent counts in disjoint intervals, and the same expected count for intervals of the same length.

A Poisson marginal distribution does not by itself license a story about a Poisson process.

Question: what if equal-length intervals receive equal probability?

On a finite interval, constant density produces probability proportional to interval length.

This is the continuous uniform distribution.

The continuous uniform distribution

For $a<b$,

$$X\sim\operatorname{Uniform}(a,b)$$

means

$$f_X(x)\equiv \begin{cases} 1/(b-a),&a\leq x\leq b,\\ 0,&\text{otherwise}. \end{cases}$$

Notes: uniform distributions · R manual

Density and probability under $\operatorname{Uniform}(0,1)$

For $X\sim\operatorname{Uniform}(0,1)$,

$$\mathbb P(.25<X<.75) =\int_{.25}^{.75}1\,\mathrm dx =.50.$$

Probability is area under the density

Equal density height means equal probability for intervals of equal length, not equal positive probability at every point.

CDF of $\operatorname{Uniform}(0,1)$

Linguistic use: randomization by design

Let $U\sim\operatorname{Uniform}(0,1)$ be a computer-generated draw used to assign a trial to one of two lists:

$$A(U)= \begin{cases} \text{list 1},&U<.5,\\ \text{list 2},&U\geq.5. \end{cases}$$

Here uniformity describes the randomization mechanism. It does not assert that an acoustic measurement is naturally uniform.

Question: how can a distribution vary within $(0,1)$?

Proportions and bounded continuous responses may cluster near the center, near one boundary, or near both boundaries.

The beta family can represent these contrasting shapes through two positive parameters.

The beta distribution

For $\alpha,\beta>0$, the beta function supplies the normalizing area

$$B(\alpha,\beta) \equiv\int_0^1 t^{\alpha-1}(1-t)^{\beta-1}\,\mathrm dt.$$

Then

$$X\sim\operatorname{Beta}(\alpha,\beta)$$

has possible values in $[0,1]$ and density

$$f_X(x)\equiv \begin{cases} \dfrac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)},&0<x<1,\\ 0,&\text{otherwise}. \end{cases}$$

Notes: beta distributions · R manual

$\operatorname{Beta}(1,1)$ is uniform on $(0,1)$

Symmetric beta densities with $\alpha=\beta>1$

Asymmetric beta densities

Boundary modes in the beta family

Two boundary modes when $\alpha,\beta<1$

Transforming beta support to $(a,b)$

If $X\sim\operatorname{Beta}(\alpha,\beta)$ and

$$Y\equiv a+(b-a)X,$$

then, for $a<y<b$,

$$f_Y(y) \equiv \frac{1}{b-a} \frac{ \left(\frac{y-a}{b-a}\right)^{\alpha-1} \left(1-\frac{y-a}{b-a}\right)^{\beta-1} }{B(\alpha,\beta)}.$$

The change-of-scale factor $1/(b-a)$ is required for the transformed density to integrate to one.

Interpreting beta parameters

For $X\sim\operatorname{Beta}(\alpha,\beta)$,

$$\begin{aligned} \mathbb E[X] &=\frac{1}{B(\alpha,\beta)} \int_0^1x^\alpha(1-x)^{\beta-1}\,\mathrm dx\\ &=\frac{B(\alpha+1,\beta)}{B(\alpha,\beta)}\\ &=\frac{\alpha}{\alpha+\beta}. \end{aligned}$$

Holding this ratio fixed while increasing $\alpha+\beta$ concentrates the distribution around the same mean.

$\alpha/(\alpha+\beta)$ controls the mean, while $\alpha+\beta$ controls concentration when the mean is held fixed.

Linguistic use: a bounded continuous response

Suppose $A$ is a participant’s sentence-acceptability rating, recorded with a continuous slider and rescaled to $(0,1)$.

A beta model treats $A$ as a measurement, not as a grammatical category. If the task permits exact endpoint responses 0 and 1, an ordinary beta distribution is not sufficient without an endpoint-handling component.

Question: how can location and scale vary independently?

The normal family uses one parameter for its center and another for its spread over the real line.

This support assumption matters: an untransformed duration or proportion cannot literally take every real value.

The normal distribution

For $\sigma>0$,

$$X\sim\mathcal N(\mu,\sigma^2)$$

means

$$f_X(x)\equiv\frac{1}{\sigma\sqrt{2\pi}} \exp\!\left[-\frac{(x-\mu)^2}{2\sigma^2}\right].$$

Notes: normal distributions · R manual

PDF of the standard normal distribution

The mean controls location; the standard deviation controls scale

Standardizing a normal variable

Standardization expresses a value’s distance from the model mean in standard-deviation units.

Define

$$Z\equiv\frac{X-\mu}{\sigma}.$$

If $X\sim\mathcal N(\mu,\sigma^2)$, then $Z\sim\mathcal N(0,1)$. Standardization removes the physical unit and retains relative position within this family.

A standardized value reports position in standard-deviation units under the assumed normal model.

A normal interval by hand

For $X\sim\mathcal N(550,40^2)$,

$$\frac{510-550}{40}=-1, \qquad \frac{590-550}{40}=1.$$

Thus $\mathbb P(510\leq X\leq590)\approx.683$ under this normal model.

Linguistic use: log duration

Let $D>0$ be a token duration and $Y\equiv\log D$. A normal model for $Y$ states

$$Y\sim\mathcal N(\mu,\sigma^2).$$

This is a distributional assumption about a transformed acoustic measurement. It is not implied by the fact that a duration histogram looks roughly bell-shaped.

Standard normal CDF

A beta CDF can also be S-shaped

Question: what is the distribution of squared normal discrepancies?

Squaring standard-normal variables makes every contribution nonnegative. Adding $\nu$ independent squared values yields a chi-squared variable.

The count $\nu$ is called its degrees of freedom: it records the number of independent squared contributions.

The chi-squared distribution

For $\nu\in\{1,2,\ldots\}$, let $Z_1,\ldots,Z_\nu$ be independent standard normal variables and define

$$Q\equiv\sum_{i=1}^{\nu}Z_i^2.$$

Then $Q\sim\chi^2_\nu$, with $\mathbb E[Q]=\nu$ and $\operatorname{Var}(Q)=2\nu$.

Notes: chi-squared distributions

Linguistic use: an aggregate discrepancy

Suppose $Z_1,Z_2,Z_3$ are independent standardized discrepancies for three predeclared acoustic measurements. Then

$$Q=Z_1^2+Z_2^2+Z_3^2\sim\chi^2_3$$

under the standard-normal assumptions. The degrees of freedom count independent squared contributions, not speech tokens.

A chi-squared discrepancy by hand

For $Z_1=1$, $Z_2=-.5$, and $Z_3=2$,

$$Q=1^2+(-.5)^2+2^2=5.25.$$

Under the independent standard-normal construction, $Q\sim\chi^2_3$ and $\mathbb P(Q\geq5.25)\approx.154$.

Chi-squared center and spread

For $Q\sim\chi^2_\nu$,

$$\mathbb E[Q]=\nu, \qquad \operatorname{Var}(Q)=2\nu.$$

Degrees of freedom determine both center and spread.

An upper-tail probability

For the constructed value $Q=5.25$ with $\nu=3$,

$$\mathbb P(Q\geq5.25)\approx.154.$$

This is probability under the reference distribution, not the probability that a hypothesis is true.

Degrees of freedom count unconstrained variation

In an $r$ by $c$ table with fixed row and column totals,

$$\nu=(r-1)(c-1).$$

The symbol $\nu$ counts independent pieces of variation remaining after constraints, not necessarily observations.

What changes when scale is estimated?

Dividing a standard-normal variable by an independent estimated scale adds uncertainty. The resulting Student’s t distribution has heavier tails, that is, it places more probability far from zero.

Student’s t distribution

Let $Z\sim\mathcal N(0,1)$ and $U\sim\chi^2_\nu$ be independent. Define

$$T\equiv\frac{Z}{\sqrt{U/\nu}}.$$

Then $T\sim t_\nu$. Because $U/\nu$ can be close to zero, extreme values of $T$ are more probable than extreme values under a standard normal distribution.

Notes: Student’s t distributions · R manual

A two-sided t probability

For $|t|=2.5$ and $\nu=5$,

$$\mathbb P(|T|\geq2.5) =2\mathbb P(T\geq2.5) \approx.0545.$$

The corresponding standard-normal probability is about $.0124$ because $t_5$ has heavier tails.

Linguistic use: keep two roles separate

An estimator is a rule that maps a sample to an estimate of an unknown parameter. A $t$ distribution may describe a standardized estimator whose scale was estimated.

It may instead be chosen as an observation model, that is, a distribution for a transformed linguistic response.

Those are different claims. Naming the random variable and its generating construction keeps them apart.

How can a CDF generate draws?

A CDF partitions probability from zero to one. A uniform draw can select one of those probability intervals.

This is the intuition behind inverse transform sampling.

Inverse transform sampling

For inverse transform sampling, the generalized inverse returns the leftmost value at which the CDF reaches $u$. Formally,

$$F^{-1}(u) \equiv\inf\{x\in\mathbb R:F(x)\geq u\}, \qquad 0<u<1.$$

If $U\sim\operatorname{Uniform}(0,1)$, define $X\equiv F^{-1}(U)$.

Notes: inverse transform sampling

Verifying the target distribution

The generalized inverse satisfies $F^{-1}(U)\leq x$ exactly when $U\leq F(x)$. Thus

$$\begin{aligned} \mathbb{P}(X\leq x) &=\mathbb{P}(F^{-1}(U)\leq x)\\ &=\mathbb{P}(U\leq F(x))\\ &=F(x). \end{aligned}$$

Discrete inverse transform sampling

For ordered values $x_1,\ldots,x_K$, define

$$c_k\equiv\sum_{j=1}^kp_X(x_j).$$

Draw $U\sim\operatorname{Uniform}(0,1)$ and return the first $x_k$ for which $U\leq c_k$.

Cumulative mass turns one uniform value into one draw from the target PMF.

Cumulative cutpoints

Work a categorical draw

Suppose a constructed three-label PMF assigns $(.50,.30,.20)$ to (Nom, Acc, Gen). Its cumulative cutpoints are $(.50,.80,1)$.

If $U=.73$, return Acc, since $.50<U\leq.80$.

The returned value is an annotated category label, not a spelling or pronunciation.

Checking the implemented cutpoints

The endpoints must be the cumulative sums $(.50,.80,1)$.

If the first endpoint were $.40$, the first label would be simulated too rarely. Large-sample proportions would expose the error.

What does a random seed guarantee?

A pseudorandom algorithm is deterministic once its initial state is fixed.

A random seed records that initial state.

Random seeds reproduce computations

A random seed fixes the initial state of a pseudorandom number generator. Reusing the seed reproduces the same sequence of draws.

It does not validate the distribution, independence assumptions, or sampling representation.

Notes: random seeds · R manual: Random

One seed, one computational sequence

Set the seed once before a related block of random draws. Resetting the same seed inside a loop repeats the same simulated result.

Record the seed with the code and place it before the first draw that affects the reported result.

Stability across seeds

Changing the seed provides a check on Monte Carlo variation. A conclusion that changes materially across seeds may require more draws or an implementation check.

More draws reduce simulation variation. They do not repair the wrong target distribution.

Reproducibility and validity

A fixed seed can reproduce an inadequate simulation exactly.

Reproducibility asks whether another analyst can regenerate the computation. Adequacy asks whether the computation represents the intended linguistic process.

Question: which values occurred together?

Separate one-variable distributions lose the pairing among attributes of the same outcome.

A joint distribution retains that pairing.

Several variables use one probability space

Let $X:\Omega\rightarrow\mathcal X$ and $Y:\Omega\rightarrow\mathcal Y$ be random variables on the same probability space.

The ordered pair $\langle X(\omega),Y(\omega)\rangle$ records two values from the same outcome $\omega$.

This is why two variables on one token do not constitute two independent observations.

Defining a joint PMF

For discrete $X$ and $Y$, define

$$p_{X,Y}(x,y) \equiv\mathbb{P}\!\left(\{\omega\in\Omega\mid X(\omega)=x,\ Y(\omega)=y\}\right).$$

After this full definition, $\mathbb P(X=x,Y=y)$ abbreviates the same joint event.

Notes: joint distributions · Module 1: joint events

Declare the annotations before reading the table

Let the outcome be an annotated dialogue-turn record. Let $P$ code whether the turn contains a marker from a predeclared politeness-marker inventory, and let $R$ code whether its measured fundamental-frequency trajectory, or $F_0$ contour, meets a predeclared final-rise criterion.

These are binary annotations, not claims that politeness or intonation is intrinsically binary.

Praat: pitch analysis produces a measured pitch curve

A joint distribution of two annotations

Reading a joint cell

The upper-left cell states

$$p_{P,R}(1,1) =.30.$$

It assigns probability to the paired values for one dialogue turn.

The cell concerns co-occurrence within one turn. It does not by itself identify a causal direction.

Marginal PMFs do not recover pairing

Separate PMFs for $X$ and $Y$ do not determine which values occur together.

Two joint tables can share the same margins while assigning different probability to the pairs.

The continuous joint distribution

A joint density $f_{X,Y}$ satisfies

$$\mathbb P\!\left(\{\omega\in\Omega\mid \langle X(\omega),Y(\omega)\rangle\in A\}\right) =\iint_A f_{X,Y}(x,y)\,\mathrm dx\,\mathrm dy.$$

Probability is area or volume under the joint density over the selected region.

Question: what remains if one coordinate is ignored?

To recover the distribution of $P$ alone, add across every possible value of $R$.

This operation is marginalization.

What is a partition?

A partition splits an event into disjoint pieces whose union recovers the event.

For marginalization, the pieces hold one value fixed and let the value being removed vary across its codomain.

Marginalization is a partition argument

For fixed $p$, the following value events partition the event for $P=p$ as $r$ ranges over $\operatorname{cod}(R)$:

$$\{\omega\in\Omega\mid P(\omega)=p\} =\biguplus_{r\in\operatorname{cod}(R)} \{\omega\in\Omega\mid P(\omega)=p,\ R(\omega)=r\}.$$

Countable additivity gives

$$\mathbb{P}(P=p) =\sum_{r\in\operatorname{cod}(R)}\mathbb{P}(P=p,R=r).$$

Applying the PMF definitions to both sides yields

$$p_P(p) =\sum_{r\in\operatorname{cod}(R)} p_{P,R}(p,r).$$

Computing a marginal PMF

For turns with a politeness marker,

$$\begin{aligned} p_P(1) &=p_{P,R}(1,1)+p_{P,R}(1,0)\\ &=.30+.20\\ &=.50. \end{aligned}$$

We sum out pitch rise while holding politeness fixed.

Notes: marginal distributions

What marginalization removes

The marginal PMFs $p_P$ and $p_R$ retain the row and column totals. They do not retain how probability is paired within the four cells.

Different joint PMFs can have the same marginals.

Marginalization preserves one coordinate’s probabilities and discards its pairing with the other coordinate.

The continuous version

For two continuous measurements, probability is volume under a surface rather than mass in separate cells.

Defining a joint density

Absolutely continuous $X$ and $Y$ have joint density $f_{X,Y}$ when, for every measurable region $A\subseteq\mathbb R^2$,

$$\mathbb P\bigl(\langle X,Y\rangle\in A\bigr) =\iint_A f_{X,Y}(x,y)\,\mathrm dx\,\mathrm dy.$$

The density is nonnegative and integrates to one over $\mathbb R^2$.

Marginal density

If $X$ and $Y$ have joint density $f_{X,Y}$, define the marginal density of $X$ by

$$f_X(x) \equiv\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,\mathrm{d}y.$$