Marginal distributions

The preceding page defined a distribution over paired values. Suppose a morphological analysis records tense T and number N for one sampled verb token.

	singular N=\mathrm{sg}	plural N=\mathrm{pl}
past T=\mathrm{past}	.30	.20
present T=\mathrm{pres}	.15	.35

For a fixed tense value t, the events \{T=t,N=n\} are disjoint as n varies, and

\{T=t\} =\biguplus_{n\in\operatorname{supp}(N)} \{T=t,N=n\}.

Countable additivity thus gives

\mathbb{P}(T=t) =\sum_{n\in\operatorname{supp}(N)} \mathbb{P}(T=t,N=n).

The PMFs are defined by

p_T(t)\equiv\mathbb{P}(T=t)

and

p_{T,N}(t,n)\equiv\mathbb{P}(T=t,N=n).

Substitution yields the marginalization identity

p_T(t) =\sum_{n\in\operatorname{supp}(N)}p_{T,N}(t,n).

This derived one-variable PMF specifies the marginal distribution of T.

Summing one row at a time

The past tense mass is

\begin{aligned} p_T(\mathrm{past}) &=p_{T,N}(\mathrm{past},\mathrm{sg}) +p_{T,N}(\mathrm{past},\mathrm{pl})\\ &=.30+.20\\ &=.50. \end{aligned}

The present tense mass is

p_T(\mathrm{pres})=.15+.35=.50.

Thus the marginal tense PMF is (.50,.50).

The same operation down columns gives the marginal number PMF:

p_N(\mathrm{sg})=.30+.15=.45

and

p_N(\mathrm{pl})=.20+.35=.55.

Adding the margins to the table

	singular	plural	p_T(t)
past	.30	.20	.50
present	.15	.35	.50
p_N(n)	.45	.55	1

Marginalizing a joint density

For variables with joint density f_{X,Y}, define the marginal density of X by

f_X(x) \equiv\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,\mathrm{d}y.

The integral ranges over the complete Y support while x remains fixed. For every measurable set A,

\begin{aligned} \mathbb{P}(X\in A) &=\int_A f_X(x)\,\mathrm{d}x\\ &=\int_A\!\left[ \int_{-\infty}^{\infty}f_{X,Y}(x,y)\,\mathrm{d}y \right]\mathrm{d}x. \end{aligned}

The inner integral sums out Y; the outer integral assigns probability to the event \{X\in A\}.

Computing the margins in base R

Code

joint_tense_number <- matrix(
  c(.30, .15, .20, .35),
  nrow = 2,
  dimnames = list(
    tense = c("past", "present"),
    number = c("singular", "plural")
  )
)

rowSums(joint_tense_number)
colSums(joint_tense_number)

The row sums give the tense margin; the column sums give the number margin.

What marginalization changes

Marginalizing number gives the tense distribution without conditioning on a particular number value. It does not assert that tense and number are independent.

In the joint table, plural tokens are more often present than past, while singular tokens are more often past than present. Those pairings disappear from the tense margin (.50,.50).

The marginal distribution is determined by the joint distribution, but the converse is false. The two one-variable marginals do not determine the cell probabilities or the association between T and N.

Check your understanding

Compute p_N(\mathrm{pl}) from the joint cells.
Verify that each marginal PMF sums to one.
Which joint information is lost when number is marginalized?
State the partition of \{T=t\} that licenses the marginalization sum.

Marginalization removes a distinction. The next page asks whether two variables remain associated after a third distinction is held fixed.

--- title: "Marginal distributions" --- The [preceding page](joint-probability.qmd) defined a distribution over paired values. Suppose a morphological analysis records tense $T$ and number $N$ for one sampled verb token. | | singular $N=\mathrm{sg}$ | plural $N=\mathrm{pl}$ | |---|---:|---:| | past $T=\mathrm{past}$ | $.30$ | $.20$ | | present $T=\mathrm{pres}$ | $.15$ | $.35$ | For a fixed tense value $t$, the events $\{T=t,N=n\}$ are disjoint as $n$ varies, and $$ \{T=t\} =\biguplus_{n\in\operatorname{supp}(N)} \{T=t,N=n\}. $$ Countable additivity thus gives $$ \mathbb{P}(T=t) =\sum_{n\in\operatorname{supp}(N)} \mathbb{P}(T=t,N=n). $$ The PMFs are defined by $$ p_T(t)\equiv\mathbb{P}(T=t) $$ and $$ p_{T,N}(t,n)\equiv\mathbb{P}(T=t,N=n). $$ Substitution yields the marginalization identity $$ p_T(t) =\sum_{n\in\operatorname{supp}(N)}p_{T,N}(t,n). $$ This derived one-variable PMF specifies the [**marginal distribution**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) of $T$. ## Summing one row at a time The past tense mass is $$ \begin{aligned} p_T(\mathrm{past}) &=p_{T,N}(\mathrm{past},\mathrm{sg}) +p_{T,N}(\mathrm{past},\mathrm{pl})\\ &=.30+.20\\ &=.50. \end{aligned} $$ The present tense mass is $$ p_T(\mathrm{pres})=.15+.35=.50. $$ Thus the marginal tense PMF is $(.50,.50)$. The same operation down columns gives the marginal number PMF: $$ p_N(\mathrm{sg})=.30+.15=.45 $$ and $$ p_N(\mathrm{pl})=.20+.35=.55. $$ ## Adding the margins to the table | | singular | plural | $p_T(t)$ | |---|---:|---:|---:| | past | $.30$ | $.20$ | $.50$ | | present | $.15$ | $.35$ | $.50$ | | $p_N(n)$ | $.45$ | $.55$ | $1$ | ## Marginalizing a joint density For variables with joint density $f_{X,Y}$, define the marginal density of $X$ by $$ f_X(x) \equiv\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,\mathrm{d}y. $$ The integral ranges over the complete $Y$ support while $x$ remains fixed. For every measurable set $A$, $$ \begin{aligned} \mathbb{P}(X\in A) &=\int_A f_X(x)\,\mathrm{d}x\\ &=\int_A\!\left[ \int_{-\infty}^{\infty}f_{X,Y}(x,y)\,\mathrm{d}y \right]\mathrm{d}x. \end{aligned} $$ The inner integral sums out $Y$; the outer integral assigns probability to the event $\{X\in A\}$. ## Computing the margins in base R ```{r} #| eval: false joint_tense_number <- matrix( c(.30, .15, .20, .35), nrow = 2, dimnames = list( tense = c("past", "present"), number = c("singular", "plural") ) ) rowSums(joint_tense_number) colSums(joint_tense_number) ``` The row sums give the tense margin; the column sums give the number margin. ## What marginalization changes Marginalizing number gives the tense distribution without conditioning on a particular number value. It does not assert that tense and number are independent. In the joint table, plural tokens are more often present than past, while singular tokens are more often past than present. Those pairings disappear from the tense margin $(.50,.50)$. The marginal distribution is determined by the joint distribution, but the converse is false. The two one-variable marginals do not determine the cell probabilities or the association between $T$ and $N$. ## Check your understanding 1. Compute $p_N(\mathrm{pl})$ from the joint cells. 2. Verify that each marginal PMF sums to one. 3. Which joint information is lost when number is marginalized? 4. State the partition of $\{T=t\}$ that licenses the marginalization sum. Marginalization removes a distinction. The [next page](conditional-independence.qmd) asks whether two variables remain associated after a third distinction is held fixed.