Joint distributions

The foundations chapter introduced the joint probability of two events. A joint distribution is the distribution of a random vector such as (P,R). Suppose each dialogue turn is annotated for two properties. Let P record whether the turn contains a politeness marker, and let R record whether it ends in a final pitch rise:

P(\omega)\equiv \begin{cases} 1,&\text{if turn }\omega\text{ contains a politeness marker},\\ 0,&\text{otherwise}, \end{cases}

and

R(\omega)\equiv \begin{cases} 1,&\text{if turn }\omega\text{ ends in a final pitch rise},\\ 0,&\text{otherwise}. \end{cases}

One sampled turn produces a pair (P,R). The possible pairs are

(0,0),\ (0,1),\ (1,0),\ (1,1).

For discrete P and R, define their joint probability mass function by

p_{P,R}(p,r) \equiv\mathbb{P}(P=p,R=r).

Reading a joint PMF table

Consider this constructed joint PMF.

	no rise R=0	rise R=1
no marker P=0	.25	.15
marker P=1	.20	.40

The lower right cell says

p_{P,R}(1,1) =.40.

It is the probability that one turn contains both a politeness marker and a final rise.

The upper right cell says

p_{P,R}(0,1)=.15,

the probability of a final rise without a politeness marker.

The subscript on p_{P,R} identifies the two variables in the joint PMF. The comma inside the probability event means that both value statements hold for the same outcome.

Checking the joint PMF requirements

Every cell is nonnegative. The cells also sum to one:

.25+.15+.20+.40=1.

The four value pairs are mutually exclusive and exhaustive under the two binary annotations. One turn belongs to exactly one cell.

A single row or column does not need to sum to one. It represents only part of the complete pair space. The whole table is the PMF.

Asking about a set of pairs

The event that at least one annotation is present contains three pairs:

\{(0,1),(1,0),(1,1)\}.

Its probability is

\begin{aligned} \mathbb{P}(P=1\text{ or }R=1) &=.15+.20+.40\\ &=.75. \end{aligned}

The remaining pair (0,0) has probability .25, so the complement check gives 1-.25=.75.

Storing and inspecting the table in base R

Code

joint_pr <- matrix(
  c(.25, .20, .15, .40),
  nrow = 2,
  dimnames = list(
    politeness = c("absent", "present"),
    rise = c("absent", "present")
  )
)

joint_pr
sum(joint_pr)
joint_pr["present", "present"]

R fills matrices by columns, so the vector order must match the declared row and column labels. The final indexing call checks the linguistic interpretation of one cell.

Why separate PMFs do not recover pairing

Suppose we know how often politeness markers occur and how often final rises occur. Those two separate summaries do not tell us which turns contain both.

Many joint tables can have the same separate probabilities while placing different amounts of mass on (1,1). The pairing information lives in the joint distribution.

An association between final rises and politeness markers concerns their pairing within turns. Separate rise and politeness frequencies cannot show whether the two annotations occur on the same turns. The dataset must retain the turn identifier that pairs the measurements.

The continuous version

For absolutely continuous X and Y, define their joint density f_{X,Y} by the requirement

\mathbb{P}((X,Y)\in A) =\iint_A f_{X,Y}(x,y)\,\mathrm{d}x\,\mathrm{d}y

for every measurable region A\subseteq\mathbb R^2. A point (x,y) has probability zero.

The discrete and continuous representations both preserve paired values from the same outcome.

Check your understanding

Translate p_{P,R}(1,0) into a linguistic event and read its probability from the table.
Compute the probability that exactly one annotation is present.
Why can a row sum be less than one in a valid joint PMF?
Explain how shuffling one annotation changes the joint distribution without changing either separate frequency table.

The joint table retains both variables. The next page removes one distinction by adding over all of its values.

--- title: "Joint distributions" --- The foundations chapter introduced the [joint probability of two events](../foundations/joint-probability-of-events.qmd). A [**joint distribution**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) is the distribution of a random vector such as $(P,R)$. Suppose each dialogue turn is annotated for two properties. Let $P$ record whether the turn contains a politeness marker, and let $R$ record whether it ends in a final pitch rise: $$ P(\omega)\equiv \begin{cases} 1,&\text{if turn }\omega\text{ contains a politeness marker},\\ 0,&\text{otherwise}, \end{cases} $$ and $$ R(\omega)\equiv \begin{cases} 1,&\text{if turn }\omega\text{ ends in a final pitch rise},\\ 0,&\text{otherwise}. \end{cases} $$ One sampled turn produces a pair $(P,R)$. The possible pairs are $$ (0,0),\ (0,1),\ (1,0),\ (1,1). $$ For discrete $P$ and $R$, define their joint probability mass function by $$ p_{P,R}(p,r) \equiv\mathbb{P}(P=p,R=r). $$ ## Reading a joint PMF table Consider this constructed joint PMF. | | no rise $R=0$ | rise $R=1$ | |---|---:|---:| | no marker $P=0$ | $.25$ | $.15$ | | marker $P=1$ | $.20$ | $.40$ | The lower right cell says $$ p_{P,R}(1,1) =.40. $$ It is the probability that one turn contains both a politeness marker and a final rise. The upper right cell says $$ p_{P,R}(0,1)=.15, $$ the probability of a final rise without a politeness marker. The subscript on $p_{P,R}$ identifies the two variables in the joint PMF. The comma inside the probability event means that both value statements hold for the same outcome. ## Checking the joint PMF requirements Every cell is nonnegative. The cells also sum to one: $$ .25+.15+.20+.40=1. $$ The four value pairs are mutually exclusive and exhaustive under the two binary annotations. One turn belongs to exactly one cell. A single row or column does not need to sum to one. It represents only part of the complete pair space. The whole table is the PMF. ## Asking about a set of pairs The event that at least one annotation is present contains three pairs: $$ \{(0,1),(1,0),(1,1)\}. $$ Its probability is $$ \begin{aligned} \mathbb{P}(P=1\text{ or }R=1) &=.15+.20+.40\\ &=.75. \end{aligned} $$ The remaining pair $(0,0)$ has probability $.25$, so the complement check gives $1-.25=.75$. ## Storing and inspecting the table in base R ```{r} #| eval: false joint_pr <- matrix( c(.25, .20, .15, .40), nrow = 2, dimnames = list( politeness = c("absent", "present"), rise = c("absent", "present") ) ) joint_pr sum(joint_pr) joint_pr["present", "present"] ``` R fills matrices by columns, so the vector order must match the declared row and column labels. The final indexing call checks the linguistic interpretation of one cell. ## Why separate PMFs do not recover pairing Suppose we know how often politeness markers occur and how often final rises occur. Those two separate summaries do not tell us which turns contain both. Many joint tables can have the same separate probabilities while placing different amounts of mass on $(1,1)$. The pairing information lives in the joint distribution. An association between final rises and politeness markers concerns their pairing within turns. Separate rise and politeness frequencies cannot show whether the two annotations occur on the same turns. The dataset must retain the turn identifier that pairs the measurements. ## The continuous version For absolutely continuous $X$ and $Y$, define their joint density $f_{X,Y}$ by the requirement $$ \mathbb{P}((X,Y)\in A) =\iint_A f_{X,Y}(x,y)\,\mathrm{d}x\,\mathrm{d}y $$ for every measurable region $A\subseteq\mathbb R^2$. A point $(x,y)$ has probability zero. The discrete and continuous representations both preserve paired values from the same outcome. ## Check your understanding 1. Translate $p_{P,R}(1,0)$ into a linguistic event and read its probability from the table. 2. Compute the probability that exactly one annotation is present. 3. Why can a row sum be less than one in a valid joint PMF? 4. Explain how shuffling one annotation changes the joint distribution without changing either separate frequency table. The joint table retains both variables. The [next page](marginal-distributions.qmd) removes one distinction by adding over all of its values.