Joint probability of events

Select one case-coded pronoun form from the population represented by the probability space below. We want the probability that the selected form is both plural and accusative. Which event must receive that probability? It is the set of outcomes shared by the plural and accusative events.

In the original fourteen-outcome uniform example, the third-person accusative event is

T_{14}\cap A_{14} =\{\textit{them},\textit{it}_{[+\mathrm{acc}]}, \textit{her},\textit{him}\}.

Because this “both” construction recurs, probability notation gives it a compressed form. The standard comma notation is defined by

\mathbb P(T_{14},A_{14}) \equiv\mathbb P(T_{14}\cap A_{14}).

Under the uniform probability measure,

\mathbb P(T_{14},A_{14})=\frac{4}{14}=\frac{2}{7}.

The reduced four-form calculation below uses the same definition with nonuniform singleton masses.

Return to the four form sample space

\Omega_4\equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}

with the probability measure below.

outcome	probability
he	.20
him	.30
they	.10
them	.40

Let P_4 be the plural event and A_4 the accusative event:

P_4\equiv\{\textit{they},\textit{them}\}

and

A_4\equiv\{\textit{him},\textit{them}\}.

Joint-probability notation

Basically, a joint probability assigns probability to outcomes for which both events occur. More specifically, the joint probability of events B and C is the probability of their intersection. We define the comma notation by

\mathbb{P}(B,C) \equiv \mathbb{P}(B\cap C).

The expression \mathbb{P}(B,C) is read as “the joint probability of B and C.” The comma means that both events occur. It is syntactic shorthand, not a new set operation: B\cap C remains the underlying event.

The joint probability

Begin with the set operation:

P_4\cap A_4=\{\textit{them}\}.

Then apply the probability measure:

\begin{aligned} \mathbb{P}(P_4,A_4) &=\mathbb{P}(\{\textit{them}\})\\ &=.40. \end{aligned}

The value \mathbb{P}(P_4,A_4)=.40 is relative to the full sample space. The conditional probability of accusative case among plural forms has a different denominator and will be defined on the next page.

Arrange all joint events in a table

The number distinction P_4 versus P_4^c and the case distinction A_4 versus A_4^c divide the sample space into four disjoint intersections.

	accusative A_4	nominative A_4^c	total
plural P_4	\mathbb{P}(P_4,A_4)=.40	\mathbb{P}(P_4,A_4^c)=.10	.50
singular P_4^c	\mathbb{P}(P_4^c,A_4)=.30	\mathbb{P}(P_4^c,A_4^c)=.20	.50
total	.70	.30	1

For instance, the upper right cell represents the event

P_4\cap A_4^c=\{\textit{they}\},

so \mathbb{P}(P_4,A_4^c)=.10. The lower left cell represents

P_4^c\cap A_4=\{\textit{him}\},

so \mathbb{P}(P_4^c,A_4)=.30.

The four interior cells are mutually exclusive and exhaustive. Their probabilities must sum to one:

.40+.10+.30+.20=1.

Marginal probabilities

Now, you might wonder how to recover the probability of plurality without keeping track of case. We add across the mutually exclusive and exhaustive case alternatives. This operation is marginalization.

The row and column totals are marginal probabilities. To obtain the marginal probability of P_4, partition the sample space by A_4 and A_4^c and add the corresponding joint probabilities:

\begin{aligned} \mathbb{P}(P_4) &=\mathbb{P}(P_4,A_4)+\mathbb{P}(P_4,A_4^c)\\ &=.40+.10\\ &=.50. \end{aligned}

This equality follows from countable additivity because P_4\cap A_4 and P_4\cap A_4^c are disjoint and their union is P_4. Marginalizing over case means summing over the exhaustive case alternatives while holding the number event fixed.

Reconstruct the table in base R

The following code stores the four joint probabilities in a matrix and computes its margins.

Code

joint <- matrix(
  c(.40, .30, .10, .20),
  nrow = 2,
  dimnames = list(
    number = c("plural", "singular"),
    case = c("accusative", "nominative")
  )
)

joint
rowSums(joint)
colSums(joint)
sum(joint)

The row sums are .50 and .50, the column sums are .70 and .30, and the four cells sum to one. These are the margins and total shown in the table.

Information in the joint-probability table

A joint table records the probabilities of the four number-by-case combinations. Two probability measures can both have \mathbb{P}(P_4)=.50 and \mathbb{P}(A_4)=.70 while assigning different values to \mathbb{P}(P_4,A_4). The two marginal probabilities do not determine the four joint probabilities.

For instance, an association between plurality and accusative case is represented by a difference between \mathbb{P}(A_4\mid P_4) and \mathbb{P}(A_4\mid P_4^c). The two margins do not determine those conditional probabilities.

Marginal and joint probabilities

In the toy model, \mathbb{P}(A_4)=.70 includes both him and them. Only the .40 assigned to them belongs to the event P_4\cap A_4, so \mathbb{P}(P_4,A_4)=.40.

To compute a joint probability, identify the intersection denoted by the comma and apply the probability measure to that event.

The exact upshot is that a joint probability uses the full sample space as its reference and assigns mass to an intersection. The next page changes that reference by restricting attention to one event.

Check your understanding

Write the event represented by each of the four interior cells in the table.
Compute \mathbb{P}(P_4^c,A_4^c) by identifying the corresponding event first.
Explain why \mathbb{P}(P_4,A_4)=.40 does not mean that 40\% of plural forms are accusative.
Construct a different four-cell table with the same row and column margins but with \mathbb{P}(P_4,A_4)=.35.

--- title: "Joint probability of events" --- Select one case-coded pronoun form from the population represented by the probability space below. We want the probability that the selected form is both plural and accusative. Which event must receive that probability? It is the set of outcomes shared by the plural and accusative events. In the original fourteen-outcome uniform example, the third-person accusative event is $$ T_{14}\cap A_{14} =\{\textit{them},\textit{it}_{[+\mathrm{acc}]}, \textit{her},\textit{him}\}. $$ Because this “both” construction recurs, probability notation gives it a compressed form. The standard comma notation is defined by $$ \mathbb P(T_{14},A_{14}) \equiv\mathbb P(T_{14}\cap A_{14}). $$ Under the uniform probability measure, $$ \mathbb P(T_{14},A_{14})=\frac{4}{14}=\frac{2}{7}. $$ The reduced four-form calculation below uses the same definition with nonuniform singleton masses. Return to the four form sample space $$ \Omega_4\equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\} $$ with the probability measure below. | outcome | probability | |---|---:| | *he* | $.20$ | | *him* | $.30$ | | *they* | $.10$ | | *them* | $.40$ | Let $P_4$ be the plural event and $A_4$ the accusative event: $$ P_4\equiv\{\textit{they},\textit{them}\} $$ and $$ A_4\equiv\{\textit{him},\textit{them}\}. $$ ## Joint-probability notation Basically, a joint probability assigns probability to outcomes for which both events occur. More specifically, the [**joint probability**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) of events $B$ and $C$ is the probability of their intersection. We define the comma notation by $$ \mathbb{P}(B,C) \equiv \mathbb{P}(B\cap C). $$ The expression $\mathbb{P}(B,C)$ is read as “the joint probability of $B$ and $C$.” The comma means that both events occur. It is syntactic shorthand, not a new set operation: $B\cap C$ remains the underlying event. ## The joint probability Begin with the set operation: $$ P_4\cap A_4=\{\textit{them}\}. $$ Then apply the probability measure: $$ \begin{aligned} \mathbb{P}(P_4,A_4) &=\mathbb{P}(\{\textit{them}\})\\ &=.40. \end{aligned} $$ The value $\mathbb{P}(P_4,A_4)=.40$ is relative to the full sample space. The conditional probability of accusative case among plural forms has a different denominator and will be defined on the next page. ## Arrange all joint events in a table The number distinction $P_4$ versus $P_4^c$ and the case distinction $A_4$ versus $A_4^c$ divide the sample space into four disjoint intersections. | | accusative $A_4$ | nominative $A_4^c$ | total | |---|---:|---:|---:| | plural $P_4$ | $\mathbb{P}(P_4,A_4)=.40$ | $\mathbb{P}(P_4,A_4^c)=.10$ | $.50$ | | singular $P_4^c$ | $\mathbb{P}(P_4^c,A_4)=.30$ | $\mathbb{P}(P_4^c,A_4^c)=.20$ | $.50$ | | total | $.70$ | $.30$ | $1$ | For instance, the upper right cell represents the event $$ P_4\cap A_4^c=\{\textit{they}\}, $$ so $\mathbb{P}(P_4,A_4^c)=.10$. The lower left cell represents $$ P_4^c\cap A_4=\{\textit{him}\}, $$ so $\mathbb{P}(P_4^c,A_4)=.30$. The four interior cells are mutually exclusive and exhaustive. Their probabilities must sum to one: $$ .40+.10+.30+.20=1. $$ ## Marginal probabilities Now, you might wonder how to recover the probability of plurality without keeping track of case. We add across the mutually exclusive and exhaustive case alternatives. This operation is [**marginalization**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html). The row and column totals are [**marginal probabilities**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html). To obtain the marginal probability of $P_4$, partition the sample space by $A_4$ and $A_4^c$ and add the corresponding joint probabilities: $$ \begin{aligned} \mathbb{P}(P_4) &=\mathbb{P}(P_4,A_4)+\mathbb{P}(P_4,A_4^c)\\ &=.40+.10\\ &=.50. \end{aligned} $$ This equality follows from countable additivity because $P_4\cap A_4$ and $P_4\cap A_4^c$ are disjoint and their union is $P_4$. Marginalizing over case means summing over the exhaustive case alternatives while holding the number event fixed. ## Reconstruct the table in base R The following code stores the four joint probabilities in a matrix and computes its margins. ```{r} #| eval: false joint <- matrix( c(.40, .30, .10, .20), nrow = 2, dimnames = list( number = c("plural", "singular"), case = c("accusative", "nominative") ) ) joint rowSums(joint) colSums(joint) sum(joint) ``` The row sums are $.50$ and $.50$, the column sums are $.70$ and $.30$, and the four cells sum to one. These are the margins and total shown in the table. ## Information in the joint-probability table A joint table records the probabilities of the four number-by-case combinations. Two probability measures can both have $\mathbb{P}(P_4)=.50$ and $\mathbb{P}(A_4)=.70$ while assigning different values to $\mathbb{P}(P_4,A_4)$. The two marginal probabilities do not determine the four joint probabilities. For instance, an association between plurality and accusative case is represented by a difference between $\mathbb{P}(A_4\mid P_4)$ and $\mathbb{P}(A_4\mid P_4^c)$. The two margins do not determine those conditional probabilities. ## Marginal and joint probabilities In the toy model, $\mathbb{P}(A_4)=.70$ includes both *him* and *them*. Only the $.40$ assigned to *them* belongs to the event $P_4\cap A_4$, so $\mathbb{P}(P_4,A_4)=.40$. To compute a joint probability, identify the intersection denoted by the comma and apply the probability measure to that event. The exact upshot is that a joint probability uses the full sample space as its reference and assigns mass to an intersection. The [next page](conditional-probability.qmd) changes that reference by restricting attention to one event. ## Check your understanding 1. Write the event represented by each of the four interior cells in the table. 2. Compute $\mathbb{P}(P_4^c,A_4^c)$ by identifying the corresponding event first. 3. Explain why $\mathbb{P}(P_4,A_4)=.40$ does not mean that $40\%$ of plural forms are accusative. 4. Construct a different four-cell table with the same row and column margins but with $\mathbb{P}(P_4,A_4)=.35$.