Conditional probability

The joint probability \mathbb{P}(P_4,A_4)=.40 is computed relative to the complete sample space. But suppose we ask a narrower question: among the plural forms, what proportion are accusative? Conditional probability changes the reference event so that the phrase among the plural forms is part of the calculation.

Definition

Basically, we keep only the probability mass inside the conditioning event and rescale that mass to sum to one. More specifically, for events B,C\in\mathcal F with \mathbb{P}(C)>0, the conditional probability of B given C is defined by

\mathbb{P}(B\mid C) \equiv \frac{\mathbb{P}(B,C)}{\mathbb{P}(C)}.

The denominator is the probability of the reference event, and dividing by it performs the rescaling; the numerator is the part of that reference event for which the target event also occurs.

The original uniform example asks for the probability of third person given accusative case. Since

\mathbb P(T_{14},A_{14})=\frac{4}{14} \qquad\text{and}\qquad \mathbb P(A_{14})=\frac{7}{14},

the conditional probability is

\begin{aligned} \mathbb P(T_{14}\mid A_{14}) &=\frac{\mathbb P(T_{14},A_{14})}{\mathbb P(A_{14})}\\ &=\frac{4/14}{7/14}\\ &=\frac47. \end{aligned}

The four-form example below repeats the calculation with the nonuniform measure used throughout the numerical exercises. Working through both makes the role of the denominator visible.

The phrase among plural forms makes P_4 the conditioning event. We write

\mathbb{P}(A_4\mid P_4)

and read it as “the probability of A_4 given P_4.” The event to the right of the bar is the condition, whereas the event to the left is the target.

Conditioning as rescaling

The plural event contains two outcomes:

P_4=\{\textit{they},\textit{them}\}.

Their original probabilities are .10 and .40, which sum to .50 because the plural event receives half of the mass in the full space.

plural outcome	probability in \Omega_4	probability within P_4
they	.10	.10/.50=.20
them	.40	.40/.50=.80

Dividing each plural probability by .50 produces a conditional probability measure on P_4. Within this restricted space, them receives probability .80. Since them is the only plural accusative outcome,

\mathbb{P}(A_4\mid P_4)=.80.

In the pronoun probability space,

\begin{aligned} \mathbb{P}(A_4\mid P_4) &=\frac{\mathbb{P}(A_4,P_4)}{\mathbb{P}(P_4)}\\ &=\frac{.40}{.50}\\ &=.80. \end{aligned}

This rescaling makes the conditioning event certain within itself:

\mathbb{P}(C\mid C) =\frac{\mathbb{P}(C,C)}{\mathbb{P}(C)} =\frac{\mathbb{P}(C)}{\mathbb{P}(C)} =1.

Reversing target and condition

Now ask for the probability of plurality among accusative forms. The numerator is still the same intersection, since P_4\cap A_4=A_4\cap P_4. The denominator changes:

\begin{aligned} \mathbb{P}(P_4\mid A_4) &=\frac{\mathbb{P}(P_4,A_4)}{\mathbb{P}(A_4)}\\ &=\frac{.40}{.70}\\ &\approx .571. \end{aligned}

Thus \mathbb{P}(A_4\mid P_4)=.80, while \mathbb{P}(P_4\mid A_4)\approx.57. The two expressions use the same joint probability but different reference events.

Base R makes the denominator visible.

Code

p_plural_and_acc <- .40
p_plural <- .50
p_acc <- .70

p_acc_given_plural <- p_plural_and_acc / p_plural
p_plural_given_acc <- p_plural_and_acc / p_acc

c(p_acc_given_plural, p_plural_given_acc)

The output is .8 and approximately .571.

Positive-probability conditions

This elementary ratio definition requires \mathbb{P}(C)>0. If \mathbb{P}(C)=0, the ratio has denominator zero and is undefined. Thus the positivity condition licenses the division in the definition.

We will later encounter settings in which conditioning on exact continuous values requires a more careful construction. Nothing from those settings is needed for the finite event calculation here.

The order of the events

Swapping the target event and conditioning event changes the conditional probability. A corpus may show that most passive clauses contain an animate subject. This is a claim about

\mathbb{P}(\text{animate subject}\mid\text{passive}).

It does not by itself show that most clauses with animate subjects are passive, which concerns the reversed conditional.

Read the event on the right of the bar first because it determines the denominator; the target event on the left determines which part of that denominator enters the numerator.

The exact upshot is that conditioning changes the reference event and then renormalizes. The next page rearranges this definition to recover a joint probability from a conditional and a marginal probability.

Check your understanding

Compute \mathbb{P}(A_4^c\mid P_4) from the table and verify that it sums with \mathbb{P}(A_4\mid P_4) to one.
Compute \mathbb{P}(P_4\mid A_4^c) and describe the reference event in words.
Explain why \mathbb{P}(A_4\mid P_4) and \mathbb{P}(P_4\mid A_4) share a numerator but need not be equal.
Give a linguistic pair of events for which reversing the condition changes the scientific question.

--- title: "Conditional probability" --- The joint probability $\mathbb{P}(P_4,A_4)=.40$ is computed relative to the complete sample space. But suppose we ask a narrower question: among the plural forms, what proportion are accusative? Conditional probability changes the reference event so that the phrase *among the plural forms* is part of the calculation. ## Definition Basically, we keep only the probability mass inside the conditioning event and rescale that mass to sum to one. More specifically, for events $B,C\in\mathcal F$ with $\mathbb{P}(C)>0$, the [**conditional probability**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) of $B$ given $C$ is defined by $$ \mathbb{P}(B\mid C) \equiv \frac{\mathbb{P}(B,C)}{\mathbb{P}(C)}. $$ The denominator is the probability of the reference event, and dividing by it performs the rescaling; the numerator is the part of that reference event for which the target event also occurs. The original uniform example asks for the probability of third person given accusative case. Since $$ \mathbb P(T_{14},A_{14})=\frac{4}{14} \qquad\text{and}\qquad \mathbb P(A_{14})=\frac{7}{14}, $$ the conditional probability is $$ \begin{aligned} \mathbb P(T_{14}\mid A_{14}) &=\frac{\mathbb P(T_{14},A_{14})}{\mathbb P(A_{14})}\\ &=\frac{4/14}{7/14}\\ &=\frac47. \end{aligned} $$ The four-form example below repeats the calculation with the nonuniform measure used throughout the numerical exercises. Working through both makes the role of the denominator visible. The phrase *among plural forms* makes $P_4$ the conditioning event. We write $$ \mathbb{P}(A_4\mid P_4) $$ and read it as “the probability of $A_4$ given $P_4$.” The event to the right of the bar is the condition, whereas the event to the left is the target. ## Conditioning as rescaling The plural event contains two outcomes: $$ P_4=\{\textit{they},\textit{them}\}. $$ Their original probabilities are $.10$ and $.40$, which sum to $.50$ because the plural event receives half of the mass in the full space. | plural outcome | probability in $\Omega_4$ | probability within $P_4$ | |---|---:|---:| | *they* | $.10$ | $.10/.50=.20$ | | *them* | $.40$ | $.40/.50=.80$ | Dividing each plural probability by $.50$ produces a conditional probability measure on $P_4$. Within this restricted space, *them* receives probability $.80$. Since *them* is the only plural accusative outcome, $$ \mathbb{P}(A_4\mid P_4)=.80. $$ In the pronoun probability space, $$ \begin{aligned} \mathbb{P}(A_4\mid P_4) &=\frac{\mathbb{P}(A_4,P_4)}{\mathbb{P}(P_4)}\\ &=\frac{.40}{.50}\\ &=.80. \end{aligned} $$ This rescaling makes the conditioning event certain within itself: $$ \mathbb{P}(C\mid C) =\frac{\mathbb{P}(C,C)}{\mathbb{P}(C)} =\frac{\mathbb{P}(C)}{\mathbb{P}(C)} =1. $$ ## Reversing target and condition Now ask for the probability of plurality among accusative forms. The numerator is still the same intersection, since $P_4\cap A_4=A_4\cap P_4$. The denominator changes: $$ \begin{aligned} \mathbb{P}(P_4\mid A_4) &=\frac{\mathbb{P}(P_4,A_4)}{\mathbb{P}(A_4)}\\ &=\frac{.40}{.70}\\ &\approx .571. \end{aligned} $$ Thus $\mathbb{P}(A_4\mid P_4)=.80$, while $\mathbb{P}(P_4\mid A_4)\approx.57$. The two expressions use the same joint probability but different reference events. Base R makes the denominator visible. ```{r} #| eval: false p_plural_and_acc <- .40 p_plural <- .50 p_acc <- .70 p_acc_given_plural <- p_plural_and_acc / p_plural p_plural_given_acc <- p_plural_and_acc / p_acc c(p_acc_given_plural, p_plural_given_acc) ``` The output is `.8` and approximately `.571`. ## Positive-probability conditions This elementary ratio definition requires $\mathbb{P}(C)>0$. If $\mathbb{P}(C)=0$, the ratio has denominator zero and is undefined. Thus the positivity condition licenses the division in the definition. We will later encounter settings in which conditioning on exact continuous values requires a more careful construction. Nothing from those settings is needed for the finite event calculation here. ## The order of the events Swapping the target event and conditioning event changes the conditional probability. A corpus may show that most passive clauses contain an animate subject. This is a claim about $$ \mathbb{P}(\text{animate subject}\mid\text{passive}). $$ It does not by itself show that most clauses with animate subjects are passive, which concerns the reversed conditional. Read the event on the right of the bar first because it determines the denominator; the target event on the left determines which part of that denominator enters the numerator. The exact upshot is that conditioning changes the reference event and then renormalizes. The [next page](probability-factorization.qmd) rearranges this definition to recover a joint probability from a conditional and a marginal probability. ## Check your understanding 1. Compute $\mathbb{P}(A_4^c\mid P_4)$ from the table and verify that it sums with $\mathbb{P}(A_4\mid P_4)$ to one. 2. Compute $\mathbb{P}(P_4\mid A_4^c)$ and describe the reference event in words. 3. Explain why $\mathbb{P}(A_4\mid P_4)$ and $\mathbb{P}(P_4\mid A_4)$ share a numerator but need not be equal. 4. Give a linguistic pair of events for which reversing the condition changes the scientific question.