Code
p_plural <- .50
p_acc_given_plural <- .80
p_plural_and_acc <- p_acc_given_plural * p_plural
p_plural_and_accThe chain rule
Suppose half of the represented pronoun tokens are plural and 80\% of the plural tokens are accusative. What proportion of all represented tokens are both plural and accusative? The chain rule recovers that joint probability from the marginal and conditional probabilities.
Because \mathbb{P}(P_4)=.50>0, begin with the definition of conditional probability:
\mathbb{P}(A_4\mid P_4) \equiv\frac{\mathbb{P}(A_4,P_4)}{\mathbb{P}(P_4)}.
Here P_4 is the plural event and A_4 is the accusative event. The notation on the right expands the conditional probability into a ratio. Since \mathbb{P}(P_4)>0, we may multiply both sides by \mathbb{P}(P_4):
\mathbb{P}(A_4\mid P_4)\mathbb{P}(P_4) =\mathbb{P}(A_4,P_4).
This multiplication cancels the denominator on the right. Writing the joint probability first gives
\mathbb{P}(A_4,P_4) =\mathbb{P}(A_4\mid P_4)\mathbb{P}(P_4).
This identity is the two-event chain rule, also called the multiplication rule. The move from a joint probability to a product is a factorization. Repeated application gives the general chain rule.
Reconstructing the joint probability
The pronoun measure gives
\mathbb{P}(P_4)=.50
and
\mathbb{P}(A_4\mid P_4)=.80.
Substitute these two values into the two-event chain rule:
\begin{aligned} \mathbb{P}(A_4,P_4) &=\mathbb{P}(A_4\mid P_4)\mathbb{P}(P_4)\\ &=.80\times.50\\ &=.40. \end{aligned}
The conditioning event P_4 has probability .50, and within that event, the target event A_4 has conditional probability .80. Their product is the probability that both events occur.
We can check the same logic with counts. Imagine 100 pronoun tokens.
- \mathbb{P}(P_4)=.50 corresponds to 50 plural tokens.
- \mathbb{P}(A_4\mid P_4)=.80 corresponds to 40 accusative tokens among those 50 plural tokens.
- Hence \mathbb{P}(A_4,P_4)=.40 corresponds to 40 tokens among all 100 tokens.
The two-event chain rule expresses the same calculation in probabilities rather than counts.
The result is .4.
Reversing the order
An intersection is symmetric:
A_4\cap P_4=P_4\cap A_4.
We can thus begin from the opposite conditional probability:
\mathbb{P}(A_4,P_4) =\mathbb{P}(P_4\mid A_4)\mathbb{P}(A_4).
In the toy measure,
\mathbb{P}(P_4\mid A_4)=\frac{.40}{.70}
and \mathbb{P}(A_4)=.70. The reversed product is
\begin{aligned} \mathbb{P}(P_4\mid A_4)\mathbb{P}(A_4) &=\frac{.40}{.70}\times.70\\ &=.40. \end{aligned}
The two products have different factors because they use different conditioning events. Both equal the same joint probability.
This gives the identity
\mathbb{P}(A_4\mid P_4)\mathbb{P}(P_4) =\mathbb{P}(P_4\mid A_4)\mathbb{P}(A_4).
We will use this equality to derive Bayes’ rule. No independence assumption is involved.
Three-event chain rule
Now, you might wonder how the same move works for more than two events. For events B,C,D\in\mathcal F, suppose \mathbb{P}(B)>0 and \mathbb{P}(B,C)>0. Begin by separating the final event from the first two:
\mathbb{P}(B,C,D) =\mathbb{P}(D\mid B,C)\mathbb{P}(B,C).
Factor the remaining joint probability:
\mathbb{P}(B,C)=\mathbb{P}(C\mid B)\mathbb{P}(B).
Substitute the second factorization for \mathbb P(B,C) in the first:
\mathbb{P}(B,C,D) =\mathbb{P}(B)\mathbb{P}(C\mid B)\mathbb{P}(D\mid B,C).
Every equality above follows from the two-event rule. Nothing new is assumed at the three-event step.
General chain rule
Suppose every conditioning event displayed below has positive probability. Applying the same step repeatedly gives
\mathbb{P}(E_1,\ldots,E_N) =\mathbb{P}(E_1) \prod_{i=2}^{N} \mathbb{P}(E_i\mid E_1,\ldots,E_{i-1}).
The ordering is arbitrary, but it must remain consistent across the factors. Each factor conditions on every event that precedes it in the chosen order.
Factorization and dependence
The chain rule is derived from conditional probability and does not assert independence. The factor \mathbb{P}(A_4\mid P_4) allows the probability of A_4 to differ inside and outside P_4.
For instance, corpus research finds that contextual predictability is associated with word duration in conversational English. Suppose the two events are that a word has short duration and that it is highly predictable in context. We can still factor the joint event as
\mathbb{P}(\text{short duration},\text{high predictability}) =\mathbb{P}(\text{short duration}\mid\text{high predictability}) \mathbb{P}(\text{high predictability}).
This factorization represents an association in the chosen population. It does not by itself show that predictability caused the shorter duration.
Any dependence between A_4 and P_4 remains in the conditional term. The multiplication sign does not justify replacing
\mathbb{P}(A_4\mid P_4)
with
\mathbb{P}(A_4).
That substitution requires independence. The chain rule requires only that each displayed conditional probability be defined.
The exact upshot is that the chain rule expands a joint probability into ordered conditional factors without assuming independence. The next page factors one joint probability in two orders and solves for a reversed conditional probability.
Check your understanding
- Starting from the definition of \mathbb{P}(P_4\mid A_4), derive \mathbb{P}(A_4,P_4)=\mathbb{P}(P_4\mid A_4)\mathbb{P}(A_4).
- Suppose \mathbb{P}(P_4)=.30 and \mathbb{P}(A_4\mid P_4)=.60. Compute \mathbb{P}(A_4,P_4) and interpret the result among 100 tokens.
- Derive the three-event chain rule by applying the two-event rule twice.
- Write the chain rule for \mathbb{P}(E_1,E_2,E_3,E_4) in the order E_1,E_2,E_3,E_4.
- Identify the independence assumption required to replace \mathbb{P}(A_4\mid P_4) with \mathbb{P}(A_4).