Suppose half of the represented pronoun tokens are plural and 80\% of the plural tokens are accusative. The chain rule recovers the probability that a token is both plural and accusative.

Because \mathbb{P}(P)=.50>0, begin with the definition of conditional probability:

\mathbb{P}(A\mid P) \equiv\frac{\mathbb{P}(A,P)}{\mathbb{P}(P)}.

Here P is the plural event and A is the accusative event. Multiply both sides by \mathbb{P}(P):

\mathbb{P}(A\mid P)\mathbb{P}(P) =\mathbb{P}(A,P).

Writing the joint probability first gives

\mathbb{P}(A,P) =\mathbb{P}(A\mid P)\mathbb{P}(P).

This identity is the two-event chain rule, also called the multiplication rule. Repeated application gives the general chain rule.

Reconstructing the joint probability

The pronoun measure gives

\mathbb{P}(P)=.50

and

\mathbb{P}(A\mid P)=.80.

Substitute these two values into the two-event chain rule:

\begin{aligned} \mathbb{P}(A,P) &=\mathbb{P}(A\mid P)\mathbb{P}(P)\\ &=.80\times.50\\ &=.40. \end{aligned}

The conditioning event P has probability .50. Within that event, the target event A has conditional probability .80. Their product is the probability that both events occur.

We can check the same logic with counts. Imagine 100 pronoun tokens.

  1. \mathbb{P}(P)=.50 corresponds to 50 plural tokens.
  2. \mathbb{P}(A\mid P)=.80 corresponds to 40 accusative tokens among those 50 plural tokens.
  3. Hence \mathbb{P}(A,P)=.40 corresponds to 40 tokens among all 100 tokens.

The two-event chain rule expresses the same calculation in probabilities rather than counts.

Code 
p_plural <- .50
p_acc_given_plural <- .80

p_plural_and_acc <- p_acc_given_plural * p_plural
p_plural_and_acc

The result is .4.

Reversing the order

An intersection is symmetric:

A\cap P=P\cap A.

We can thus begin from the opposite conditional probability:

\mathbb{P}(A,P) =\mathbb{P}(P\mid A)\mathbb{P}(A).

In the toy measure,

\mathbb{P}(P\mid A)=\frac{.40}{.70}

and \mathbb{P}(A)=.70. The reversed product is

\begin{aligned} \mathbb{P}(P\mid A)\mathbb{P}(A) &=\frac{.40}{.70}\times.70\\ &=.40. \end{aligned}

The two products have different factors because they use different conditioning events. Both equal the same joint probability.

This gives the identity

\mathbb{P}(A\mid P)\mathbb{P}(P) =\mathbb{P}(P\mid A)\mathbb{P}(A).

We will use this equality to derive Bayes’ rule. No independence assumption is involved.

Three-event chain rule

For events B,C,D\in\mathcal F, suppose \mathbb{P}(B)>0 and \mathbb{P}(B,C)>0. Begin by separating the final event from the first two:

\mathbb{P}(B,C,D) =\mathbb{P}(D\mid B,C)\mathbb{P}(B,C).

Factor the remaining joint probability:

\mathbb{P}(B,C)=\mathbb{P}(C\mid B)\mathbb{P}(B).

Substitution gives

\mathbb{P}(B,C,D) =\mathbb{P}(B)\mathbb{P}(C\mid B)\mathbb{P}(D\mid B,C).

Every equality above follows from the two-event rule.

General chain rule

Suppose every conditioning event displayed below has positive probability. Applying the same step repeatedly gives

\mathbb{P}(E_1,\ldots,E_N) =\mathbb{P}(E_1) \prod_{i=2}^{N} \mathbb{P}(E_i\mid E_1,\ldots,E_{i-1}).

The ordering is arbitrary, but it must remain consistent across the factors. Each factor conditions on every event that precedes it in the chosen order.

Factorization and dependence

The chain rule is derived from conditional probability and does not assert independence. The factor \mathbb{P}(A\mid P) allows the probability of A to differ inside and outside P.

For instance, suppose a speaker’s use of a reduced form depends on whether the word is predictable in context. We can still factor the joint event as

\mathbb{P}(\text{reduced},\text{predictable}) =\mathbb{P}(\text{reduced}\mid\text{predictable}) \mathbb{P}(\text{predictable}).

Any dependence between A and P remains in the conditional term. The multiplication sign does not justify replacing

\mathbb{P}(A\mid P)

with

\mathbb{P}(A).

That substitution requires independence. The chain rule requires only that each displayed conditional probability be defined.

Check your understanding

  1. Starting from the definition of \mathbb{P}(P\mid A), derive \mathbb{P}(A,P)=\mathbb{P}(P\mid A)\mathbb{P}(A).
  2. Suppose \mathbb{P}(P)=.30 and \mathbb{P}(A\mid P)=.60. Compute \mathbb{P}(A,P) and interpret the result among 100 tokens.
  3. Derive the three-event chain rule by applying the two-event rule twice.
  4. Write the chain rule for \mathbb{P}(E_1,E_2,E_3,E_4) in the order E_1,E_2,E_3,E_4.
  5. Identify the independence assumption required to replace \mathbb{P}(A\mid P) with \mathbb{P}(A).