Conditional independence

The foundations chapter defined independence for events, and the preceding pages extended probability to joint distributions. We now ask whether two variables are independent after conditioning on a third. Suppose a corpus records subject omission O, verb final order V, and clause type C.

Conditional independence means that, once the conditioning variable is known, learning one target variable does not change the conditional distribution of the other.

We write

O\perp\!\!\!\perp V\mid C

and read the statement as “O is independent of V given C.”

For p_C(c)>0, define

p_{O,V\mid C}(o,v\mid c) \equiv\mathbb{P}(O=o,V=v\mid C=c).

The corresponding one-variable conditional PMFs are

p_{O\mid C}(o\mid c)\equiv\mathbb{P}(O=o\mid C=c)

and p_{V\mid C}(v\mid c)\equiv\mathbb{P}(V=v\mid C=c).

For p_{V,C}(v,c)>0, define

p_{O\mid V,C}(o\mid v,c) \equiv\mathbb{P}(O=o\mid V=v,C=c).

The conditional-invariance criterion for O\perp\!\!\!\perp V\mid C is

p_{O\mid V,C}(o\mid v,c) =p_{O\mid C}(o\mid c)

for every o and every (v,c) with p_{V,C}(v,c)>0.

Constructing an independent table for main clauses

Within main clauses, suppose

p_{O\mid C}(1\mid\mathrm{main})=.80

and

p_{V\mid C}(1\mid\mathrm{main})=.75.

If O and V are independent within this clause type, the joint target probability is

.80(.75)=.60.

The full conditional joint PMF is

main clauses	V=0	V=1	total
O=0	.05	.15	.20
O=1	.20	.60	.80
total	.25	.75	1

Every cell equals its row margin times its column margin.

Constructing an independent table for subordinate clauses

Within subordinate clauses, suppose

p_{O\mid C}(1\mid\mathrm{sub})=.20

and

p_{V\mid C}(1\mid\mathrm{sub})=.25.

The joint target probability under conditional independence is .20(.25)=.05.

subordinate clauses	V=0	V=1	total
O=0	.60	.20	.80
O=1	.15	.05	.20
total	.75	.25	1

Again, every cell factors into the corresponding conditional margins.

Deriving the conditional factorization

The conditional chain rule gives

p_{O,V\mid C}(o,v\mid c) =p_{O\mid V,C}(o\mid v,c)p_{V\mid C}(v\mid c).

Substituting the conditional-invariance criterion gives

p_{O,V\mid C}(o,v\mid c) =p_{O\mid C}(o\mid c)p_{V\mid C}(v\mid c).

This factorization is the equivalent criterion that remains applicable to zero-mass value combinations.

Pooling the clause types

Suppose main and subordinate clauses each receive probability .50. Averaging corresponding cells across the two tables gives

pooled clauses	V=0	V=1	total
O=0	.325	.175	.50
O=1	.175	.325	.50
total	.50	.50	1

In the pooled table,

\begin{aligned} p_{O,V}(1,1) &=\sum_c p_{O,V\mid C}(1,1\mid c)p_C(c)\\ &=.60(.50)+.05(.50)\\ &=.325. \end{aligned}

The product of the pooled margins is

p_O(1)p_V(1) =.50(.50) =.25.

Since .325\neq.25, the variables are dependent before conditioning on clause type.

Main clauses have high probabilities for both properties, while subordinate clauses have low probabilities for both. Mixing clause types produces a pooled association even though the properties are independent within each type.

Verifying one stratum in base R

Code

main <- matrix(c(.05, .20, .15, .60), nrow = 2)
subordinate <- matrix(c(.60, .15, .20, .05), nrow = 2)
pooled <- .50 * main + .50 * subordinate

main[2, 2]
sum(main[2, ]) * sum(main[, 2])
pooled

The first two results both equal .60, verifying one main clause cell. The pooled matrix shows the induced association.

Conditional and unconditional claims differ

The example establishes that

O\perp\!\!\!\perp V\mid C

does not imply

O\perp\!\!\!\perp V.

The reverse implication also need not hold. An independence claim is incomplete unless it states which variables are conditioned on.

Stating the conditioning set

A conditional independence claim must state its conditioning set. Saying “subject omission is independent of verb order” erases the fact that the equality holds only within clause types in this model.

Include the vertical bar in notation and prose: independent given clause type.

Check your understanding

Verify the cell p_{O,V\mid C}(0,1\mid\mathrm{main})=.15 from its margins.
Compute p_{O\mid V,C}(1\mid1,\mathrm{sub}) and compare it with p_{O\mid C}(1\mid\mathrm{sub}).
Why are O and V dependent in the pooled table?
State precisely which independence claim the two stratum tables support.

Conditional independence concerns complete distributions within groups. The next page summarizes the center of a response distribution within each condition.

--- title: "Conditional independence" --- The foundations chapter defined [independence for events](../foundations/independence.qmd), and the preceding pages extended probability to [joint distributions](joint-probability.qmd). We now ask whether two variables are independent after conditioning on a third. Suppose a corpus records subject omission $O$, verb final order $V$, and clause type $C$. [**Conditional independence**](https://www.probabilitycourse.com/chapter1/1_4_4_conditional_independence.php) means that, once the conditioning variable is known, learning one target variable does not change the conditional distribution of the other. We write $$ O\perp\!\!\!\perp V\mid C $$ and read the statement as “$O$ is independent of $V$ given $C$.” For $p_C(c)>0$, define $$ p_{O,V\mid C}(o,v\mid c) \equiv\mathbb{P}(O=o,V=v\mid C=c). $$ The corresponding one-variable conditional PMFs are $$ p_{O\mid C}(o\mid c)\equiv\mathbb{P}(O=o\mid C=c) $$ and $p_{V\mid C}(v\mid c)\equiv\mathbb{P}(V=v\mid C=c)$. For $p_{V,C}(v,c)>0$, define $$ p_{O\mid V,C}(o\mid v,c) \equiv\mathbb{P}(O=o\mid V=v,C=c). $$ The conditional-invariance criterion for $O\perp\!\!\!\perp V\mid C$ is $$ p_{O\mid V,C}(o\mid v,c) =p_{O\mid C}(o\mid c) $$ for every $o$ and every $(v,c)$ with $p_{V,C}(v,c)>0$. ## Constructing an independent table for main clauses Within main clauses, suppose $$ p_{O\mid C}(1\mid\mathrm{main})=.80 $$ and $$ p_{V\mid C}(1\mid\mathrm{main})=.75. $$ If $O$ and $V$ are independent within this clause type, the joint target probability is $$ .80(.75)=.60. $$ The full conditional joint PMF is | main clauses | $V=0$ | $V=1$ | total | |---|---:|---:|---:| | $O=0$ | $.05$ | $.15$ | $.20$ | | $O=1$ | $.20$ | $.60$ | $.80$ | | total | $.25$ | $.75$ | $1$ | Every cell equals its row margin times its column margin. ## Constructing an independent table for subordinate clauses Within subordinate clauses, suppose $$ p_{O\mid C}(1\mid\mathrm{sub})=.20 $$ and $$ p_{V\mid C}(1\mid\mathrm{sub})=.25. $$ The joint target probability under conditional independence is $.20(.25)=.05$. | subordinate clauses | $V=0$ | $V=1$ | total | |---|---:|---:|---:| | $O=0$ | $.60$ | $.20$ | $.80$ | | $O=1$ | $.15$ | $.05$ | $.20$ | | total | $.75$ | $.25$ | $1$ | Again, every cell factors into the corresponding conditional margins. ## Deriving the conditional factorization The conditional chain rule gives $$ p_{O,V\mid C}(o,v\mid c) =p_{O\mid V,C}(o\mid v,c)p_{V\mid C}(v\mid c). $$ Substituting the conditional-invariance criterion gives $$ p_{O,V\mid C}(o,v\mid c) =p_{O\mid C}(o\mid c)p_{V\mid C}(v\mid c). $$ This factorization is the equivalent criterion that remains applicable to zero-mass value combinations. ## Pooling the clause types Suppose main and subordinate clauses each receive probability $.50$. Averaging corresponding cells across the two tables gives | pooled clauses | $V=0$ | $V=1$ | total | |---|---:|---:|---:| | $O=0$ | $.325$ | $.175$ | $.50$ | | $O=1$ | $.175$ | $.325$ | $.50$ | | total | $.50$ | $.50$ | $1$ | In the pooled table, $$ \begin{aligned} p_{O,V}(1,1) &=\sum_c p_{O,V\mid C}(1,1\mid c)p_C(c)\\ &=.60(.50)+.05(.50)\\ &=.325. \end{aligned} $$ The product of the pooled margins is $$ p_O(1)p_V(1) =.50(.50) =.25. $$ Since $.325\neq.25$, the variables are dependent before conditioning on clause type. Main clauses have high probabilities for both properties, while subordinate clauses have low probabilities for both. Mixing clause types produces a pooled association even though the properties are independent within each type. ## Verifying one stratum in base R ```{r} #| eval: false main <- matrix(c(.05, .20, .15, .60), nrow = 2) subordinate <- matrix(c(.60, .15, .20, .05), nrow = 2) pooled <- .50 * main + .50 * subordinate main[2, 2] sum(main[2, ]) * sum(main[, 2]) pooled ``` The first two results both equal `.60`, verifying one main clause cell. The pooled matrix shows the induced association. ## Conditional and unconditional claims differ The example establishes that $$ O\perp\!\!\!\perp V\mid C $$ does not imply $$ O\perp\!\!\!\perp V. $$ The reverse implication also need not hold. An independence claim is incomplete unless it states which variables are conditioned on. ## Stating the conditioning set A conditional independence claim must state its conditioning set. Saying “subject omission is independent of verb order” erases the fact that the equality holds only within clause types in this model. Include the vertical bar in notation and prose: independent given clause type. ## Check your understanding 1. Verify the cell $p_{O,V\mid C}(0,1\mid\mathrm{main})=.15$ from its margins. 2. Compute $p_{O\mid V,C}(1\mid1,\mathrm{sub})$ and compare it with $p_{O\mid C}(1\mid\mathrm{sub})$. 3. Why are $O$ and $V$ dependent in the pooled table? 4. State precisely which independence claim the two stratum tables support. Conditional independence concerns complete distributions within groups. The [next page](conditional-expectation.qmd) summarizes the center of a response distribution within each condition.