Bayes’ rule

Suppose a pronunciation feature is strongly associated with a small dialect group. Hearing the feature may provide evidence about a speaker’s dialect, but the probability of group membership given the feature cannot be read directly from the probability of the feature given group membership. Bayes’ rule relates these two conditional probabilities.

Recall from the chain rule that the same joint probability can be factored in two orders:

\mathbb{P}(B,C) =\mathbb{P}(B\mid C)\mathbb{P}(C)

and

\mathbb{P}(B,C) =\mathbb{P}(C\mid B)\mathbb{P}(B).

Since the left sides are identical, the right sides are equal:

\mathbb{P}(B\mid C)\mathbb{P}(C) =\mathbb{P}(C\mid B)\mathbb{P}(B).

If \mathbb{P}(C)>0, divide both sides by \mathbb{P}(C):

\mathbb{P}(B\mid C) =\frac{\mathbb{P}(C\mid B)\mathbb{P}(B)} {\mathbb{P}(C)}.

This identity is Bayes’ rule. It follows from applying the chain rule to one joint probability in two orders. No independence assumption is used.

The pronoun measure

Let A again be the accusative event and P the plural event. We want \mathbb{P}(A\mid P). Bayes’ rule gives

\mathbb{P}(A\mid P) =\frac{\mathbb{P}(P\mid A)\mathbb{P}(A)} {\mathbb{P}(P)}.

The values from the joint table are

\mathbb{P}(P\mid A)=\frac{.40}{.70}, \qquad \mathbb{P}(A)=.70, \qquad \mathbb{P}(P)=.50.

Substituting them gives

\begin{aligned} \mathbb{P}(A\mid P) &=\frac{(.40/.70)\times.70}{.50}\\ &=\frac{.40}{.50}\\ &=.80. \end{aligned}

This is the same value obtained from the definition of conditional probability. Bayes’ rule computes it from the reversed conditional probability and the two marginal probabilities.

A dialect-feature example

Now let D be membership in a dialect group and F be use of a pronunciation feature. Imagine a represented population of 1,000 tokens with these counts.

	feature F	no feature F^c	total
dialect group D	90	10	100
other group D^c	180	720	900
total	270	730	1,000

Within the dialect group, the feature is very common:

\mathbb{P}(F\mid D)=\frac{90}{100}=.90.

We want the reversed probability, the probability of group membership given the feature:

\mathbb{P}(D\mid F).

The other required values are

\mathbb{P}(D)=\frac{100}{1000}=.10

and

\mathbb{P}(F)=\frac{270}{1000}=.27.

Bayes’ rule gives

\begin{aligned} \mathbb{P}(D\mid F) &=\frac{\mathbb{P}(F\mid D)\mathbb{P}(D)} {\mathbb{P}(F)}\\ &=\frac{.90\times.10}{.27}\\ &=\frac{1}{3}. \end{aligned}

The feature occurs in ninety percent of the dialect-group tokens, but only one third of all tokens containing the feature come from that group. The group accounts for only ten percent of the population, and the feature also occurs outside the group. Both facts enter the reversed conditional probability.

Code

p_feature_given_dialect <- .90
p_dialect <- .10
p_feature <- .27

p_dialect_given_feature <-
  p_feature_given_dialect * p_dialect / p_feature

p_dialect_given_feature

The result is approximately .333.

Evidence and prevalence

In the dialect calculation, \mathbb{P}(F\mid D) gives the probability of the feature within the group, \mathbb{P}(D) gives the prevalence of the group in the represented population, and \mathbb{P}(F) gives the overall probability of the feature.

All three quantities are tied to an observation process. A sample balanced to contain equal numbers of speakers from two groups does not directly provide the population value of \mathbb{P}(D). Using the sample proportion would answer a question about the balanced sample unless the design is adjusted to represent the target population.

The order of the events

As discussed in the introduction to conditional probability, \mathbb{P}(F\mid D) cannot be substituted for \mathbb{P}(D\mid F). Bayes’ rule shows exactly what that substitution omits: the prevalence of D and the overall probability of F.

A feature may be common within a rare group even though most tokens containing that feature come from elsewhere. We can verify the direction of the conditional in the count table: 90/270 is the proportion of feature tokens from the dialect group, while 90/100 is the proportion of dialect-group tokens containing the feature.

Check your understanding

Derive Bayes’ rule by equating the two chain-rule factorizations of \mathbb{P}(B,C).
In the dialect table, compute \mathbb{P}(F\mid D^c) and \mathbb{P}(D^c\mid F).
Explain in words why .90 and 1/3 answer different questions.
If the dialect group made up half of the represented population while the two within group feature rates stayed fixed, would \mathbb{P}(D\mid F) increase or decrease? Explain without introducing a new formula.

--- title: "Bayes' rule" --- Suppose a pronunciation feature is strongly associated with a small dialect group. Hearing the feature may provide evidence about a speaker's dialect, but the probability of group membership given the feature cannot be read directly from the probability of the feature given group membership. Bayes' rule relates these two conditional probabilities. Recall from [the chain rule](probability-factorization.qmd) that the same joint probability can be factored in two orders: $$ \mathbb{P}(B,C) =\mathbb{P}(B\mid C)\mathbb{P}(C) $$ and $$ \mathbb{P}(B,C) =\mathbb{P}(C\mid B)\mathbb{P}(B). $$ Since the left sides are identical, the right sides are equal: $$ \mathbb{P}(B\mid C)\mathbb{P}(C) =\mathbb{P}(C\mid B)\mathbb{P}(B). $$ If $\mathbb{P}(C)>0$, divide both sides by $\mathbb{P}(C)$: $$ \mathbb{P}(B\mid C) =\frac{\mathbb{P}(C\mid B)\mathbb{P}(B)} {\mathbb{P}(C)}. $$ This identity is [**Bayes' rule**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html). It follows from applying the chain rule to one joint probability in two orders. No independence assumption is used. ## The pronoun measure Let $A$ again be the accusative event and $P$ the plural event. We want $\mathbb{P}(A\mid P)$. Bayes' rule gives $$ \mathbb{P}(A\mid P) =\frac{\mathbb{P}(P\mid A)\mathbb{P}(A)} {\mathbb{P}(P)}. $$ The values from the joint table are $$ \mathbb{P}(P\mid A)=\frac{.40}{.70}, \qquad \mathbb{P}(A)=.70, \qquad \mathbb{P}(P)=.50. $$ Substituting them gives $$ \begin{aligned} \mathbb{P}(A\mid P) &=\frac{(.40/.70)\times.70}{.50}\\ &=\frac{.40}{.50}\\ &=.80. \end{aligned} $$ This is the same value obtained from the definition of conditional probability. Bayes' rule computes it from the reversed conditional probability and the two marginal probabilities. ## A dialect-feature example Now let $D$ be membership in a dialect group and $F$ be use of a pronunciation feature. Imagine a represented population of 1,000 tokens with these counts. | | feature $F$ | no feature $F^c$ | total | |---|---:|---:|---:| | dialect group $D$ | 90 | 10 | 100 | | other group $D^c$ | 180 | 720 | 900 | | total | 270 | 730 | 1,000 | Within the dialect group, the feature is very common: $$ \mathbb{P}(F\mid D)=\frac{90}{100}=.90. $$ We want the reversed probability, the probability of group membership given the feature: $$ \mathbb{P}(D\mid F). $$ The other required values are $$ \mathbb{P}(D)=\frac{100}{1000}=.10 $$ and $$ \mathbb{P}(F)=\frac{270}{1000}=.27. $$ Bayes' rule gives $$ \begin{aligned} \mathbb{P}(D\mid F) &=\frac{\mathbb{P}(F\mid D)\mathbb{P}(D)} {\mathbb{P}(F)}\\ &=\frac{.90\times.10}{.27}\\ &=\frac{1}{3}. \end{aligned} $$ The feature occurs in ninety percent of the dialect-group tokens, but only one third of all tokens containing the feature come from that group. The group accounts for only ten percent of the population, and the feature also occurs outside the group. Both facts enter the reversed conditional probability. ```{r} #| eval: false p_feature_given_dialect <- .90 p_dialect <- .10 p_feature <- .27 p_dialect_given_feature <- p_feature_given_dialect * p_dialect / p_feature p_dialect_given_feature ``` The result is approximately `.333`. ## Evidence and prevalence In the dialect calculation, $\mathbb{P}(F\mid D)$ gives the probability of the feature within the group, $\mathbb{P}(D)$ gives the prevalence of the group in the represented population, and $\mathbb{P}(F)$ gives the overall probability of the feature. All three quantities are tied to an observation process. A sample balanced to contain equal numbers of speakers from two groups does not directly provide the population value of $\mathbb{P}(D)$. Using the sample proportion would answer a question about the balanced sample unless the design is adjusted to represent the target population. ## The order of the events As discussed in the [introduction to conditional probability](conditional-probability.qmd#the-order-of-the-events), $\mathbb{P}(F\mid D)$ cannot be substituted for $\mathbb{P}(D\mid F)$. Bayes' rule shows exactly what that substitution omits: the prevalence of $D$ and the overall probability of $F$. A feature may be common within a rare group even though most tokens containing that feature come from elsewhere. We can verify the direction of the conditional in the count table: $90/270$ is the proportion of feature tokens from the dialect group, while $90/100$ is the proportion of dialect-group tokens containing the feature. ## Check your understanding 1. Derive Bayes' rule by equating the two chain-rule factorizations of $\mathbb{P}(B,C)$. 2. In the dialect table, compute $\mathbb{P}(F\mid D^c)$ and $\mathbb{P}(D^c\mid F)$. 3. Explain in words why $.90$ and $1/3$ answer different questions. 4. If the dialect group made up half of the represented population while the two within group feature rates stayed fixed, would $\mathbb{P}(D\mid F)$ increase or decrease? Explain without introducing a new formula.