Probability measures

The sample space \Omega specifies which outcomes the model represents, and the sigma-algebra \mathcal{F} specifies which events are measurable. To assign probabilities, we define a probability measure. A probability measure assigns a number to each event in \mathcal{F}. We write it as

\mathbb{P}:\mathcal{F}\rightarrow[0,1].

Here \mathcal{F} is the domain of \mathbb{P}, and the closed interval [0,1] is its codomain. The function takes an event in \mathcal{F} as input and returns its probability.

The three objects

(\Omega,\mathcal{F},\mathbb{P})

together form a probability space: represented outcomes, measurable events, and numerical probabilities.

Probabilities of individual outcomes

Return to the four pronoun forms

\Omega_4 \equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}.

For this finite example, define \mathcal{F}\equiv2^{\Omega_4} so that every subset is measurable. Suppose a toy corpus motivates these assignments.

outcome	assigned probability
he	.20
him	.30
they	.10
them	.40

The table assigns probabilities to singleton events. The probability of a larger event is the sum of the probabilities assigned to its constituent outcomes.

Let

A\equiv\{\textit{him},\textit{them}\}

be the accusative event. Its probability is

\begin{aligned} \mathbb{P}(A) &=\mathbb{P}(\{\textit{him}\}) +\mathbb{P}(\{\textit{them}\})\\ &=.30+.40\\ &=.70. \end{aligned}

This particular probability measure assigns .70 to A. Another measure on the same events could assign a different probability. The value .70 describes accusative case only within this probability space.

The three requirements

Not every numerical assignment is a probability measure. The assignment must satisfy three requirements.

First, every event has nonnegative probability:

\mathbb{P}(B)\geq 0 \qquad\text{for every }B\in\mathcal{F}.

Second, the full sample space has probability one:

\mathbb{P}(\Omega)=1.

Third, probabilities add across a countable collection of events that share no outcomes. If B_1,B_2,\ldots are pairwise disjoint, then

\mathbb{P}\!\left(\bigcup_i B_i\right) =\sum_i\mathbb{P}(B_i).

Verification of the toy measure

Each value in the table is nonnegative, so the first requirement holds for the singleton events. The singleton events divide \Omega_4 into nonoverlapping parts. Their values sum to

.20+.30+.10+.40=1,

so the probability of \Omega_4 is one.

Now combine he and him. Since the two singleton events share no outcomes,

\mathbb{P}(\{\textit{he},\textit{him}\}) =.20+.30 =.50.

By countable additivity, the singleton probabilities determine the probability of the larger event. We do not need another estimate for \{\textit{he},\textit{him}\}.

A few base R checks reproduce the arithmetic.

Code

pronoun_mass <- c(he = .20, him = .30, they = .10, them = .40)

all(pronoun_mass >= 0)
sum(pronoun_mass)
sum(pronoun_mass[c("him", "them")])

The three results are TRUE, 1, and .7.

Two consequences of the axioms

The impossible event has probability zero. Since \Omega and \varnothing are disjoint and \Omega\cup\varnothing=\Omega,

\mathbb{P}(\Omega) =\mathbb{P}(\Omega)+\mathbb{P}(\varnothing).

Subtracting \mathbb{P}(\Omega) from both sides gives

\mathbb{P}(\varnothing)=0.

The probability of a complement also follows from the requirements. Since B and B^c are disjoint and their union is \Omega,

\mathbb{P}(B)+\mathbb{P}(B^c)=1.

Thus

\mathbb{P}(B^c)=1-\mathbb{P}(B).

For the accusative event, this gives \mathbb{P}(A^c)=1-.70=.30, which agrees with the mass assigned to he and they.

Assignments that violate the probability axioms

An assignment that violates any of the three requirements is not a probability measure. Suppose the four pronoun values were .20, .30, .10, and .50. Though each value lies between zero and one, their sum is 1.10, so the assignment violates normalization.

Renormalizing the values might produce a probability measure, but it changes every numerical claim. Before doing so, we should ask why the original values did not sum to one. They may be raw corpus counts, percentages computed from different denominators, or model scores that were never intended as probabilities.

Check your understanding

Using the toy measure, compute the probability of the plural event P=\{\textit{they},\textit{them}\}.
Compute \mathbb{P}(P^c) in two ways: by adding singleton probabilities and by using the complement rule.
Explain why four nonnegative values that sum to one define a coherent assignment to the singleton events in this finite space.
A table reports values .40, .35, .20, and .10 for four exhaustive outcomes. Which requirement fails?

--- title: "Probability measures" --- The [sample space](possibilities-and-events.qmd) $\Omega$ specifies which outcomes the model represents, and the [sigma-algebra](event-spaces.qmd) $\mathcal{F}$ specifies which events are measurable. To assign probabilities, we define a [**probability measure**](https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_%28Siegrist%29/02%3A_Probability_Spaces/2.09%3A_Probability_Spaces_Revisited). A probability measure assigns a number to each event in $\mathcal{F}$. We write it as $$ \mathbb{P}:\mathcal{F}\rightarrow[0,1]. $$ Here $\mathcal{F}$ is the domain of $\mathbb{P}$, and the closed interval $[0,1]$ is its codomain. The function takes an event in $\mathcal{F}$ as input and returns its probability. The three objects $$ (\Omega,\mathcal{F},\mathbb{P}) $$ together form a [**probability space**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html): represented outcomes, measurable events, and numerical probabilities. ## Probabilities of individual outcomes Return to the four pronoun forms $$ \Omega_4 \equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}. $$ For this finite example, define $\mathcal{F}\equiv2^{\Omega_4}$ so that every subset is measurable. Suppose a toy corpus motivates these assignments. | outcome | assigned probability | |---|---:| | *he* | $.20$ | | *him* | $.30$ | | *they* | $.10$ | | *them* | $.40$ | The table assigns probabilities to singleton events. The probability of a larger event is the sum of the probabilities assigned to its constituent outcomes. Let $$ A\equiv\{\textit{him},\textit{them}\} $$ be the accusative event. Its probability is $$ \begin{aligned} \mathbb{P}(A) &=\mathbb{P}(\{\textit{him}\}) +\mathbb{P}(\{\textit{them}\})\\ &=.30+.40\\ &=.70. \end{aligned} $$ This particular probability measure assigns $.70$ to $A$. Another measure on the same events could assign a different probability. The value $.70$ describes accusative case only within this probability space. ## The three requirements Not every numerical assignment is a probability measure. The assignment must satisfy three requirements. First, every event has nonnegative probability: $$ \mathbb{P}(B)\geq 0 \qquad\text{for every }B\in\mathcal{F}. $$ Second, the full sample space has probability one: $$ \mathbb{P}(\Omega)=1. $$ Third, probabilities add across a countable collection of events that share no outcomes. If $B_1,B_2,\ldots$ are pairwise disjoint, then $$ \mathbb{P}\!\left(\bigcup_i B_i\right) =\sum_i\mathbb{P}(B_i). $$ ## Verification of the toy measure Each value in the table is nonnegative, so the first requirement holds for the singleton events. The singleton events divide $\Omega_4$ into nonoverlapping parts. Their values sum to $$ .20+.30+.10+.40=1, $$ so the probability of $\Omega_4$ is one. Now combine *he* and *him*. Since the two singleton events share no outcomes, $$ \mathbb{P}(\{\textit{he},\textit{him}\}) =.20+.30 =.50. $$ By countable additivity, the singleton probabilities determine the probability of the larger event. We do not need another estimate for $\{\textit{he},\textit{him}\}$. A few base R checks reproduce the arithmetic. ```{r} #| eval: false pronoun_mass <- c(he = .20, him = .30, they = .10, them = .40) all(pronoun_mass >= 0) sum(pronoun_mass) sum(pronoun_mass[c("him", "them")]) ``` The three results are `TRUE`, `1`, and `.7`. ## Two consequences of the axioms The impossible event has probability zero. Since $\Omega$ and $\varnothing$ are disjoint and $\Omega\cup\varnothing=\Omega$, $$ \mathbb{P}(\Omega) =\mathbb{P}(\Omega)+\mathbb{P}(\varnothing). $$ Subtracting $\mathbb{P}(\Omega)$ from both sides gives $$ \mathbb{P}(\varnothing)=0. $$ The probability of a complement also follows from the requirements. Since $B$ and $B^c$ are disjoint and their union is $\Omega$, $$ \mathbb{P}(B)+\mathbb{P}(B^c)=1. $$ Thus $$ \mathbb{P}(B^c)=1-\mathbb{P}(B). $$ For the accusative event, this gives $\mathbb{P}(A^c)=1-.70=.30$, which agrees with the mass assigned to *he* and *they*. ## Assignments that violate the probability axioms An assignment that violates any of the three requirements is not a probability measure. Suppose the four pronoun values were $.20$, $.30$, $.10$, and $.50$. Though each value lies between zero and one, their sum is $1.10$, so the assignment violates normalization. Renormalizing the values might produce a probability measure, but it changes every numerical claim. Before doing so, we should ask why the original values did not sum to one. They may be raw corpus counts, percentages computed from different denominators, or model scores that were never intended as probabilities. ## Check your understanding 1. Using the toy measure, compute the probability of the plural event $P=\{\textit{they},\textit{them}\}$. 2. Compute $\mathbb{P}(P^c)$ in two ways: by adding singleton probabilities and by using the complement rule. 3. Explain why four nonnegative values that sum to one define a coherent assignment to the singleton events in this finite space. 4. A table reports values $.40$, $.35$, $.20$, and $.10$ for four exhaustive outcomes. Which requirement fails?