Sigma-algebras

Suppose that our pronoun model assigns a probability to the event that a form is plural. Which other events must the model be able to measure? It must assign a probability to the event that the form is not plural, since that event is the complement of the first. If the model assigns probabilities to two events, it must also assign a probability to their union. The need to keep these operations available is the closure problem.

Basically, we solve the closure problem by collecting events in a way that never strands us without a complement or required union. More specifically, a sigma-algebra, also called an event space, is a collection of events that contains the sample space and is closed under complements and countable unions. The model assigns probabilities only to events in this collection, which we write as \mathcal{F}.

The person event space

For the twelve-form pronoun space on the preceding page, let T_{12} be the third-person event. The sigma-algebra generated by this distinction is

\mathcal F_{\mathrm{person}} \equiv\sigma(T_{12}) =\{\varnothing,T_{12},T_{12}^c,\Omega\}.

It contains exactly the unions of the two atoms T_{12} and T_{12}^c.

The case event space

Let A_{12} be the accusative event and N_{12} the nominative event. Since

A_{12}\cap N_{12}=\{\textit{you},\textit{it}\},

the two events are not complements. The three nonempty atoms are

A_{12}\cap N_{12}, \qquad A_{12}\setminus N_{12}, \qquad N_{12}\setminus A_{12}.

Thus

\mathcal F_{\mathrm{case}} \equiv\sigma(A_{12},N_{12})

contains the eight unions of these atoms. In particular, closure requires the overlap A_{12}\cap N_{12} and its complement to be measurable.

To see why the collection needs structure, use the four outcome sample space

\Omega_4 \equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}.

Let P_4 be the plural event:

P_4\equiv\{\textit{they},\textit{them}\}.

Its complement is

P_4^c=\{\textit{he},\textit{him}\}.

If plural number is the only distinction that the model measures, one possible sigma-algebra is

\mathcal{F}_{P_4} \equiv\{\varnothing,P_4,P_4^c,\Omega_4\}.

Plural and singular are measurable under \mathcal{F}_{P_4}. Accusative case is not, because the accusative event does not belong to \mathcal{F}_{P_4}.

Closure requirements

A collection \mathcal{F} is a sigma-algebra only if it satisfies three requirements.

The complete sample space \Omega belongs to \mathcal{F}.
If B belongs to \mathcal{F}, then its complement B^c also belongs to \mathcal{F}.
If B_1,B_2,\ldots belong to \mathcal{F}, then their union \bigcup_i B_i also belongs to \mathcal{F}.

The first requirement includes the certain event among the measurable events; the second includes the complement of every measurable event; and the third includes the union of any countable collection of measurable events.

Now we can derive consequences one move at a time. Since \Omega\in\mathcal{F}, closure under complements gives

\Omega^c=\varnothing\in\mathcal{F}.

What about intersections, which do not appear as a separate requirement? They follow from complements and unions. De Morgan’s law gives

B\cap C=(B^c\cup C^c)^c.

The right side first takes complements, then a union, and then one more complement, with each move licensed by the stated closure requirements. Thus a sigma-algebra that is closed under complements and countable unions is also closed under intersections.

A collection that is not a sigma-algebra

Consider the proposed collection

\mathcal{G}\equiv\{\Omega_4,P_4\}.

Check the proposed collection against each requirement.

It contains \Omega_4, so it passes the first requirement.
It contains P_4 but not P_4^c, so it fails the second requirement.
It also lacks \varnothing, which must appear as \Omega_4^c.

Adding P_4^c and \varnothing repairs these failures and produces \mathcal{F}_{P_4}.

The following R list represents the four events in \mathcal{F}_{P_4}.

Code

omega <- c("he", "him", "they", "them")
plural <- c("they", "them")
singular <- setdiff(omega, plural)

event_space <- list(
  impossible = character(0),
  plural = plural,
  singular = singular,
  certain = omega
)

event_space

The list omits the singleton event \{\textit{them}\}. This set is a subset of \Omega_4, but it is not an event in \mathcal{F}_{P_4}. The model represents them as an outcome, but it cannot assign a probability to the event containing them alone.

Sample spaces and event spaces

\Omega states which individual outcomes exist in the model.
\mathcal{F} states which groupings of those outcomes can receive probabilities.

For a small finite sample space, we will often use the power set 2^\Omega, the collection containing every subset of \Omega. In that case, every possible grouping of the represented outcomes is measurable. The smaller sigma-algebra \mathcal{F}_{P_4} shows that this choice is not automatic.

An event outside the sigma-algebra

In the example above, the accusative event does not belong to \mathcal{F}_{P_4}. We thus cannot write its probability under a probability measure whose domain is \mathcal{F}_{P_4}. To assign a probability to accusative case, we must use a sigma-algebra that contains the accusative event.

For the finite examples in this course, taking \mathcal{F}\equiv2^\Omega makes every subset measurable. We still write \mathcal{F} separately because this choice is part of the probability-space specification.

The exact upshot is that \Omega lists outcomes, whereas \mathcal F lists the outcome groupings that can receive probabilities. The next page constructs the smallest such collection forced by a chosen set of linguistic distinctions.

Check your understanding

Start with \{\Omega_4,P_4\}. Which two events must be added to obtain \mathcal{F}_{P_4}?
Is \{\varnothing,\Omega_4\} a sigma-algebra? Check all three requirements.
Why does \{\textit{them}\}\subseteq\Omega_4 not guarantee that \{\textit{them}\}\in\mathcal{F}_{P_4}?
Use De Morgan’s law to explain why closure under complements and unions gives closure under intersections.

--- title: "Sigma-algebras" --- Suppose that our pronoun model assigns a probability to the event that a form is plural. Which other events must the model be able to measure? It must assign a probability to the event that the form is not plural, since that event is the complement of the first. If the model assigns probabilities to two events, it must also assign a probability to their union. The need to keep these operations available is the **closure problem**. Basically, we solve the closure problem by collecting events in a way that never strands us without a complement or required union. More specifically, a [**sigma-algebra**](https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_%28Siegrist%29/01%3A_Foundations/1.11%3A_Measurable_Spaces), also called an event space, is a collection of events that contains the sample space and is closed under complements and countable unions. The model assigns probabilities only to events in this collection, which we write as $\mathcal{F}$. ## The person event space For the twelve-form pronoun space on the [preceding page](events.qmd), let $T_{12}$ be the third-person event. The sigma-algebra generated by this distinction is $$ \mathcal F_{\mathrm{person}} \equiv\sigma(T_{12}) =\{\varnothing,T_{12},T_{12}^c,\Omega\}. $$ It contains exactly the unions of the two atoms $T_{12}$ and $T_{12}^c$. ## The case event space Let $A_{12}$ be the accusative event and $N_{12}$ the nominative event. Since $$ A_{12}\cap N_{12}=\{\textit{you},\textit{it}\}, $$ the two events are not complements. The three nonempty atoms are $$ A_{12}\cap N_{12}, \qquad A_{12}\setminus N_{12}, \qquad N_{12}\setminus A_{12}. $$ Thus $$ \mathcal F_{\mathrm{case}} \equiv\sigma(A_{12},N_{12}) $$ contains the eight unions of these atoms. In particular, closure requires the overlap $A_{12}\cap N_{12}$ and its complement to be measurable. To see why the collection needs structure, use the four outcome sample space $$ \Omega_4 \equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}. $$ Let $P_4$ be the plural event: $$ P_4\equiv\{\textit{they},\textit{them}\}. $$ Its complement is $$ P_4^c=\{\textit{he},\textit{him}\}. $$ If plural number is the only distinction that the model measures, one possible sigma-algebra is $$ \mathcal{F}_{P_4} \equiv\{\varnothing,P_4,P_4^c,\Omega_4\}. $$ Plural and singular are measurable under $\mathcal{F}_{P_4}$. Accusative case is not, because the accusative event does not belong to $\mathcal{F}_{P_4}$. ## Closure requirements A collection $\mathcal{F}$ is a sigma-algebra only if it satisfies three requirements. 1. The complete sample space $\Omega$ belongs to $\mathcal{F}$. 2. If $B$ belongs to $\mathcal{F}$, then its complement $B^c$ also belongs to $\mathcal{F}$. 3. If $B_1,B_2,\ldots$ belong to $\mathcal{F}$, then their union $\bigcup_i B_i$ also belongs to $\mathcal{F}$. The first requirement includes the certain event among the measurable events; the second includes the complement of every measurable event; and the third includes the union of any countable collection of measurable events. Now we can derive consequences one move at a time. Since $\Omega\in\mathcal{F}$, closure under complements gives $$ \Omega^c=\varnothing\in\mathcal{F}. $$ What about intersections, which do not appear as a separate requirement? They follow from complements and unions. De Morgan's law gives $$ B\cap C=(B^c\cup C^c)^c. $$ The right side first takes complements, then a union, and then one more complement, with each move licensed by the stated closure requirements. Thus a sigma-algebra that is closed under complements and countable unions is also closed under intersections. ## A collection that is not a sigma-algebra Consider the proposed collection $$ \mathcal{G}\equiv\{\Omega_4,P_4\}. $$ Check the proposed collection against each requirement. 1. It contains $\Omega_4$, so it passes the first requirement. 2. It contains $P_4$ but not $P_4^c$, so it fails the second requirement. 3. It also lacks $\varnothing$, which must appear as $\Omega_4^c$. Adding $P_4^c$ and $\varnothing$ repairs these failures and produces $\mathcal{F}_{P_4}$. The following R list represents the four events in $\mathcal{F}_{P_4}$. ```{r} #| eval: false omega <- c("he", "him", "they", "them") plural <- c("they", "them") singular <- setdiff(omega, plural) event_space <- list( impossible = character(0), plural = plural, singular = singular, certain = omega ) event_space ``` The list omits the singleton event $\{\textit{them}\}$. This set is a subset of $\Omega_4$, but it is not an event in $\mathcal{F}_{P_4}$. The model represents *them* as an outcome, but it cannot assign a probability to the event containing *them* alone. ## Sample spaces and event spaces 1. $\Omega$ states which individual outcomes exist in the model. 2. $\mathcal{F}$ states which groupings of those outcomes can receive probabilities. For a small finite sample space, we will often use the power set $2^\Omega$, the collection containing every subset of $\Omega$. In that case, every possible grouping of the represented outcomes is measurable. The smaller sigma-algebra $\mathcal{F}_{P_4}$ shows that this choice is not automatic. ## An event outside the sigma-algebra In the example above, the accusative event does not belong to $\mathcal{F}_{P_4}$. We thus cannot write its probability under a probability measure whose domain is $\mathcal{F}_{P_4}$. To assign a probability to accusative case, we must use a sigma-algebra that contains the accusative event. For the finite examples in this course, taking $\mathcal{F}\equiv2^\Omega$ makes every subset measurable. We still write $\mathcal{F}$ separately because this choice is part of the probability-space specification. The exact upshot is that $\Omega$ lists outcomes, whereas $\mathcal F$ lists the outcome groupings that can receive probabilities. The [next page](generating-event-spaces.qmd) constructs the smallest such collection forced by a chosen set of linguistic distinctions. ## Check your understanding 1. Start with $\{\Omega_4,P_4\}$. Which two events must be added to obtain $\mathcal{F}_{P_4}$? 2. Is $\{\varnothing,\Omega_4\}$ a sigma-algebra? Check all three requirements. 3. Why does $\{\textit{them}\}\subseteq\Omega_4$ not guarantee that $\{\textit{them}\}\in\mathcal{F}_{P_4}$? 4. Use De Morgan's law to explain why closure under complements and unions gives closure under intersections.