Suppose that we want to assign probabilities to two distinctions: whether a pronoun is plural and whether it is accusative. The sigma-algebra must contain both events, their complements, and every union required by closure.
Recall from event spaces that a sigma-algebra is a collection of measurable events. If P is the plural event and A is the accusative event, then
\sigma(P,A)
denotes the smallest sigma-algebra containing both P and A. The collection \{P,A\} is a generating set, and \sigma(P,A) is the sigma-algebra generated by that set.
Why do we ask for the smallest sigma-algebra? Any sigma-algebra containing P and A must also contain their complements, intersections, and countable unions. The smallest one contains the events forced by these closure requirements and no additional distinctions.
Use the four outcome sample space
\Omega_4 \equiv\{\textit{he},\textit{him},\textit{they},\textit{them}\}.
Define the generators
P\equiv\{\textit{they},\textit{them}\}
and
A\equiv\{\textit{him},\textit{them}\}.
The two distinctions cross. A form may be plural or singular, and it may be accusative or nominative.
For a finite space, intersect each generator or its complement. The nonempty intersections are
\begin{aligned} P\cap A &= \{\textit{them}\},\\ P\cap A^c &= \{\textit{they}\},\\ P^c\cap A &= \{\textit{him}\},\\ P^c\cap A^c &= \{\textit{he}\}. \end{aligned}
These four sets are the atoms produced by the generators. They do not overlap, and every outcome belongs to exactly one of them.
Every event constructed from P, A, and the set operations must treat all outcomes within an atom alike. No operation available from the generators can separate two outcomes that have identical membership in every generator.
In this small example, each atom contains one outcome. Every subset of \Omega_4 can thus be written as a union of atoms. It follows that
\sigma(P,A)=2^{\Omega_4}.
Since there are four atoms, there are 2^4=16 unions of atoms, including the empty union \varnothing and the union of all four atoms \Omega_4.
The finite set operations can be reproduced directly.
omega <- c("he", "him", "they", "them")
plural <- c("they", "them")
accusative <- c("him", "them")
plural_acc <- intersect(plural, accusative)
plural_nom <- intersect(plural, setdiff(omega, accusative))
singular_acc <- intersect(setdiff(omega, plural), accusative)
singular_nom <- intersect(setdiff(omega, plural),
setdiff(omega, accusative))
list(plural_acc, plural_nom, singular_acc, singular_nom)R returns the four singleton atoms listed above.
Now generate a sigma-algebra from P alone. Its two atoms are
P=\{\textit{they},\textit{them}\}
and
P^c=\{\textit{he},\textit{him}\}.
Every event in \sigma(P) is a union of these two atoms. The generated sigma-algebra is thus
\sigma(P)=\{\varnothing,P,P^c,\Omega_4\}.
This space cannot distinguish they from them. Both outcomes have the same membership in the only generator. Adding A splits each number atom by case, producing the four smaller atoms above.
With P as the only generator, case is not measurable. Adding A makes both number and case measurable.
If two outcomes have identical membership in every generator, no event in the generated sigma-algebra can separate them. For instance, suppose a model uses only P to generate its sigma-algebra but later asks whether plural pronouns are nominative or accusative. The outcomes they and them occupy the same atom, so the model cannot represent that case distinction.
Even a corpus containing one million tokens would not put a case event in \sigma(P). We must add a case generator before assigning a probability to that event.
For the finite spaces used here, the procedure is mechanical.
The resulting collection is the smallest sigma-algebra containing the generators.