Discrete random variables

The formal definition of a random variable requires a measurable mapping from outcomes to values. We now ask a different question: can the possible values be listed one at a time? The answer separates discrete variables from the continuous variables introduced later.

Suppose L maps each utterance outcome to its number of word tokens. We declare the codomain \operatorname{cod}(L)=\mathbb Z_{\geq0}, whose values begin

0,1,2,3,\ldots

There may be no fixed largest utterance length in the model, but the codomain values can be placed in a list. This property makes L a discrete random variable.

Here word token means a unit produced by a declared tokenization scheme. Word segmentation is not identical across languages or annotation projects, so a real analysis must document the rule used to obtain the count (see the Universal Dependencies tokenization guidelines).

The codomain declares which values the function may return. The support is the subset that receives positive probability:

\operatorname{supp}(X) \equiv\{x\in\operatorname{cod}(X):\mathbb{P}(X=x)>0\}.

Here \operatorname{cod}(X) is the declared value space, following the notation introduced on the preceding page. For the utterance-length model, the support might equal the full codomain, but it need not. A model could assign zero probability to some declared codomain values.

More specifically, a random variable X is discrete when some finite or countably infinite set \mathcal S satisfies \mathbb{P}(X\in\mathcal S)=1. In this module, we declare a countable codomain for a discrete variable and sum over x\in\operatorname{cod}(X). Countably infinite means that every value can be assigned a position in a sequence, even though the sequence has no final value.

Comparing finite and countably infinite support

An annotation variable P might record grammatical person:

\operatorname{supp}(P) =\{\mathrm{first},\mathrm{second},\mathrm{third}\}.

This support is finite. The utterance length support is countably infinite. Both variables are discrete because their values occur as separate, listable possibilities.

The support values need not be numerical. Grammatical person is discrete even when represented with linguistic labels. Numerical coding may be convenient in software, but it does not create discreteness.

Values define separate events

The statement L=5 corresponds to the event

\{\omega\in\Omega\mid L(\omega)=5\},

which contains every five word utterance outcome. The statement L\geq5 corresponds to

\{\omega\in\Omega\mid L(\omega)\in\{5,6,7,\ldots\}\}.

For a discrete variable, each support value can be considered separately. A single value such as L=5 may receive positive probability.

Working through a finite sample

Suppose a conversation excerpt contains utterance lengths

2,5,1,5,3.

The observed values are \{1,2,3,5\}, but the modeled support may still include 0, 4, 6, and larger integers. An observed sample does not determine the full support.

The event L\geq3 contains the third, fifth, and larger support values, but in this sample it selects the observed lengths 5, 5, and 3.

Code

utterance_length <- c(2, 5, 1, 5, 3)

sort(unique(utterance_length))
utterance_length[utterance_length >= 3]

The first line reports observed distinct values. It should not be interpreted as a complete statement of modeled support.

Discrete is not the same as unordered

Grammatical person is often treated as a set of categories without a numerical distance between them. Utterance length is ordered, and differences such as 5-3=2 have an interpretation in words. Both variables are discrete.

Thus discreteness concerns whether the probability-carrying values are separate and listable. It does not determine whether the codomain is ordered or whether arithmetic on its labels is meaningful.

Observed values do not determine the complete support

The distinct values in one sample need not exhaust the modeled support. If the longest utterance in a small sample contains five words, that fact does not imply that \operatorname{supp}(L)=\{0,1,2,3,4,5\}.

The support is a modeling statement about possible values. The sample reports which of those values happened to occur in the observed records.

Check your understanding

Is the number of relative clauses in a sentence finite or countably infinite under a model with no fixed sentence length limit?
Why is a part of speech variable discrete even when its values are not numbers?
Give one discrete ordered variable and one discrete unordered variable from linguistics.
Explain why the largest observed value does not automatically determine the upper end of the support.

The codomain declares the values to which a PMF assigns mass, including zero mass. The next page defines that assignment.

--- title: "Discrete random variables" --- The [formal definition of a random variable](defining-random-variables.qmd) requires a measurable mapping from outcomes to values. We now ask a different question: can the possible values be listed one at a time? The answer separates discrete variables from the continuous variables introduced later. Suppose $L$ maps each utterance outcome to its number of word tokens. We declare the codomain $\operatorname{cod}(L)=\mathbb Z_{\geq0}$, whose values begin $$ 0,1,2,3,\ldots $$ There may be no fixed largest utterance length in the model, but the codomain values can be placed in a list. This property makes $L$ a [**discrete random variable**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html). Here *word token* means a unit produced by a declared tokenization scheme. Word segmentation is not identical across languages or annotation projects, so a real analysis must document the rule used to obtain the count (see the [Universal Dependencies tokenization guidelines](https://universaldependencies.org/u/overview/tokenization.html)). The codomain declares which values the function may return. The [**support**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) is the subset that receives positive probability: $$ \operatorname{supp}(X) \equiv\{x\in\operatorname{cod}(X):\mathbb{P}(X=x)>0\}. $$ Here $\operatorname{cod}(X)$ is the declared value space, following the [notation introduced on the preceding page](defining-random-variables.qmd#distinguishing-the-two-spaces). For the utterance-length model, the support might equal the full codomain, but it need not. A model could assign zero probability to some declared codomain values. More specifically, a random variable $X$ is discrete when some finite or countably infinite set $\mathcal S$ satisfies $\mathbb{P}(X\in\mathcal S)=1$. In this module, we declare a countable codomain for a discrete variable and sum over $x\in\operatorname{cod}(X)$. *Countably infinite* means that every value can be assigned a position in a sequence, even though the sequence has no final value. ## Comparing finite and countably infinite support An annotation variable $P$ might record grammatical person: $$ \operatorname{supp}(P) =\{\mathrm{first},\mathrm{second},\mathrm{third}\}. $$ This support is finite. The utterance length support is countably infinite. Both variables are discrete because their values occur as separate, listable possibilities. The support values need not be numerical. Grammatical person is discrete even when represented with linguistic labels. Numerical coding may be convenient in software, but it does not create discreteness. ## Values define separate events The statement $L=5$ corresponds to the event $$ \{\omega\in\Omega\mid L(\omega)=5\}, $$ which contains every five word utterance outcome. The statement $L\geq5$ corresponds to $$ \{\omega\in\Omega\mid L(\omega)\in\{5,6,7,\ldots\}\}. $$ For a discrete variable, each support value can be considered separately. A single value such as $L=5$ may receive positive probability. ## Working through a finite sample Suppose a conversation excerpt contains utterance lengths $$ 2,5,1,5,3. $$ The observed values are $\{1,2,3,5\}$, but the modeled support may still include $0$, $4$, $6$, and larger integers. An observed sample does not determine the full support. The event $L\geq3$ contains the third, fifth, and larger support values, but in this sample it selects the observed lengths 5, 5, and 3. ```{r} #| eval: false utterance_length <- c(2, 5, 1, 5, 3) sort(unique(utterance_length)) utterance_length[utterance_length >= 3] ``` The first line reports observed distinct values. It should not be interpreted as a complete statement of modeled support. ## Discrete is not the same as unordered Grammatical person is often treated as a set of categories without a numerical distance between them. Utterance length is ordered, and differences such as $5-3=2$ have an interpretation in words. Both variables are discrete. Thus discreteness concerns whether the probability-carrying values are separate and listable. It does not determine whether the codomain is ordered or whether arithmetic on its labels is meaningful. ## Observed values do not determine the complete support The distinct values in one sample need not exhaust the modeled support. If the longest utterance in a small sample contains five words, that fact does not imply that $\operatorname{supp}(L)=\{0,1,2,3,4,5\}$. The support is a modeling statement about possible values. The sample reports which of those values happened to occur in the observed records. ## Check your understanding 1. Is the number of relative clauses in a sentence finite or countably infinite under a model with no fixed sentence length limit? 2. Why is a part of speech variable discrete even when its values are not numbers? 3. Give one discrete ordered variable and one discrete unordered variable from linguistics. 4. Explain why the largest observed value does not automatically determine the upper end of the support. The codomain declares the values to which a PMF assigns mass, including zero mass. The [next page](probability-mass-functions.qmd) defines that assignment.