Discrete random variables

The formal definition of a random variable requires a measurable mapping from outcomes to values. We now classify that variable by the possible values in its distribution.

Suppose L maps each utterance outcome to its number of word tokens. The possible values begin

0,1,2,3,\ldots

There may be no fixed largest utterance length in the model, but the possible values can be placed in a list. This property makes L a discrete random variable.

For a discrete random variable, define its support as

\operatorname{supp}(X) \equiv\{x\in\mathcal X:\mathbb{P}(X=x)>0\}.

For the utterance-length model, \operatorname{supp}(L)=\{0,1,2,3,\ldots\}.

A random variable X is discrete when some finite or countably infinite set \mathcal S satisfies \mathbb{P}(X\in\mathcal S)=1. Equivalently, its probability is concentrated on a countable support. Countably infinite means that every support value can be assigned a position in a sequence, even though the sequence has no final value.

Comparing finite and countably infinite support

An annotation variable P might record grammatical person:

\operatorname{supp}(P) =\{\textsc{first},\textsc{second},\textsc{third}\}.

This support is finite. The utterance length support is countably infinite. Both variables are discrete because their values occur as separate, listable possibilities.

The support values need not be numerical. Grammatical person is discrete even when represented with linguistic labels. Numerical coding may be convenient in software, but it does not create discreteness.

Values define separate events

The statement L=5 corresponds to the event

\{\omega\in\Omega:L(\omega)=5\},

which contains every five word utterance outcome. The statement L\geq5 corresponds to

\{\omega\in\Omega:L(\omega)\in\{5,6,7,\ldots\}\}.

For a discrete variable, each support value can be considered separately. A single value such as L=5 may receive positive probability.

Working through a finite sample

Suppose a conversation excerpt contains utterance lengths

2,5,1,5,3.

The observed values are \{1,2,3,5\}, but the modeled support may still include 0, 4, 6, and larger integers. An observed sample does not determine the full support.

The event L\geq3 contains the third, fifth, and larger support values, but in this sample it selects the observed lengths 5, 5, and 3.

Code

utterance_length <- c(2, 5, 1, 5, 3)

sort(unique(utterance_length))
utterance_length[utterance_length >= 3]

The first line reports observed distinct values. It should not be interpreted as a complete statement of modeled support.

Discrete is not the same as unordered

Grammatical person is often treated as a set of categories without a numerical distance between them. Utterance length is ordered, and differences such as 5-3=2 have an interpretation in words. Both variables are discrete.

Thus discreteness concerns whether the support values are separate and listable. It does not determine whether they are ordered or whether arithmetic on their labels is meaningful.

Observed values do not determine the complete support

The distinct values in one sample need not exhaust the modeled support. If the longest utterance in a small sample contains five words, that fact does not imply that \operatorname{supp}(L)=\{0,1,2,3,4,5\}.

The support is a modeling statement about possible values. The sample reports which of those values happened to occur in the observed records.

Check your understanding

Is the number of relative clauses in a sentence finite or countably infinite under a model with no fixed sentence length limit?
Why is a part of speech variable discrete even when its values are not numbers?
Give one discrete ordered variable and one discrete unordered variable from linguistics.
Explain why the largest observed value does not automatically determine the upper end of the support.

The support lists the separate values. The next page assigns probability to those values.

--- title: "Discrete random variables" --- The [formal definition of a random variable](defining-random-variables.qmd) requires a measurable mapping from outcomes to values. We now classify that variable by the possible values in its distribution. Suppose $L$ maps each utterance outcome to its number of word tokens. The possible values begin $$ 0,1,2,3,\ldots $$ There may be no fixed largest utterance length in the model, but the possible values can be placed in a list. This property makes $L$ a [**discrete random variable**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html). For a discrete random variable, define its [**support**](https://bruno.nicenboim.me/bayescogsci/ch-intro.html) as $$ \operatorname{supp}(X) \equiv\{x\in\mathcal X:\mathbb{P}(X=x)>0\}. $$ For the utterance-length model, $\operatorname{supp}(L)=\{0,1,2,3,\ldots\}$. A random variable $X$ is discrete when some finite or countably infinite set $\mathcal S$ satisfies $\mathbb{P}(X\in\mathcal S)=1$. Equivalently, its probability is concentrated on a countable support. *Countably infinite* means that every support value can be assigned a position in a sequence, even though the sequence has no final value. ## Comparing finite and countably infinite support An annotation variable $P$ might record grammatical person: $$ \operatorname{supp}(P) =\{\textsc{first},\textsc{second},\textsc{third}\}. $$ This support is finite. The utterance length support is countably infinite. Both variables are discrete because their values occur as separate, listable possibilities. The support values need not be numerical. Grammatical person is discrete even when represented with linguistic labels. Numerical coding may be convenient in software, but it does not create discreteness. ## Values define separate events The statement $L=5$ corresponds to the event $$ \{\omega\in\Omega:L(\omega)=5\}, $$ which contains every five word utterance outcome. The statement $L\geq5$ corresponds to $$ \{\omega\in\Omega:L(\omega)\in\{5,6,7,\ldots\}\}. $$ For a discrete variable, each support value can be considered separately. A single value such as $L=5$ may receive positive probability. ## Working through a finite sample Suppose a conversation excerpt contains utterance lengths $$ 2,5,1,5,3. $$ The observed values are $\{1,2,3,5\}$, but the modeled support may still include $0$, $4$, $6$, and larger integers. An observed sample does not determine the full support. The event $L\geq3$ contains the third, fifth, and larger support values, but in this sample it selects the observed lengths 5, 5, and 3. ```{r} #| eval: false utterance_length <- c(2, 5, 1, 5, 3) sort(unique(utterance_length)) utterance_length[utterance_length >= 3] ``` The first line reports observed distinct values. It should not be interpreted as a complete statement of modeled support. ## Discrete is not the same as unordered Grammatical person is often treated as a set of categories without a numerical distance between them. Utterance length is ordered, and differences such as $5-3=2$ have an interpretation in words. Both variables are discrete. Thus discreteness concerns whether the support values are separate and listable. It does not determine whether they are ordered or whether arithmetic on their labels is meaningful. ## Observed values do not determine the complete support The distinct values in one sample need not exhaust the modeled support. If the longest utterance in a small sample contains five words, that fact does not imply that $\operatorname{supp}(L)=\{0,1,2,3,4,5\}$. The support is a modeling statement about possible values. The sample reports which of those values happened to occur in the observed records. ## Check your understanding 1. Is the number of relative clauses in a sentence finite or countably infinite under a model with no fixed sentence length limit? 2. Why is a part of speech variable discrete even when its values are not numbers? 3. Give one discrete ordered variable and one discrete unordered variable from linguistics. 4. Explain why the largest observed value does not automatically determine the upper end of the support. The support lists the separate values. The [next page](probability-mass-functions.qmd) assigns probability to those values.