From linguistic records to probabilities

Statistical analysis begins with a decision about what was observed. A recording of a speaker saying heed could contribute one row for the recording, one row for the word token, one row for the vowel token, or several rows for acoustic measurements taken during the vowel. These rows represent different observations, even when they were extracted from the same recording.

The same issue arises with every kind of linguistic data. A corpus can be divided into documents, sentences, word tokens, or word types. A reading-time experiment can retain one row for each trial or average those trials by participant or sentence. Before we count anything, we need to know what one row represents and which differences among speakers, items, and tokens remain in the data.

Linguistic data and comparisons

We begin by turning linguistic records into data. A ten-minute interview is one audio file, but it contains many word and vowel tokens. Once we distinguish the file from the observations extracted from it, we can state what each row represents, which columns identify it, and which speakers or words occur in more than one row.

We then turn a research question into a comparison. If an aspectually mismatched sentence takes 45 milliseconds longer to read than a matched sentence, we need to know which response was measured, how the conditions differed, and whether participants and sentence items occurred in both conditions. These facts determine which difference we have estimated.

We then work through Zipf’s law by counting word tokens, grouping them into types, ordering the types by frequency, and comparing the observed counts with an inverse rank function. A rank frequency function may describe a corpus well, but that fit does not tell us which process produced the corpus.

Probability

Probability theory requires us to specify the possible results before assigning probabilities to them. A sample space is the set of possible outcomes represented by the model. An event is a set of those outcomes. If the outcomes are the forms he, him, they, and them, for instance, the plural event contains they and them.

We do not always assign probabilities to every subset of the sample space. A sigma-algebra specifies which events are measurable. We will construct the sigma-algebra generated by number and case for the four pronoun forms, then define a probability measure that assigns a probability to each measurable event. The sample space, sigma-algebra, and probability measure form the probability space

(\Omega,\mathcal{F},\mathbb{P}).

Once the probability space is defined, we can calculate probabilities for relations among events. We begin with mutually exclusive events, whose intersection is empty, and introduce the standard comma notation for joint probabilities. We then define conditional probability by restricting probability to a conditioning event. The chain rule follows by rearranging that definition, and Bayes’ rule follows by factoring the same joint probability in two orders. We conclude by defining independence as invariance under conditioning and deriving the equivalent product factorization.

The chapter dependency graph shows which later chapters use these definitions. Start with turning linguistic records into data.

--- title: "From linguistic records to probabilities" --- Statistical analysis begins with a decision about what was observed. A recording of a speaker saying *heed* could contribute one row for the recording, one row for the word token, one row for the vowel token, or several rows for acoustic measurements taken during the vowel. These rows represent different observations, even when they were extracted from the same recording. The same issue arises with every kind of linguistic data. A corpus can be divided into documents, sentences, word tokens, or word types. A reading-time experiment can retain one row for each trial or average those trials by participant or sentence. Before we count anything, we need to know what one row represents and which differences among speakers, items, and tokens remain in the data. ## Linguistic data and comparisons We begin by [turning linguistic records into data](linguistic-data.qmd). A ten-minute interview is one audio file, but it contains many word and vowel tokens. Once we distinguish the file from the observations extracted from it, we can state what each row represents, which columns identify it, and which speakers or words occur in more than one row. We then [turn a research question into a comparison](making-sense-of-data.qmd). If an aspectually mismatched sentence takes 45 milliseconds longer to read than a matched sentence, we need to know which response was measured, how the conditions differed, and whether participants and sentence items occurred in both conditions. These facts determine which difference we have estimated. We then work through [Zipf's law](zipfs-law.qmd) by counting word tokens, grouping them into types, ordering the types by frequency, and comparing the observed counts with an inverse rank function. A rank frequency function may describe a corpus well, but that fit does not tell us which process produced the corpus. ## Probability Probability theory requires us to specify the possible results before assigning probabilities to them. A [sample space](possibilities-and-events.qmd) is the set of possible outcomes represented by the model. An [event](events.qmd) is a set of those outcomes. If the outcomes are the forms *he*, *him*, *they*, and *them*, for instance, the plural event contains *they* and *them*. We do not always assign probabilities to every subset of the sample space. A [sigma-algebra](event-spaces.qmd) specifies which events are measurable. We will construct the [sigma-algebra generated by number and case](generating-event-spaces.qmd) for the four pronoun forms, then define a [probability measure](probability-measures.qmd) that assigns a probability to each measurable event. The sample space, sigma-algebra, and probability measure form the probability space $$ (\Omega,\mathcal{F},\mathbb{P}). $$ Once the probability space is defined, we can calculate probabilities for relations among events. We begin with [mutually exclusive events](probability-concepts.qmd), whose intersection is empty, and introduce the standard comma notation for [joint probabilities](joint-probability-of-events.qmd). We then define [conditional probability](conditional-probability.qmd) by restricting probability to a conditioning event. The [chain rule](probability-factorization.qmd) follows by rearranging that definition, and [Bayes' rule](bayes-rule.qmd) follows by factoring the same joint probability in two orders. We conclude by defining [independence](independence.qmd) as invariance under conditioning and deriving the equivalent product factorization. The [chapter dependency graph](../learning-path.qmd) shows which later chapters use these definitions. Start with [turning linguistic records into data](linguistic-data.qmd).