The chi-squared distribution

The normal distribution page introduced standard normal variables. Suppose we want one nonnegative summary of several standardized discrepancies, with large departures in either direction contributing to the total. What reference distribution results if we square and add independent standard normal variables?

For a concrete calculation, suppose three discrepancies have values

Z_1=1, \qquad Z_2=-.5, \qquad Z_3=2.

These are hypothetical standardized numbers. Treating real annotation discrepancies as independent standard normal variables would be an additional measurement model, not a consequence of calling them annotation errors.

Define their squared sum by

Q\equiv Z_1^2+Z_2^2+Z_3^2.

Basically, squaring removes direction and adding accumulates discrepancy. More specifically, when Z_1,\ldots,Z_\nu are independent standard normal variables, this sum has a chi-squared distribution with \nu degrees of freedom:

Q\sim\chi^2_\nu.

The support is [0,\infty) because a sum of squares cannot be negative.

Calculating one squared discrepancy

For the three constructed values,

\begin{aligned} Q &=1^2+(-.5)^2+2^2\\ &=1+.25+4\\ &=5.25. \end{aligned}

The value 2 makes the largest contribution because squaring gives greater weight to larger standardized departures.

If these were three independent standard normal discrepancies, the reference distribution would have \nu=3.

Reading center and spread from degrees of freedom

For Q\sim\chi^2_\nu,

\mathbb{E}[Q]=\nu

and

\operatorname{Var}(Q)=2\nu.

With three degrees of freedom, the mean is 3 and the variance is 6. The observed value 5.25 lies above the reference mean, but a tail probability is needed to state how much reference mass lies at least that far right.

Calculating the upper tail

The upper tail probability is

\mathbb{P}(Q\geq5.25).

In R,

Code

pchisq(5.25, df = 3, lower.tail = FALSE)

The result is approximately .154. Under the reference assumptions, about .154 of the probability lies on values at least as large as 5.25.

This is a statement about the statistic’s reference distribution. It is not the probability that a null hypothesis is true.

Examining what changing degrees of freedom does

With one degree of freedom, the distribution is strongly right skewed because one squared value often lies near zero but can occasionally be large. Adding more independent squared terms moves the center right and makes the shape less sharply skewed.

Code

q <- seq(0, 20, length.out = 300)

plot(q, dchisq(q, df = 1), type = "l",
     xlab = "squared discrepancy", ylab = "density")
lines(q, dchisq(q, df = 4), lty = 2)
lines(q, dchisq(q, df = 10), lty = 3)

Degrees of freedom determine both center and spread for this family.

Degrees of freedom count unconstrained variation

The symbol \nu does not always count observations directly. It counts independent pieces of variation remaining after constraints.

In an r by c contingency table with row and column totals treated as fixed, the final row and column values are determined by the earlier cells. The number of free cell comparisons is

\nu=(r-1)(c-1).

The chi-squared test page later derives this count for a small linguistic table.

A tail probability is not a hypothesis probability

An upper tail probability is not the probability that a hypothesis is true. The tail calculation conditions on a reference model and asks how often the statistic would be at least as large.

It does not reverse that conditional statement. Interpreting hypotheses requires a separate inferential framework developed later.

Check your understanding

Why can a chi-squared value not be negative?
If \nu=8, what are the mean and variance?
Which constructed discrepancy contributes most to Q=5.25, and why?
State the upper tail probability in words without treating it as a hypothesis probability.

The chi-squared family supplies the random denominator used to define Student’s t distribution on the next page.

--- title: "The chi-squared distribution" --- The [normal distribution page](normal-distribution.qmd) introduced standard normal variables. Suppose we want one nonnegative summary of several standardized discrepancies, with large departures in either direction contributing to the total. What reference distribution results if we square and add independent standard normal variables? For a concrete calculation, suppose three discrepancies have values $$ Z_1=1, \qquad Z_2=-.5, \qquad Z_3=2. $$ These are hypothetical standardized numbers. Treating real annotation discrepancies as independent standard normal variables would be an additional measurement model, not a consequence of calling them annotation errors. Define their squared sum by $$ Q\equiv Z_1^2+Z_2^2+Z_3^2. $$ Basically, squaring removes direction and adding accumulates discrepancy. More specifically, when $Z_1,\ldots,Z_\nu$ are independent standard normal variables, this sum has a [**chi-squared distribution**](https://openstax.org/books/introductory-statistics-2e/pages/11-1-facts-about-the-chi-square-distribution) with $\nu$ degrees of freedom: $$ Q\sim\chi^2_\nu. $$ The support is $[0,\infty)$ because a sum of squares cannot be negative. ## Calculating one squared discrepancy For the three constructed values, $$ \begin{aligned} Q &=1^2+(-.5)^2+2^2\\ &=1+.25+4\\ &=5.25. \end{aligned} $$ The value 2 makes the largest contribution because squaring gives greater weight to larger standardized departures. If these were three independent standard normal discrepancies, the reference distribution would have $\nu=3$. ## Reading center and spread from degrees of freedom For $Q\sim\chi^2_\nu$, $$ \mathbb{E}[Q]=\nu $$ and $$ \operatorname{Var}(Q)=2\nu. $$ With three degrees of freedom, the mean is 3 and the variance is 6. The observed value 5.25 lies above the reference mean, but a tail probability is needed to state how much reference mass lies at least that far right. ## Calculating the upper tail The upper tail probability is $$ \mathbb{P}(Q\geq5.25). $$ In R, ```{r} #| eval: false pchisq(5.25, df = 3, lower.tail = FALSE) ``` The result is approximately $.154$. Under the reference assumptions, about $.154$ of the probability lies on values at least as large as 5.25. This is a statement about the statistic's reference distribution. It is not the probability that a null hypothesis is true. ## Examining what changing degrees of freedom does With one degree of freedom, the distribution is strongly right skewed because one squared value often lies near zero but can occasionally be large. Adding more independent squared terms moves the center right and makes the shape less sharply skewed. ```{r} #| eval: false q <- seq(0, 20, length.out = 300) plot(q, dchisq(q, df = 1), type = "l", xlab = "squared discrepancy", ylab = "density") lines(q, dchisq(q, df = 4), lty = 2) lines(q, dchisq(q, df = 10), lty = 3) ``` Degrees of freedom determine both center and spread for this family. ## Degrees of freedom count unconstrained variation The symbol $\nu$ does not always count observations directly. It counts independent pieces of variation remaining after constraints. In an $r$ by $c$ contingency table with row and column totals treated as fixed, the final row and column values are determined by the earlier cells. The number of free cell comparisons is $$ \nu=(r-1)(c-1). $$ The [chi-squared test page](../statistical-inference/chi-squared-test.qmd) later derives this count for a small linguistic table. ## A tail probability is not a hypothesis probability An upper tail probability is not the probability that a hypothesis is true. The tail calculation conditions on a reference model and asks how often the statistic would be at least as large. It does not reverse that conditional statement. Interpreting hypotheses requires a separate inferential framework developed later. ## Check your understanding 1. Why can a chi-squared value not be negative? 2. If $\nu=8$, what are the mean and variance? 3. Which constructed discrepancy contributes most to $Q=5.25$, and why? 4. State the upper tail probability in words without treating it as a hypothesis probability. The chi-squared family supplies the random denominator used to define [Student's t distribution on the next page](student-t-distribution.qmd).