The negative binomial distribution

The geometric distribution counted the draw on which the first target occurs. Subtracting one gives the number of failures before that target. Suppose we continue sampling corpus tokens until we encounter the third loanword. Let K record the number of nonloanwords observed before that third target.

The negative binomial distribution represents this waiting count when each draw is independent and has the same target probability \pi.

Let r\in\{1,2,\ldots\} be the target number of loanwords. We write

K\sim\operatorname{NegBin}(r,\pi).

Under the failures convention,

\operatorname{supp}(K)=\{0,1,2,3,\ldots\}.

The value zero is possible because the first r draws might all be targets.

Examining why the final draw is fixed

Take r=3 and K=2. Every qualifying sequence contains three target outcomes, written S, and two nonloanwords, written N. The final draw must be the third target. The earlier four positions contain two targets and two nonloanwords.

The possible sequences are

NNSSS, \quad NSNSS, \quad NSSNS, \quad SNNSS, \quad SNSNS, \quad SSNNS.

There are six sequences. This count equals

{2+3-1\choose2}={4\choose2}=6.

The final target is reserved, and the failures can occupy any two of the first four positions.

Deriving the PMF

For general r and k, every qualifying sequence has r targets and k nonloanwords. Its probability is

\pi^r(1-\pi)^k.

The number of sequences is

{k+r-1\choose k}.

Multiplying arrangement count by sequence probability gives

p_K(k) \equiv\mathbb{P}(K=k) ={k+r-1\choose k}(1-\pi)^k\pi^r, \qquad k\in\{0,1,2,\ldots\}.

Calculating one waiting probability

Let r=3, \pi=.20, and k=4. Then

\begin{aligned} p_K(4) &={6\choose4}(.80)^4(.20)^3\\ &=15(.4096)(.008)\\ &=.049152. \end{aligned}

This is the probability of observing exactly four nonloanwords before the third loanword.

Base R uses the same failures convention:

Code

dnbinom(4, size = 3, prob = .20)

The argument size specifies the target number of successes, and the first argument specifies failures before that target.

Reading the mean and variance

For this parameterization,

\mathbb{E}[K] =r\frac{1-\pi}{\pi}

and

\operatorname{Var}(K) =r\frac{1-\pi}{\pi^2}.

With r=3 and \pi=.20,

\mathbb{E}[K] =3\frac{.80}{.20} =12

and

\operatorname{Var}(K) =3\frac{.80}{.20^2} =60.

The family allows a long right tail because many failures may occur before the required target count is reached.

Recovering the first target case

Set r=1. The arrangement count becomes one:

{k\choose k}=1.

The PMF reduces to

p_K(k)=(1-\pi)^k\pi,

which is the geometric waiting process under the failures before first target convention.

Checking the parameterization

Negative binomial distributions can count failures, total draws, or target events. These parameterizations cannot be mixed. If K counts failures before target r, then the total number of draws is

T\equiv K+r.

Some texts or software model T instead. Others use a mean and dispersion parameter for the same family. A numerical parameter is interpretable only after the counted quantity and convention have been declared.

Check your understanding

List every sequence with one failure before the third target.
With r=2 and \pi=.25, calculate p_K(3).
What total draw count corresponds to K=4 and r=3?
Explain why setting r=1 recovers the geometric waiting process.

The negative binomial family models a count generated by waiting for a fixed target number. The next page models event counts in a declared unit of exposure.

--- title: "The negative binomial distribution" --- The [geometric distribution](geometric-distribution.qmd) counted the draw on which the first target occurs. Subtracting one gives the number of failures before that target. Suppose we continue sampling corpus tokens until we encounter the third loanword. Let $K$ record the number of nonloanwords observed before that third target. The [**negative binomial distribution**](https://online.stat.psu.edu/stat414/lesson/11/11.4) represents this waiting count when each draw is independent and has the same target probability $\pi$. Let $r\in\{1,2,\ldots\}$ be the target number of loanwords. We write $$ K\sim\operatorname{NegBin}(r,\pi). $$ Under the failures convention, $$ \operatorname{supp}(K)=\{0,1,2,3,\ldots\}. $$ The value zero is possible because the first $r$ draws might all be targets. ## Examining why the final draw is fixed Take $r=3$ and $K=2$. Every qualifying sequence contains three target outcomes, written $S$, and two nonloanwords, written $N$. The final draw must be the third target. The earlier four positions contain two targets and two nonloanwords. The possible sequences are $$ NNSSS, \quad NSNSS, \quad NSSNS, \quad SNNSS, \quad SNSNS, \quad SSNNS. $$ There are six sequences. This count equals $$ {2+3-1\choose2}={4\choose2}=6. $$ The final target is reserved, and the failures can occupy any two of the first four positions. ## Deriving the PMF For general $r$ and $k$, every qualifying sequence has $r$ targets and $k$ nonloanwords. Its probability is $$ \pi^r(1-\pi)^k. $$ The number of sequences is $$ {k+r-1\choose k}. $$ Multiplying arrangement count by sequence probability gives $$ p_K(k) \equiv\mathbb{P}(K=k) ={k+r-1\choose k}(1-\pi)^k\pi^r, \qquad k\in\{0,1,2,\ldots\}. $$ ## Calculating one waiting probability Let $r=3$, $\pi=.20$, and $k=4$. Then $$ \begin{aligned} p_K(4) &={6\choose4}(.80)^4(.20)^3\\ &=15(.4096)(.008)\\ &=.049152. \end{aligned} $$ This is the probability of observing exactly four nonloanwords before the third loanword. Base R uses the same failures convention: ```{r} #| eval: false dnbinom(4, size = 3, prob = .20) ``` The argument `size` specifies the target number of successes, and the first argument specifies failures before that target. ## Reading the mean and variance For this parameterization, $$ \mathbb{E}[K] =r\frac{1-\pi}{\pi} $$ and $$ \operatorname{Var}(K) =r\frac{1-\pi}{\pi^2}. $$ With $r=3$ and $\pi=.20$, $$ \mathbb{E}[K] =3\frac{.80}{.20} =12 $$ and $$ \operatorname{Var}(K) =3\frac{.80}{.20^2} =60. $$ The family allows a long right tail because many failures may occur before the required target count is reached. ## Recovering the first target case Set $r=1$. The arrangement count becomes one: $$ {k\choose k}=1. $$ The PMF reduces to $$ p_K(k)=(1-\pi)^k\pi, $$ which is the geometric waiting process under the failures before first target convention. ## Checking the parameterization Negative binomial distributions can count failures, total draws, or target events. These parameterizations cannot be mixed. If $K$ counts failures before target $r$, then the total number of draws is $$ T\equiv K+r. $$ Some texts or software model $T$ instead. Others use a mean and dispersion parameter for the same family. A numerical parameter is interpretable only after the counted quantity and convention have been declared. ## Check your understanding 1. List every sequence with one failure before the third target. 2. With $r=2$ and $\pi=.25$, calculate $p_K(3)$. 3. What total draw count corresponds to $K=4$ and $r=3$? 4. Explain why setting $r=1$ recovers the geometric waiting process. The negative binomial family models a count generated by waiting for a fixed target number. The [next page](poisson-distribution.qmd) models event counts in a declared unit of exposure.