The posterior summaries page used averages and proportions of draws without yet explaining the numerical approximation. Monte Carlo integration supplies that explanation by connecting the expectation of a function to a sample average. Consider a constructed production study in which a token is coded one when a consonant has secondary palatal articulation, meaning that the tongue body is raised toward the hard palate while the consonant retains another primary constriction. Suppose the posterior distribution for the token probability is
\Pi\mid\mathbf{x} \sim \operatorname{Beta}(5,3).
Let \Pi:\Omega\to[0,1] be the random variable that maps each outcome \omega to a token probability. We want the posterior probability of the event
B_\Pi \equiv \{\omega\in\Omega\mid\Pi(\omega)>.60\}.
Write this probability as I\equiv\mathbb{P}(B_\Pi\mid\mathbf{x}). It is an integral of the posterior density. The question is why an average of simulated values can stand in for that integral. Monte Carlo integration makes the connection using random draws from the target distribution.
Define the indicator \mathbf{1}_{B_\Pi}:\Omega\to\{0,1\} by
\mathbf{1}_{B_\Pi}(\omega) = \begin{cases} 1 & \text{if }\omega\in B_\Pi,\\ 0 & \text{if }\omega\notin B_\Pi. \end{cases}
For integration over values of \Pi, define g:[0,1]\to\{0,1\} by g(\pi)=1 when \pi>.60 and g(\pi)=0 otherwise. Thus g(\Pi(\omega))=\mathbf{1}_{B_\Pi}(\omega). Its posterior expectation is
\begin{aligned} \mathbb{E}[g(\Pi)\mid\mathbf{x}] &= \int_0^1 g(\pi) f_{\Pi\mid\mathbf X}(\pi\mid\mathbf{x}) \,\mathrm{d}\pi\\ &= \mathbb{P}(B_\Pi\mid\mathbf{x}). \end{aligned}
Thus a probability can be calculated as the expectation of an indicator.
Draw
\pi^{(1)},\ldots,\pi^{(S)} \overset{\mathrm{iid}}{\sim} \operatorname{Beta}(5,3).
The Monte Carlo estimator is
\widehat{I}_S = \frac{1}{S} \sum_{s=1}^S g(\pi^{(s)}).
It equals the proportion of posterior draws above .60.
set.seed(642)
draw_count <- 50000
pi_draw <- rbeta(draw_count, shape1 = 5, shape2 = 3)
mc_estimate <- mean(pi_draw > .60)
exact_value <- 1 - pbeta(.60, shape1 = 5, shape2 = 3)
stopifnot(abs(mc_estimate - exact_value) < .01)
c(Monte_Carlo = mc_estimate, exact = exact_value)The beta cumulative distribution function supplies an exact benchmark for this simple target. Complicated posteriors may not provide such a benchmark, which is why the simulation calculation matters.
For a continuous target with density f(\theta) and an integrable function g,
I = \mathbb{E}_{f}[g(\Theta)] = \int g(\theta)f(\theta)\,\mathrm{d}\theta.
Independent target draws give
\widehat{I}_S = \frac{1}{S} \sum_{s=1}^Sg(\theta^{(s)}).
For a discrete target with PMF p(\theta), the expectation is a sum rather than an integral, but the same sample average applies. We use f for a continuous density and p for a discrete PMF throughout.
For a posterior mean, use g(\theta)=\theta. For a posterior second moment, use g(\theta)=\theta^2. For an interval endpoint, use the corresponding empirical quantile rather than a simple average.
The bootstrap page first distinguished simulation error from sampling uncertainty. In the present setting, variation caused by using a finite number S of simulation draws is Monte Carlo error. For independent draws and a binary indicator, an estimated Monte Carlo standard error is
\widehat{\operatorname{MCSE}}(\widehat{I}_S) = \sqrt{ \frac{ \widehat{I}_S(1-\widehat{I}_S) }{S} }.
mcse <- sqrt(mc_estimate * (1 - mc_estimate) / draw_count)
stopifnot(mcse < .003)
c(estimate = mc_estimate, MCSE = mcse)Multiplying S by four tends to divide MCSE by two. This square-root rate is the same rate that appears in sampling standard errors, but the source of variation differs.
The law of large numbers supplies the basic justification. As S grows, the average of independent target draws tends to approach the target expectation when that expectation exists. When g(\Theta) also has finite variance, a central limit theorem says that the distribution of the standardized average tends toward a normal distribution. This result motivates the approximate normal distribution for the average and the MCSE calculation above.
This result does not guarantee that every finite simulation is precise enough. Monte Carlo error should be small relative to posterior uncertainty and the precision used in substantive claims. Reporting six decimal places when MCSE is .003 gives false numerical precision.
The posterior spread describes uncertainty about the palatalization probability under the model and observed data. Monte Carlo error describes uncertainty in a numerical approximation to a posterior summary.
Increasing S reduces Monte Carlo error. It does not add speakers, tokens, or elicitation contexts, so it cannot narrow the target posterior. More simulation draws are not more linguistic evidence.
Setting a seed makes the teaching calculation reproducible. Changing the seed provides a simple stability check. Agreement across seeds is evidence about simulation error only; it does not check the likelihood or prior.