Concept and reasoning
Standard deviation (SD) describes spread of observations within a sample. A sampling distribution describes how an estimator such as the sample mean would vary across repeated samples. Its standard deviation is the standard error (SE).
For independent identically distributed observations with finite variance, Var(X̄)=σ²/n and SE(X̄)=σ/√n. When population σ is unknown, estimate it with s/√n. SD and SE answer different questions.
Under suitable independence and finite-variance conditions, the central limit theorem gives an approximately normal standardised sample mean for sufficiently large n. It does not imply raw observations are normal or every small-sample mean is approximately normal.
The law of large numbers concerns convergence of the sample mean toward the population mean; the CLT concerns the limiting shape of scaled sampling error. More data can reduce random error but do not automatically fix systematic selection bias.
Formula
Var(X̄)=σ²/n SE(X̄)=σ/√n (X̄−μ)/(σ/√n) ≈ N(0,1)
Worked example
- Let population SD be σ=12 and sample size n=25. The mean's SE is 12/√25=2.4.
- At n=100 with the same SD, SE=12/√100=1.2. Quadrupling n halves SE.
- Individual observations may still have SD 12 in both situations; it is uncertainty about the sample mean that changes.
Independent exercises
- With σ=15 and n=100, what is SE(X̄)?
- With fixed σ, how does SE change as n goes from 36 to 144?
- Why does the CLT not require raw data to follow a normal distribution?
Written solutions stay in private instructor notes.
Base R verification
set.seed(2026)
pop_sd <- 12
se25 <- pop_sd/sqrt(25)
se100 <- pop_sd/sqrt(100)
c(se25=se25, se100=se100)
# Compare mean of skewed exponential samples with normal approximation
means <- replicate(5000, mean(rexp(50, rate=1)))
hist(means, main='Sampling distribution of a mean')
c(mean=mean(means), standard_error=sd(means))