Confidence Intervals for a binomial proportion and asymptotic expansions

Lawrence D. Brown, Tommaso Cai, Anirban DasguptaThe Annals of Statistics 2002 · DOI

Abstract

We address the classic problem of interval estimation of a binomial proportion. The Wald interval $\hat{p}\pm z_{\alpha/2} n^{-1/2} (\hat{p} (1 - \hat{p}))^{1/2}$ is currently in near universal use. We first show that the coverage properties of the Wald interval are persistently poor and defy virtually all conventional wisdom. We then proceed to a theoretical comparison of the standard interval and four additional alternative intervals by asymptotic expansions of their coverage probabilities and expected lengths. The four additional interval methods we study in detail are the score-test interval (Wilson), the likelihood-ratio-test interval, a Jeffreys prior Bayesian interval and an interval suggested by Agresti and Coull. The asymptotic expansions for coverage show that the first three of these alternative methods have coverages that fluctuate about the nominal value, while the Agresti–Coull interval has a somewhat larger and more nearly conservative coverage function. For the five interval methods we also investigate asymptotically their average coverage relative to distributions for $p$ supported within (0 1) . In terms of expected length, asymptotic expansions show that the Agresti–Coull interval is always the longest of these. The remaining three are rather comparable and are shorter than the Wald interval except for $p$ near 0 or 1. These analytical calculations support and complement the findings and the recommendations in Brown, Cai and DasGupta (Statist. Sci. (2001) 16 101–133).

Introduction

In introductory statistics, the Wald interval is presented as the gold standard for estimating a binomial proportion $p$. Defined as $\hat{p} \pm z\sqrt{\hat{p}\hat{q}/n}$, it relies on the normal approximation to the binomial distribution. However, this paper demonstrates that the Wald interval is fundamentally flawed, exhibiting erratic coverage behavior that persists even in large samples.

The Failure of the Wald Interval

The Wald interval suffers from two primary defects: erratic oscillations and systematic bias. Conventional wisdom suggests that the interval performs well if $npq$ is sufficiently large (typically $> 5$ or $10$). In reality, the actual coverage probability—the frequency with which the interval contains the true $p$—fluctuates wildly. Even for a nominal 95% confidence level, the actual coverage often drops significantly lower, especially when $p$ is near the boundaries of 0 or 1.

The coverage probability of the nominal 95% Wald interval as a function of the true proportion p. The plot reveals the characteristic 'zig-zag' oscillations and the severe drop in coverage near the boundaries.

Theoretical Framework: Asymptotic Expansions

To analyze these phenomena, the authors move beyond simple normal approximations by using two-term Edgeworth expansions. Unlike standard one-term approximations that smooth over the discrete nature of the binomial distribution, these higher-order expansions capture both the smooth bias (order $n^{-1}$) and the oscillatory behavior (order $n^{-1/2}$). This rigorous approach allows for a precise comparison of the Wald interval against four alternatives: the Wilson (score), Likelihood Ratio Test (LRT), Jeffreys prior (Bayesian), and Agresti-Coull intervals.

Debunking the $npq$ Myth

The study reveals that the common $npq$ rules of thumb are insufficient to guarantee nominal coverage. For instance, even at $p=0.5$, the Wald interval can fail to reach 94% coverage for surprisingly large $n$. The following table illustrates the minimum sample sizes required for various intervals to maintain reliable coverage across different values of $p$.

Minimum sample size n required for different interval methods to ensure coverage remains above a threshold. Note that the Wald interval (n_S) consistently requires much larger samples than alternatives like Wilson (n_W) or Agresti-Coull (n_AC).

Comparison of Alternative Intervals

  • Wilson and Jeffreys Intervals: These methods provide coverage probabilities that fluctuate near the nominal level without the deep downward spikes characteristic of the Wald interval. The Jeffreys interval is particularly efficient, often providing the shortest expected length among reliable alternatives.
  • Agresti-Coull Interval: This interval, often simplified as the "add 2 successes and 2 failures" rule, is the most conservative. It rarely falls below the nominal coverage level but is consistently the longest of the intervals studied.
  • LRT Interval: Similar to the Wilson interval, it performs well in terms of coverage and length, though it is computationally more intensive.

Conclusion

The Wald interval's ubiquity in textbooks is misplaced. Its poor performance is not merely a small-sample issue but a structural flaw related to its centering and the discrete nature of the binomial distribution. For practical applications, the Wilson, Jeffreys, or Agresti-Coull intervals should be preferred. The Agresti-Coull interval is recommended for its simplicity and conservatism, while the Wilson and Jeffreys intervals offer a superior balance of coverage accuracy and interval precision.

Summary updated

Abstract

We address the classic problem of interval estimation of a binomial proportion. The Wald interval $\hat{p}\pm z_{\alpha/2} n^{-1/2} (\hat{p} (1 - \hat{p}))^{1/2}$ is currently in near universal use. We first show that the coverage properties of the Wald interval are persistently poor and defy virtually all conventional wisdom. We then proceed to a theoretical comparison of the standard interval and four additional alternative intervals by asymptotic expansions of their coverage probabilities and expected lengths. The four additional interval methods we study in detail are the score-test interval (Wilson), the likelihood-ratio-test interval, a Jeffreys prior Bayesian interval and an interval suggested by Agresti and Coull. The asymptotic expansions for coverage show that the first three of these alternative methods have coverages that fluctuate about the nominal value, while the Agresti–Coull interval has a somewhat larger and more nearly conservative coverage function. For the five interval methods we also investigate asymptotically their average coverage relative to distributions for $p$ supported within (0 1) . In terms of expected length, asymptotic expansions show that the Agresti–Coull interval is always the longest of these. The remaining three are rather comparable and are shorter than the Wald interval except for $p$ near 0 or 1. These analytical calculations support and complement the findings and the recommendations in Brown, Cai and DasGupta (Statist. Sci. (2001) 16 101–133).
BibTeX
@article{brown2002confidence,
  author = {Lawrence D. Brown and Tommaso Cai and Anirban Dasgupta},
  title = {{Confidence Intervals for a binomial proportion and asymptotic expansions}},
  journal = {The Annals of Statistics},
  year = {2002},
  doi = {10.1214/aos/1015362189},
}
Figure