TL;DR
Jerzy Neyman’s seminal 1937 paper provides the first rigorous mathematical foundation for confidence intervals. He moves beyond single-point estimates to define a method for calculating ranges that—in the long run—are guaranteed to contain the true, unknown population parameter with a pre-defined frequency.
What Problem Does This Paper Address?
In the early 20th century, statisticians were divided on how to estimate unknown constants (like the average height of a population or a physical constant). There were two main approaches, both with flaws:
- Unique Estimates: Methods like Maximum Likelihood or Least Squares provided a single number (a point estimate). However, these didn't inherently communicate their own accuracy or the "margin of error."
- Bayesian Inference: This used Bayes’ Theorem to provide a range, but it required an a priori probability distribution—a "prior guess" about the parameter. Neyman argued that for most scientific constants, a prior distribution simply doesn't exist or is purely arbitrary, making the resulting "a posteriori" probabilities mathematically unjustifiable in a frequentist framework.
Neyman sought a way to provide an interval estimate that was based purely on the "classical" theory of probability (frequentism), where the parameter is a fixed constant and the randomness comes entirely from the sampling process.
Methods
Neyman introduces a geometric and probabilistic framework to solve the estimation problem:
- Sample Space vs. General Space: He visualizes every possible experimental outcome as a point in a multi-dimensional "sample space." He then constructs a "general space" that adds the unknown parameter as an additional dimension.
- Regions of Acceptance: The core innovation is the construction of regions in the sample space for every possible value of the parameter. If the true parameter is $\theta$, we define a region such that the probability of our data falling into it is exactly $\alpha$ (e.g., 0.95).
- Confidence Limits: By "inverting" these regions, Neyman defines two functions of the data, $\underline{\theta}(E)$ and $\bar{\theta}(E)$. These are the confidence limits.
- Sufficient Statistics: To handle complex cases where multiple unknown parameters exist (nuisance parameters), he utilizes "sufficient statistics" to create "similar regions"—subsets of the data that allow for estimation of one parameter regardless of the values of the others.
- Optimization (Shortest Intervals): Neyman doesn't just want any interval; he wants the "best" one. He defines "shortest" systems of confidence intervals as those that minimize the probability of accidentally covering false values of the parameter.
Datasets
As a foundational theoretical paper, it does not use modern experimental datasets. However, Neyman includes:
- Synthetic Examples: He provides detailed mathematical examples using the Uniform Distribution and the Normal Distribution.
- Tippett’s Random Number Tables (1927): He suggests a "sampling experiment" using these tables to demonstrate that his theory holds up in practice, showing that the calculated intervals will indeed cover the true value with the predicted frequency.
Key Insights and Ideas
- The Interval is Random, the Parameter is Fixed: This is the "aha moment" of frequentist statistics. We don't say there is a 95% probability the parameter is in the interval. Instead, we say that if we repeat the experiment many times, 95% of the intervals we calculate will contain the fixed, true parameter.
- Rejection of the "Prior": Neyman explicitly separates his theory from the work of Jeffreys and others who relied on prior beliefs. He frames estimation as a "rule of behavior"—if you follow this mathematical procedure, you will be right a known percentage of the time in the long run.
- The Impossibility of the "Shortest" Interval: In a surprising result (Proposition X), Neyman proves that for many common distributions (like the Normal distribution), a "uniformly shortest" interval—one that is the best for every possible value of the parameter—does not actually exist.
- Unbiased Intervals: Because "shortest" intervals are rare, he introduces the concept of "unbiased" confidence intervals, which ensure that the probability of covering the true value is always higher than the probability of covering any false value.
Key Results
- Formalization of Confidence: He defines the Confidence Coefficient ($\alpha$) and the Confidence Interval.
- Practical Formulas: He derives the now-standard formulas for one-sided and two-sided estimation of the mean ($\xi$) of a normal distribution when the variance is unknown, utilizing Student’s $t$-distribution.
- Geometric Proofs: He provides a rigorous proof that if a system of "regions of acceptance" satisfies certain conditions, the resulting confidence intervals are mathematically valid under the classical theory of probability.
Limitations and Future Work
- Model Dependency: The entire theory relies on the assumption that the "probability law" (the mathematical model of the experiment) is correct. If the model is wrong, the confidence coefficient is meaningless.
- Existence of Shortest Intervals: As noted in Proposition X, the "ideal" shortest interval is often mathematically impossible to achieve, forcing statisticians to settle for "unbiased" or "short" intervals instead.
- Computational Complexity: While Neyman provides the theory, the actual construction of these regions for complex, multi-parameter models was (in 1937) a daunting task, a challenge that would later be addressed by computational statistics and MCMC methods.
This paper remains a cornerstone of modern science, providing the rigorous justification for the "$\pm$" values seen in everything from engineering tolerances to clinical trial results.
Summary updated