On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection

Jerzy NeymanJournal of the Royal Statistical Society 1934 · DOI

This 1934 paper by Jerzy Neyman is widely considered the "birth certificate" of modern scientific sampling. In it, Neyman fundamentally shifts the field of statistics by formalizing the theory of stratified random sampling and introducing the revolutionary concept of confidence intervals. By mathematically dismantling the then-popular method of "purposive selection," he provided the rigorous foundation necessary for government censuses and social surveys to produce reliable, defensible data.

TL;DR

Neyman’s landmark paper revolutionized statistics by formalizing stratified random sampling and introducing the concept of confidence intervals. He proved that random selection within strata is mathematically superior to hand-picking "representative" groups, establishing the framework for all modern survey design.

What Problem Does This Paper Address?

In the early 20th century, statisticians were divided on how to create a "representative" sample of a large population. The dominant approach was purposive selection, where researchers hand-picked a few groups (like specific districts or towns) that, on average, matched known population characteristics (e.g., average birth rate or income).

The challenge was that while these samples looked "representative" on paper for a few chosen variables, they often failed spectacularly for others. Furthermore, there was no way to mathematically measure the precision of these estimates without making massive, often untestable assumptions about the population's structure. Neyman sought a method that was consistent—meaning it would yield the correct answer as the sample size grew—regardless of the unknown properties of the population.

Methods

Neyman formalizes two main approaches and compares them using the Gauss-Markov theorem:

  1. Stratified Random Sampling: The population is divided into groups (strata), and individuals are selected at random from each group. Neyman introduces the "Optimum Allocation" (now known as Neyman Allocation), proving that to minimize error, the sample size in each stratum should be proportional to the product of the stratum's size and its internal standard deviation ($m_i \propto M_i \sigma_i$).
  2. Confidence Intervals: Building on R.A. Fisher’s "fiducial limits," Neyman introduces the "confidence belt." He defines a method to construct intervals that will contain the true population parameter with a fixed probability (the confidence coefficient), independent of any "prior" beliefs about the data.
  3. Critique of Purposive Selection: He mathematically demonstrates that purposive selection only works if the relationship between the "control" variables and the variables of interest is strictly linear—an assumption that rarely holds in the real world.

Datasets

Neyman illustrates his theories using several real-world examples:

  • The Italian General Census of 1921: He analyzes a failed attempt by statisticians Gini and Galvani to select 29 districts out of 214 to represent all of Italy.
  • The Polish General Census of 1931: Neyman discusses his own work with the Polish Institute for Social Problems, which used stratified sampling across 113 strata to study the working class.
  • Bulgarian Agricultural Data: Mentioned in the discussion section, this provided further evidence for the linearity (or lack thereof) in agricultural variables.

Key Insights and Ideas

  • The Fallacy of Hand-Picking: Neyman shows that purposive selection is a "gambler's bet." Even if you match the averages of your sample to the population, you might be introducing massive bias if the underlying regression is non-linear.

    Figure 1: Scatter plot showing the relationship between mean income and birth rate in Italian districts.
  • Non-Linearity is the Enemy: As seen in Figure 2, the "true" regression line for social data is often far from straight. Purposive selection assumes a linear relationship; when that fails, the estimates become biased.

    Figure 2: The weighted regression of income on birth rate, illustrating significant deviations from linearity.
  • Randomness as a Shield: By using random selection, the statistician is protected from the "uncontrollable intuition" of the investigator. Randomness allows for a mathematical guarantee of accuracy (confidence) that hand-picking cannot provide.

  • Confidence vs. Probability: Neyman clarifies that a confidence interval doesn't mean the parameter has a "probability" of being in the range (which would require Bayesian priors); rather, it means the procedure for creating the range is right 95% (or 99%) of the time in the long run.

Key Results

  • Mathematical Superiority: Neyman proved that stratified random sampling is always consistent, whereas purposive selection is only consistent under extremely restrictive conditions that are almost never met in social science.
  • The Optimum Allocation Formula: He provided the definitive proof for how to distribute samples across strata to achieve the highest possible precision.
  • Validation of Random Sampling: The paper successfully convinced the Royal Statistical Society (and eventually the world) that random sampling was not just a "convenience" but a mathematical necessity for valid inference.

Limitations and Future Work

Neyman acknowledges that random sampling of individuals is often physically or financially impossible (e.g., you can't easily visit 1,000 random homes across a whole country). This leads to cluster sampling (sampling groups like villages), which he notes is more practical but requires careful handling to maintain precision.

He also admits that if a population is very small and the correlation between variables is extremely high, purposive selection could be more precise, but he argues this is an exceptional case and not a sound basis for general scientific methodology. The discussion at the end of the paper (featuring giants like Bowley and Fisher) highlights the ongoing debate over the philosophical interpretation of "confidence" versus "fiducial" probability.

Summary updated

BibTeX
@article{neyman1934two,
  author = {Jerzy Neyman},
  title = {{On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection}},
  journal = {Journal of the Royal Statistical Society},
  year = {1934},
  doi = {10.2307/2342192},
}
Figure