On the Probable Error of the Correlation Coefficient to a Second Approximation

H. E. SoperBiometrika 1913 · DOI

This classic 1913 paper by H. E. Soper, published in Biometrika, represents a foundational step in small-sample statistics. Soper provides a rigorous second-order approximation for the distribution of the correlation coefficient ($r$), moving beyond the standard normal assumptions of his era to account for the inherent skewness found in small datasets.

TL;DR

H. E. Soper derives second-order mathematical approximations for the mean, mode, and standard deviation of the correlation coefficient ($r$) in small samples. His work reveals that for small $n$ or high population correlation ($\rho$), the distribution of $r$ is significantly skewed, meaning the "most likely" value observed in a single sample is often higher than the true population value.

What Problem Does This Paper Address?

In the early 20th century, statisticians generally assumed that the correlation coefficient ($r$) followed a normal (Gaussian) distribution. This assumption works well for large samples, but it fails spectacularly when the sample size ($n$) is small or when the true correlation ($\rho$) is near $+1$ or $-1$.

In these cases, the distribution of $r$ becomes "markedly skew." If you rely on standard normal errors, you will likely misinterpret the significance of your results. Soper’s goal was to find a more accurate way to describe this frequency distribution and its "probable error," providing researchers with formulas that remain reliable even when data is scarce.

Methods

Soper approaches the problem by modeling the distribution of $r$ using Pearson Type I skew curves. To find the constants for these curves, he employs a "grade group" method to evaluate the moments of deviations.

The core of the paper involves a complex mathematical expansion of the product-moment formula for $r$. By expanding this formula in powers of deviations and applying Stirling’s Theorem, Soper derives second-order approximations for the distribution's key moments. Unlike previous first-order methods that provided a single "standard error," Soper’s second-order approach captures the asymmetry of the distribution, allowing for separate calculations of the mean ($\bar{r}$), the mode ($\check{r}$), and the standard deviation ($\sigma_r$).

A Pearson Type I curve illustrating the skewness of the correlation coefficient distribution, where the mode ($\check{r}$) and mean ($\bar{r}$) diverge from the true population value.

Datasets

Soper validates his theoretical formulas using experimental sampling data previously published by "Student" (William Sealy Gosset) in 1908. This empirical data was derived from W. R. Macdonell’s measurements of finger length and stature in 3,000 criminals. From this "population," Student had drawn thousands of random samples of sizes $n=4, 8,$ and $30$ to observe the actual behavior of $r$. Soper uses these results as a benchmark to see how well his new approximations fit reality.

Key Insights and Ideas

  • The Bias of Small Samples: Soper demonstrates that the mean value of $r$ calculated from many small samples will slightly underestimate the true population correlation $\rho$.
  • Mode vs. Truth: Conversely, the modal value (the most frequent or "most likely" value to appear in a single sample) is numerically greater than the true $\rho$. This means a single small sample is more likely to overstate the strength of a correlation than to hit the true value.
  • The "J-Curve" Trap: For extremely small samples (like $n=3$ or $4$) and high true correlations, the distribution can become "J-shaped." In these cases, there is no central peak; instead, the most frequent observation is a perfect correlation of $1.0$, simply due to the mathematical constraints of small-sample geometry.
  • Refining the Standard Deviation: Soper shows that the standard deviation is not just a function of $n$, but is heavily influenced by the magnitude of $\rho$ itself, necessitating a more complex formula than the one used for normal distributions.

Key Results

Soper’s primary contribution is a set of refined formulas for the moments of $r$:

  1. Mean ($\bar{r}$): $\bar{r} \approx \rho \left(1 - \frac{1-\rho^2}{2n}\right)$
  2. Standard Deviation ($\sigma_r$): $\sigma_r \approx \frac{1-\rho^2}{\sqrt{n-1}} \left(1 + \frac{11\rho^2}{4n}\right)$

When compared against "Student’s" empirical data, Soper’s second-order approximations showed a much tighter fit than the standard Gaussian model. For $n=30$, the fit was nearly perfect, and even for $n=8$, the formulas provided a significantly better description of the observed variance and skewness than previous methods.

Limitations and Future Work

Soper acknowledges that his formulas are still approximations. They begin to break down at the extreme edges of the scale—for instance, they can occasionally predict a modal value slightly greater than $1.0$ for very small $n$.

Furthermore, the entire derivation assumes that the underlying population characters follow a normal distribution. Soper concludes by suggesting that further experimental work is needed to determine how these distributions behave when the parent population is non-normal, a question that would continue to occupy statisticians for decades.

Summary updated

BibTeX
@inproceedings{soper1913probable,
  author = {H. E. Soper},
  title = {{On the Probable Error of the Correlation Coefficient to a Second Approximation}},
  booktitle = {Biometrika},
  year = {1913},
  doi = {10.2307/2331802},
}
Frequency Distribution of the Values of the Correlation Coefficient in Samples from an Indefinitely Large PopulationBiometrika · 1915
Mathematical contributions to the theory of evolution. IV. On the probable errors of frequency constants and on the influence of random selection on variation and correlationProceedings of the Royal Society of London · 1898
The Proof and Measurement of Association between Two ThingsThe American Journal of Psychology · 1904
On Some Useful “Inefficient” StatisticsSpringer Series in Statistics
On the mathematical foundations of theoretical statisticsPhilosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · 1922
Some Tests of Significance, Treated by the Theory of ProbabilityMathematical Proceedings of the Cambridge Philosophical Society · 1935
Figure