This 1915 paper by R.A. Fisher is a landmark in the history of statistics. In it, Fisher derives the exact probability distribution of the sample correlation coefficient ($r$)—a problem that had stumped the leading statisticians of the day. By abandoning tedious algebra in favor of elegant $n$-dimensional geometry, Fisher provided the tools necessary for rigorous hypothesis testing in bivariate data.
TL;DR
Fisher solves a major open problem in early 20th-century statistics by deriving the exact frequency distribution for the sample correlation coefficient ($r$). Using a novel geometric approach, he proves that previous approximations were significantly inaccurate for small samples, laying the foundation for modern exact statistical inference.
What Problem Does This Paper Address?
In the early 1900s, statisticians knew how to calculate the correlation between two variables, but they didn't fully understand how that value fluctuated due to random sampling. If you take a small sample (say, 10 pairs of observations) from a population where the true correlation is $\rho$, what is the probability of getting a specific sample correlation $r$?
Previous attempts, notably by H.E. Soper in 1913, relied on "method of moments" approximations. These worked okay for massive datasets but failed miserably for small samples or when the true correlation was very high. Fisher sought an exact mathematical solution that would hold true regardless of sample size.
Methods
Fisher’s breakthrough was moving from algebra to geometry. He realized that a sample of $n$ observations can be viewed as a single point in an $n$-dimensional space.
- Geometric Interpretation: He represents the sample of $x$-values and $y$-values as two vectors in $n$-dimensional space. The sample correlation $r$ is elegantly revealed to be the cosine of the angle between these two vectors (after they are shifted to the mean).
- Volume Elements: To find the frequency distribution, Fisher calculates the "volume" of the region in this high-dimensional space that corresponds to a specific range of $r$.
- Integration: He uses the geometry of $n$-dimensional spheres to set up an integral for the probability density. As shown in Figure 1, he visualizes the sample point $P$ and its relationship to the mean $M$ and the axes.
Datasets
Fisher does not use "training data" in the modern machine learning sense. Instead, he uses a dataset of 745 observations originally provided by "Student" (William Sealy Gosset) to validate his theoretical results. He compares how well his exact formula fits this empirical data versus Soper’s previous approximations.
Key Insights and Ideas
- The Power of Geometry: Fisher demonstrated that complex statistical distributions often have simple geometric interpretations in high-dimensional space. This "geometric insight" became a hallmark of his later work.
- Small Sample Sensitivity: He showed that the distribution of $r$ is highly non-normal and skewed, especially when the sample size is small or the population correlation is high. This meant that the standard errors used by his contemporaries were often misleading.
- The Seed of the Z-Transformation: Toward the end of the paper, Fisher hints at a transformation ($tanh^{-1} \rho$) that could make the distribution more normal and stabilize its variance. This would later become the famous "Fisher Z-transformation," a staple of modern statistics.
Key Results
Fisher’s exact formula provided a significantly better fit to experimental data than previous methods.
In a direct head-to-head comparison (Figure 2), Fisher calculated the expected frequencies for "Student's" data. Using a goodness-of-fit measure ($e^2/m$), Fisher’s distribution achieved a total value of 23.61, while Soper’s approximation lagged behind at 84.17. A lower value indicates a better fit, proving Fisher's theoretical derivation was far more accurate.
Limitations and Future Work
- Mathematical Complexity: The exact distribution involves complex integrals and Legendre functions, making it difficult to calculate by hand in 1915. Fisher acknowledges that for very large samples, simpler approximations are still useful.
- Normality Assumption: The derivation assumes the underlying population follows a normal bivariate distribution. Fisher notes that the behavior of $r$ in non-normal populations remains an open question.
- Search for Simplicity: Fisher concludes by suggesting that finding a "more natural variable" (the aforementioned Z-transformation) would simplify the treatment of these distributions in the future.
Summary updated