On the power of Chatterjee rank correlation

Hongjian Shi, Mathias Drton, Fang HanarXiv 2020 · DOI · arXiv

Abstract

Chatterjee (2021) introduced a simple new rank correlation coefficient, $\xi_n$, that has attracted significant attention for its ability to estimate a population quantity that is zero if and only if the underlying variables are independent. This paper provides a critical evaluation of $\xi_n$ against established rank correlations—Hoeffding's $D$, Blum-Kiefer-Rosenblatt's $R$, and Bergsma-Dassios-Yanagimoto's $\tau^*$. While $\xi_n$ is computationally efficient and versatile, our analysis reveals it is rate sub-optimal for testing independence compared to these classic measures. These findings suggest that for the purpose of general independence testing, $D, R,$ and $\tau^*$ remain superior choices.

Introduction

In the field of statistics, detecting dependence between two random variables, $X$ and $Y$, is a fundamental task. While Pearson’s correlation coefficient is a staple for linear relationships and Spearman’s $\rho$ captures monotonic trends, they fail to detect more complex, non-monotonic dependencies. Recently, Sourav Chatterjee (2021) introduced a rank correlation coefficient ($\xi_n$) that promised a breakthrough: a simple, $O(n \log n)$ measure capable of detecting any type of dependence, which is asymptotically normal under the null hypothesis of independence.

However, the ability to detect any dependence does not necessarily imply the ability to detect every dependence efficiently. This research provides a "reality check" on $\xi_n$, comparing its statistical power and computational demands against established "heavyweight" measures like Hoeffding’s $D$, $R$, and $\tau^*$.

Methodology

The study evaluates these correlation measures across three primary dimensions: statistical consistency, computational efficiency, and statistical power (efficiency). To rigorously assess power, the authors utilize Hájek representation theory and Le Cam’s third lemma to mathematically derive the "local power" of the tests.

The theoretical findings are supplemented by extensive simulations using synthetic datasets designed to test specific dependency structures:

  • Rotation (Konijn) Alternatives: Linear relationships subjected to rotation to test sensitivity to orientation.
  • Mixture (Farlie-type) Alternatives: Distributions that blend independent and dependent components.
  • Generalized Rotation Alternatives: Complex non-linear shapes, including step functions, W-shapes, and sinusoids.

Key Insights: The Power Gap

The most significant finding of this research is that Chatterjee’s $\xi_n$ is "rate sub-optimal." In statistical testing, a test is evaluated by its ability to detect "local alternatives"—dependencies that become weaker as the sample size $n$ increases.

Mathematically, for dependencies shrinking at a rate of $n^{-1/2}$, the power of classic measures like Hoeffding’s $D$ approaches 100% as the sample size grows. In contrast, the power of Chatterjee’s test remains essentially stuck at the baseline significance level (e.g., 5%). This means that in the presence of subtle or weak signals, $\xi_n$ requires significantly larger sample sizes than $D, R,$ or $\tau^*$ to achieve the same level of detection.

Computational Myths and Realities

A primary argument in favor of Chatterjee’s $\xi_n$ has been its computational simplicity and $O(n \log n)$ time complexity. Historically, measures like Hoeffding’s $D$ were viewed as computationally expensive. However, this paper highlights that recent algorithmic advances have leveled the playing field.

Benchmarks conducted with 10,000 data points demonstrate speed parity:

  • Chatterjee’s $\xi_n$: ~3.09 seconds
  • Hoeffding’s $D$: ~3.32 seconds

With the "speed advantage" largely neutralized, the choice between these measures shifts primarily to their statistical power.

Functional vs. General Dependence

The research clarifies the specific niche where $\xi_n$ excels: detecting functional dependence. If $Y$ is a perfect, noiseless function of $X$ (e.g., $Y = \sin(X)$), $\xi_n$ performs admirably. However, for the broader and more common goal of testing whether $X$ and $Y$ are independent in a general sense—where the relationship may be noisy or non-functional—$\xi_n$ acts as a relatively blunt instrument.

In a rotation test simulation with 500 samples, the performance gap was stark: Hoeffding’s $D$ achieved a detection power of 0.954, while Chatterjee’s $\xi_n$ managed only 0.103.

Conclusion

While Chatterjee’s rank correlation is a mathematically elegant and simple tool, it is not a "silver bullet" for independence testing. Its rate sub-optimality makes it less effective than classic measures for detecting weak but important dependencies. Given that modern algorithms have made Hoeffding’s $D$, $R$, and $\tau^*$ computationally competitive, these established measures remain the preferred choice for practitioners seeking robust and powerful tests of independence.

Future work may further explore the behavior of these tests in high-dimensional settings or in the presence of data "ties," as well as potential improvements to the related (but currently slower) estimator proposed by Dette et al. (2013).

Summary updated

Abstract

Chatterjee (2021) introduced a simple new rank correlation coefficient that has attracted much recent attention. The coefficient has the unusual appeal that it not only estimates a population quantity first proposed by Dette et al. (2013) that is zero if and only if the underlying pair of random variables is independent, but also is asymptotically normal under independence. This paper compares Chatterjee's new correlation coefficient to three established rank correlations that also facilitate consistent tests of independence, namely, Hoeffding's $D$, Blum-Kiefer-Rosenblatt's $R$, and Bergsma-Dassios-Yanagimoto's $τ^*$. We contrast their computational efficiency in light of recent advances, and investigate their power against local rotation and mixture alternatives. Our main results show that Chatterjee's coefficient is unfortunately rate sub-optimal compared to $D$, $R$, and $τ^*$. The situation is more subtle for a related earlier estimator of Dette et al. (2013). These results favor $D$, $R$, and $τ^*$ over Chatterjee's new correlation coefficient for the purpose of testing independence.
BibTeX
@misc{shi2020power,
  author = {Hongjian Shi and Mathias Drton and Fang Han},
  title = {{On the power of Chatterjee rank correlation}},
  year = {2020},
  eprint = {2008.11619},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.11619},
}
Figure