Resolution of Simpson's paradox via the common cause principle

A. Hovhannisyan, Armen Allahverdyan

Introduction

Simpson’s paradox—where a statistical trend appears in several distinct groups but disappears or reverses when those groups are combined—is more than a mere mathematical quirk; it is a fundamental challenge to probabilistic inference and decision-making. Imagine a medical study where Treatment A appears more effective than Treatment B for both men and women individually. Yet, when the data for both genders is aggregated, Treatment B suddenly appears superior. This leaves researchers in a bind: should they trust the aggregate data with its larger sample size, or the partitioned, specific data?

Examples of various causal diagrams (TODAGs) that typically require perfect mapping to resolve paradoxes.

Previous frameworks, such as Judea Pearl’s causal diagrams, offer solutions but often require researchers to know the exact "Time-Ordered Directed Acyclic Graph" (TODAG). Essentially, this demands a perfect, real-world map of what causes what—a luxury rarely available in practice. This paper addresses two critical questions: Is Simpson's paradox just an artifact of deficient data collection, and can we resolve it without a perfect causal map?

Methods: Frequency and the Common Cause Principle

The researchers tackle the paradox through two primary lenses: frequency estimation and theoretical resolution.

Frequency Estimation: To determine if the paradox is merely a rare fluke of bad data, the authors utilized a non-informative Dirichlet density process. This method generates unbiased random probabilities to observe how often the paradox naturally emerges in three-variable systems.

The Common Cause Principle: The core proposal of the paper is that most instances of Simpson's paradox can be resolved by assuming the existence of an unobserved "common cause" (or screening variable), denoted as $C$.

A causal diagram illustrating a common cause C influencing variables A1, A2, and B.

Mathematical Proofs: The authors provide two key theorems to support this principle:

  • Theorem 1: Focuses on binary variables (e.g., Yes/No scenarios). It proves mathematically that if a binary common cause exists, the fine-grained (partitioned) conclusion is always the correct basis for decision-making.
  • Theorem 2: Extends this logic to continuous variables, specifically those modeled by Gaussian (normal) distributions, reaching the same conclusion.

Key Insights and Real-World Applications

The study yields several transformative insights into how we handle contradictory data:

  • A Structural Feature, Not Bad Data: The Dirichlet sampling ($N = 10^7$ samples) revealed that Simpson’s paradox occurs in roughly 4.29% of random 3-variable binary systems. This means in every 1,000 datasets, about 42 will likely contain this contradiction. It is a genuine structural feature of probability, not just a data collection error.
  • The "Minimal" Rule: The most powerful takeaway is that if the hidden cause is "minimal" (e.g., binary), you do not actually need to observe it to know how to act. Its mere plausible existence is enough mathematical justification to trust the fine-grained, partitioned data over the aggregate.
  • Causality vs. Probability: The paper argues that Simpson’s paradox is primarily a decision-making problem. If the statistical dependence between variables isn't caused by one directly influencing the other, a common cause is the most natural and mathematically sound explanation.

The authors validate their approach by revisiting famous real-world cases. For instance, in COVID-19 case fatality rates comparing Italy and China, aggregate data suggested Italy performed worse. However, by applying the Common Cause logic (using "healthcare system quality" as a plausible binary common cause), the authors demonstrate that the fine-grained, age-stratified view—which suggested China performed worse in each specific age group—is the statistically correct one to trust.

Limitations and Future Work

While the Common Cause Principle offers a robust resolution, the authors are transparent about its limitations:

  • The "Minimal" Assumption: The definitive resolution holds strictly if the common cause is minimal (binary or scalar). If the hidden cause is complex (tertiary or higher), detailed knowledge of that variable is still required to resolve the paradox.
  • Inference vs. Observation: In many practical examples, the common cause is inferred rather than directly measured. Proving the existence of these specific causes in real-world, noisy data remains a challenge.
  • Approximate Causes: The current proofs assume a "precise" common cause. Future research will need to explore "approximate" causes—situations typical in big data and machine learning where a hidden variable explains most, but not all, of the dependency.

Reference: Hovhannisyan, A., & Allahverdyan, A. (2025). Resolution of Simpson’s paradox via the common cause principle. 39th Conference on Neural Information Processing Systems (NeurIPS 2025).

Summary updated

Abstract

Simpson's paradox poses a challenge in probabilistic inference and decision-making. Our study revisits the paradox by re-estimating its frequency with an unbiased data generation process and reaffirms that it is not an artifact of deficient data collection. Thus, it can lead to incorrect recommendations in fields as diverse as statistics, psychology, and artificial intelligence. We show that the paradox can be resolved by assuming a minimal — though not necessarily observed — common cause (or screening) variable for the involved random variables. In our approach, conditioning on this minimal common cause establishes the correct association between events, which coincides with the conditioning (i.e., fine-grained) option of the original Simpson paradox. This resolution applies to both discrete cases of binary variables and continuous settings modeled by Gaussian variables. For a non-minimal common cause, the resolution of the paradox is possible, but detailed knowledge of the common cause is required. Our findings extend traditional understandings of the paradox and offer practical guidance for resolving apparent contradictions in probabilistic inference, ultimately enhancing decision-making processes. This point is illustrated by several examples.
BibTeX
@inproceedings{hovhannisyan2025resolution,
  author = {A. Hovhannisyan and Armen Allahverdyan},
  title = {{Resolution of Simpson's paradox via the common cause principle}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=1DmvJtwaDg},
}
Figure