Causal and Associational Language in Observational Health Research: A Systematic Evaluation

Noah Haber, Sarah Wieten, Julia M. Rohrer, Onyebuchi A. Arah, Peter W. G. Tennant, Elizabeth A. Stuart, Eleanor J. Murray, Sophie Pilleron, Sze Tung Lam, Emily Riederer, Sarah J. Howcutt, Alison E. Simmons, Clémence Leyrat, Philipp Schoenegger, Anna Booman, Mi‐Suk Kang Dufour, Ashley L. O’Donoghue, Rebekah Baglini, Stefanie Do, Mari Takashima, Thomas Rhys Evans, Daloha Rodríguez-Molina, Taym Alsalti, Daniel J. Dunleavy, Gideon Meyerowitz‐Katz, Alberto Antonietti, Jose Andrés Calvache, Mark Kelson, Meg G. Salvia, Camila Olarte Parra, Saman Khalatbari‐Soltani, Taylor McLinden, Arthur Chatton, Jessie Seiler, Andreea Steriu, Talal S. Alshihayb, Sarah E. Twardowski, Julia Dabravolskaj, Eric Au, Rachel A. Hoopsick, Shashank Suresh, Nicholas Judd, Sebastián Peña, Cathrine Axfors, Palwasha Khan, Ariadne Rivera‐Aguirre, Nnaemeka U. Odo, Ian Schmid, Matthew P. Fox

TL;DR

This systematic evaluation of 1,170 high-profile medical articles reveals a profound disconnect between the "associational" language researchers use to describe results and the "causal" weight of their practical recommendations. The study finds that avoiding explicit causal language does not lead to clarity; instead, it creates "Schrödinger’s causal inference," where findings are simultaneously treated as mere correlations and as the basis for clinical action.

What Problem Does This Paper Address?

In observational health research, there is a long-standing "taboo" against using causal language (e.g., "X causes Y") because these studies are prone to confounding and bias. To be "cautious," researchers often employ associational euphemisms (e.g., "X is associated with Y").

However, this creates a gap: if a study isn't claiming causality, why should a doctor change their practice based on it? The authors tackle the "linguistic gymnastics" used in high-profile journals to see if avoiding causal words actually improves clarity or if it simply allows researchers to imply causality without being held accountable to the rigorous methodological standards required for causal inference.

Methods

The authors conducted a massive systematic evaluation using a "crowdsourced" approach with 48 reviewers (primarily epidemiology and statistics experts).

  • Sampling: They screened 1,170 articles from 18 high-profile journals (e.g., The Lancet, JAMA, NEJM) published between 2010 and 2019.
  • Review Process: Each article was reviewed by three unique randomly selected reviewers (two independent and one arbitrator).
  • Rating Scale: Reviewers rated the "causal implication strength" of linking sentences and action recommendations on a four-point scale: None, Weak, Moderate, or Strong.
  • Linguistic Analysis: They performed a "root word" exercise, where reviewers rated the inherent causal strength of specific words (like "impact," "link," or "associate") both in and out of context.

Datasets

The study utilized a newly curated dataset of 1,170 articles (1,065 observational studies and 105 Randomized Controlled Trials for comparison) across 18 top-tier journals. The journals were selected based on impact factor and SJR scores across medicine, public health, and epidemiology.

Key Insights and Ideas

  • The "Association" Trap: "Associate" was the most common linking word (45.7%). While often used to avoid causal claims, over half of the reviewers rated the word "association" as having at least some causal implication.
  • Action vs. Language Disconnect: In 44.5% of articles, the action recommendations (what people should do) implied a stronger causal link than the sentences describing the results. Essentially, authors were "cautious" in their results but "bold" in their advice.
  • Schrödinger’s Causal Inference: The authors coin this term to describe studies that avoid stating a causal interest yet are embedded with causal intent, inference, and recommendations.
  • Linguistic Heterogeneity: There is no consensus on what words mean. Reviewers’ ratings of the same root words were highly varied, suggesting that the "technical" vocabulary of epidemiology is far more subjective than researchers realize.
  • RCT Similarity: Surprisingly, Randomized Controlled Trials (the gold standard for causality) used similar linking words to observational studies, suggesting a universal "scientific dialect" that obscures the underlying strength of evidence.

Key Results

  • Implied Causality: Even in observational abstracts, only 13.8% of linking language was rated as having "No" causal implication. The rest were rated as Weak (34.2%), Moderate (33.2%), or Strong (18.7%).
  • Policy Impact: Journals with explicit policies against causal language (like JAMA) did have lower causal language strength, but they also had a lower prevalence of action recommendations.
  • Causal Intent: While only 3.8% of studies used formal causal models (like Directed Acyclic Graphs), 80% offered some theoretical discussion of causal mechanisms, showing that causality is almost always the "elephant in the room."

Limitations and Future Work

The authors acknowledge that their ratings are subjective; however, they argue this is a feature, not a bug, as it reflects how the "consumers" of research actually interpret language. The study was limited to high-profile journals, so results might differ in niche or lower-tier publications.

Future Work: The authors suggest moving away from "policing" specific words. Instead, they recommend that researchers clearly state their target estimand—explicitly defining whether they are seeking a causal effect or a simple prediction. Clarity, they argue, comes from methodological transparency, not from stripping "causal" words out of the dictionary.

Summary updated

Abstract

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.
BibTeX
@inproceedings{haber2022causal,
  author = {Noah Haber and Sarah Wieten and Julia M. Rohrer and Onyebuchi A. Arah and Peter W. G. Tennant and Elizabeth A. Stuart and Eleanor J. Murray and Sophie Pilleron and Sze Tung Lam and Emily Riederer and Sarah J. Howcutt and Alison E. Simmons and Clémence Leyrat and Philipp Schoenegger and Anna Booman and Mi‐Suk Kang Dufour and Ashley L. O’Donoghue and Rebekah Baglini and Stefanie Do and Mari Takashima and Thomas Rhys Evans and Daloha Rodríguez-Molina and Taym Alsalti and Daniel J. Dunleavy and Gideon Meyerowitz‐Katz and Alberto Antonietti and Jose Andrés Calvache and Mark Kelson and Meg G. Salvia and Camila Olarte Parra and Saman Khalatbari‐Soltani and Taylor McLinden and Arthur Chatton and Jessie Seiler and Andreea Steriu and Talal S. Alshihayb and Sarah E. Twardowski and Julia Dabravolskaj and Eric Au and Rachel A. Hoopsick and Shashank Suresh and Nicholas Judd and Sebastián Peña and Cathrine Axfors and Palwasha Khan and Ariadne Rivera‐Aguirre and Nnaemeka U. Odo and Ian Schmid and Matthew P. Fox},
  title = {{Causal and Associational Language in Observational Health Research: A Systematic Evaluation}},
  booktitle = {American},
  year = {2022},
  doi = {10.1093/aje/kwac137},
}
Figure