The Multiple Dimensions of Spuriousness in Machine Learning

Samuel J. Bell, Skyler WangarXiv 2024 · arXiv · OpenReview

Abstract

Learning correlations from data forms the foundation of today's machine learning (ML) and artificial intelligence (AI) research. While such an approach enables the automatic discovery of patterned relationships within big data corpora, it is susceptible to failure modes when unintended correlations are captured. This vulnerability has expanded interest in interrogating spuriousness, often critiqued as an impediment to model performance, fairness, and robustness. In this article, we trace deviations from the conventional definition of statistical spuriousness—which denotes a non-causal observation arising from either coincidence or confounding variables—to articulate how ML researchers make sense of spuriousness in practice. Drawing on a broad survey of ML literature, we conceptualize the "multiple dimensions of spuriousness," encompassing: relevance, generalizability, human-likeness, and harmfulness. These dimensions demonstrate that ML spuriousness goes beyond the causal/non-causal dichotomy and that the disparate interpretative paths researchers choose could meaningfully influence the trajectory of ML development. By underscoring how a fundamental problem in ML is contingently negotiated in research contexts, we contribute to ongoing debates about responsible practices in AI development.

1. Introduction

Modern machine learning (ML) is built upon the identification of predictive correlations. However, models frequently latch onto "shortcuts"—features that are statistically predictive in a training set but do not represent the intended underlying concept. A classic example is a model identifying a cow based on a grassy background rather than the animal's features. While the technical community often labels these failures as "spurious correlations," there is a significant conceptual gap between the formal statistical definition and its operational use in ML research.

Conceptual framework of spuriousness: (a) Causal diagram illustrating relationships between a common cause (the moon) and various effects. (b-f) Empirical data demonstrating causal (b, c), illusory (d), and spurious (e, f) correlations.

In statistics, spuriousness typically refers to non-causal relationships arising from confounding variables (as illustrated in the figure above). However, as this paper demonstrates, ML researchers use the term much more loosely, reflecting subjective judgments about what constitutes a "good" or "bad" correlation based on human expectations and values.

2. Methodology and Datasets

We conducted a schematic analysis of ML literature to uncover the "operational pragmatics" of the field. Using the Semantic Scholar API, we retrieved 713 publications related to keywords such as "spurious correlations," "spurious features," and "shortcuts." Through an iterative process of coding and discussion, we narrowed this sample to a core set of 65 highly relevant and influential papers.

The analysis highlights several iconic datasets used to illustrate these failures, including ImageNet (where models rely on texture or human handlers), Waterbirds (testing susceptibility to background correlations), CelebA (gender and hair color correlations), and Chest X-ray datasets (where models learn to detect metal tokens placed by technicians instead of pathologies).

3. The Four Pragmatic Frames of Spuriousness

Our analysis reveals that "spuriousness" is not an inherent property of data but a situated judgment. We identify four primary frames used by researchers to interpret model failures:

  • Relevance: Researchers expect models to use features "inherently relevant" to the task. If a model uses a background feature (e.g., a desert for a camel), it is labeled spurious because it misaligns with human intent, even if it is statistically predictive.
  • Generalizability: A correlation is deemed spurious if it fails to hold in "out-of-distribution" (OOD) settings. This frame transforms spuriousness into a requirement for stability across environments.
  • Human-likeness: This frame judges models against human cognitive standards. For example, if a model relies on "texture" while a human uses "shape," the texture-based correlation is labeled a "shortcut."
  • Harmfulness: This normative frame labels correlations as spurious if they amplify societal stereotypes or lead to unfair outcomes (e.g., using gender as a proxy for occupation), regardless of their presence in the training data.

4. Discussion and Conclusion

The shift from a purely statistical view to these pragmatic frames suggests that researchers use "spuriousness" as a rhetorical tool to align model behavior with human expectations. For instance, in ImageNet's "tench" classification, models often rely on the presence of human fingers. While this is a valid correlation within the dataset, it is labeled spurious because it violates the Relevance frame.

By skirting causality and employing these frames, researchers implicitly encode their own values into the definition of "correct" machine learning. We call for the ML community to be more explicit about the justifications behind labeling correlations as spurious, which could lead to more robust benchmarks and socially-responsible AI development.

Summary updated

Abstract

Learning correlations from data forms the foundation of today's machine learning (ML) and artificial intelligence (AI) research. While such an approach enables the automatic discovery of patterned relationships within big data corpora, it is susceptible to failure modes when unintended correlations are captured. This vulnerability has expanded interest in interrogating spuriousness, often critiqued as an impediment to model performance, fairness, and robustness. In this article, we trace deviations from the conventional definition of statistical spuriousness-which denotes a non-causal observation arising from either coincidence or confounding variables-to articulate how ML researchers make sense of spuriousness in practice. Drawing on a broad survey of ML literature, we conceptualize the "multiple dimensions of spuriousness," encompassing: relevance ("Models should only use correlations that are relevant to the task."), generalizability ("Models should only use correlations that generalize to unseen data"), human-likeness ("Models should only use correlations that a human would use to perform the same task"), and harmfulness ("Models should only use correlations that are not harmful"). These dimensions demonstrate that ML spuriousness goes beyond the causal/non-causal dichotomy and that the disparate interpretative paths researchers choose could meaningfully influence the trajectory of ML development. By underscoring how a fundamental problem in ML is contingently negotiated in research contexts, we contribute to ongoing debates about responsible practices in AI development.
BibTeX
@misc{bell2024multiple,
  author = {Samuel J. Bell and Skyler Wang},
  title = {{The Multiple Dimensions of Spuriousness in Machine Learning}},
  year = {2024},
  eprint = {2411.04696},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.04696},
}
Figure