Quantifying Spurious Feature Reliance in Machine Learning

26 concepts19 papers1 proposal

Deep learning models are notoriously opportunistic. If a dataset contains a strong but non-causal correlation—such as watermarks on images of a specific class, or hospital-specific text formatting in medical records—a model will often latch onto these "shortcuts" rather than learning the true underlying concept.

Standard validation metrics fail to expose this behavior because validation sets typically share the same distribution and biases as the training data. To build truly robust systems, we must move beyond aggregate accuracy and find ways to precisely quantify a model's reliance on spurious features. The core intuition behind recent work in this space is simple: to measure a shortcut's influence, we must turn it into an independent variable, breaking its correlation with the target label to observe how the model reacts.

Counterfactual Interventions: Isolating the Shortcut

The most direct way to prove a model is relying on a shortcut is to alter only that shortcut while keeping the core semantic content identical. If the model's prediction changes, it was using a crutch.

In the visual domain, Nina Weng et al. demonstrate this in Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation by using diffusion models to digitally add or remove clinical artifacts (like pacemakers) from chest X-rays. By comparing the classifier's output on the original image versus the counterfactual image, they establish a direct numerical metric for shortcut bias. Similarly, Valentyn Boreiko et al. leverage optimization techniques in Sparse Visual Counterfactual Explanations in Image Space to discover the minimal, localized image modifications required to flip a classifier's decision, exposing hidden dependencies on backgrounds or watermarks.

Original Image (Cattle on Grass) ----> Model Predicts: "Cow"
                                              |
                                    [Counterfactual Masking]
                                              v
Modified Image (Cattle on Sand)  ----> Model Predicts: "Camel" (Bias Detected!)

However, evaluating these counterfactuals requires care. As noted by Amar Kumar et al. in Debiasing Counterfactuals In the Presence of Spurious Correlations, standard metrics often fail to distinguish between a model tracking a true clinical change versus simply tracking the deletion of an artifact. They introduce the Spurious Correlation Latching Score (SCLS) to explicitly measure how much a counterfactual transformation alters known spurious elements versus relevant features.

To ensure these counterfactuals reveal meaningful biases rather than trivial variations, Massimo Caccia et al. propose in Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations to intentionally mask the most dominant features during the search process, forcing the system to uncover non-obvious attributes the model relies on. This builds on generative frameworks like the one proposed by Shusen Liu et al. in Generative Counterfactual Introspection for Explainable Deep Learning, which uses generative adversarial networks to provide actionable "knobs" that map out decision boundaries.

This interventionist logic applies equally to text. In Counterfactual Inference for Text Classification Debiasing, Chen Qian 0003 et al. mask primary content features versus context features to isolate and subtract class-level keyword biases during inference.

Sensitivity Scoring and Performance Gaps

When generating synthetic counterfactual images is too expensive or difficult, we can instead exploit the natural variations within a dataset by partitioning it based on the presence or absence of specific attributes.

Guangtao Zheng et al. formalize this approach in Learning Robust Classifiers with Self-Guided Spurious Correlation Mitigation and Spuriousness-Aware Meta-Learning for Learning Robust Classifiers using attribute-conditional sensitivity scoring. By measuring the absolute difference in model accuracy on subsets of data where a non-essential attribute is present versus where it is absent, they create a proxy metric for shortcut reliance. A large performance gap indicates that the classifier treats the attribute as a predictive crutch.

$$\text{Sensitivity} = | \text{Accuracy}_{\text{Attribute Present}} - \text{Accuracy}_{\text{Attribute Absent}} |$$

This paradigm is highly effective for tracking how biases evolve over time. In Issues for Continual Learning in the Presence of Dataset Bias and Continual Learning in the Presence of Spurious Correlations: Analyses and a Simple Baseline, Donggyu Lee et al. use classwise accuracy differences and classification flip rates under attribute modification to show that standard continual learning algorithms actively exacerbate the transfer of spurious correlations across sequential tasks.

In natural language processing, Lukáš Mikula et al. apply a similar partitioning strategy in Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models. They define a "Prediction bias" metric by splitting question-answering datasets using heuristic-based feature flags and bootstrapping the performance differences between the splits. For hate speech detection, Zhehao Zhang et al. design a validation metric in Mitigating Biases in Hate Speech Detection from A Causal Perspective that compares a feature's local predictive influence against its global importance, flagging features that the model has over-indexed on.

Stress Testing via Inverted Correlations

If a model has truly learned a shortcut, it expects the training correlation to hold up at test time. We can design the ultimate stress test by deliberately violating this expectation.

In Artifact-Based Domain Generalization of Skin Lesion Models, Alceu Bissoto et al. introduce inverted correlation evaluation. They construct "trap sets" where spurious features (such as medical rulers or skin hair) are correlated with target labels in the exact opposite direction of the training set.

If a skin lesion model relies on the presence of a ruler to predict malignancy, placing rulers exclusively on benign lesions in the test set will cause its accuracy to plummet. This catastrophic failure provides an unambiguous, quantifiable metric of how heavily the model was leaning on the artifact rather than the actual pathology.

Looking Inside the Network and the Dataset

While behavioral metrics treat the model as a black box, another line of research aims to quantify spuriosity by looking directly at internal representations or the data itself.

To pinpoint where shortcuts live inside a network, Phuong Quynh Le et al. introduce the neuron spurious score ($s$-score) in An XAI-based Analysis of Shortcut Learning in Neural Networks. By calculating the spatial overlap between a neuron's attention heatmap (via GradCAM) and a physical mask of the spurious feature, they quantify the exact percentage of a network's internal representation dedicated to tracking non-causal artifacts.

[GradCAM Attention Heatmap]  ∩  [Spurious Feature Mask]  ===>  Neuron Spurious Score (s-score)

Alternatively, we can quantify spuriosity before a model is even trained. Mazda Moayeri et al. demonstrate in Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases that datasets can be systematically organized by scoring samples based on the average activation of identified spurious features, allowing practitioners to isolate "clean" data points.

Taking this a step further, Barproda Halder et al. argue in Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition that dataset bias should be measured preemptively. They develop an information-theoretic metric ($M_{sp}$) using Partial Information Decomposition (PID). This framework quantifies a dataset's inherent spuriousness by measuring the net undesirable influence of non-target features, penalizing unique information residing in background features while accounting for whether core features are sufficient to predict the label.

Key Foundational Work

Counterfactual Inference for Text Classification Debiasing (2021) — contributed 1 concept elements to this direction.
Generative Counterfactual Introspection for Explainable Deep Learning (2019) — contributed 1 concept elements to this direction.
Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_spurious_correlation_quantification_acb465f0,
  title = {spurious correlation quantification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spurious-correlation-quantification/rdr_acb465f0b55646afb67837d83e5f2858},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_spurious_correlation_quantification_acb465f0_v1,
  title = {spurious correlation quantification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spurious-correlation-quantification/rdr_acb465f0b55646afb67837d83e5f2858/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake