Automated Discovery of Spurious Correlations in Neural Networks

This direction is merged into another direction: latent bias discovery. Its last version stays readable and citable here.

Version 1 of 162 papers65 conceptsfrozen 2026-10-04

Neural networks are notorious for being "lazy." When tasked with identifying a cow, a model might ignore the animal entirely and simply look for green grass. This phenomenon, often called the Clever Hans effect, occurs because superficial statistical shortcuts—spurious correlations—are often easier to learn than the actual underlying logic of a task.

While traditional debiasing required researchers to manually identify and label these shortcuts (e.g., "ignore the background"), a new research direction focuses on automated bias discovery. By leveraging the internal dynamics of learning, researchers are developing systems that can autonomously detect, isolate, and neutralize shortcuts that even humans might not have noticed.

The Intuition of the Weak Learner

The core insight behind many automated debiasing techniques is that shortcuts are "easy." In the early stages of training, or when using a model with very low capacity, the network will naturally gravitate toward these simple patterns before it attempts to learn complex semantic features.

Rabeeh Karimi mahabadi and James Henderson pioneered this via the Product of Experts (PoE) framework in Simple but effective techniques to reduce biases. The strategy is elegant: first, train a "bias-only" model (a weak learner) that is intentionally restricted to seeing only potential shortcuts. Then, train the main model to focus on the examples the weak learner fails to solve. By "learning from others' mistakes," as described by Victor Sanh and Thomas Wolf in Learning from others' mistakes: Avoiding dataset biases without modeling them, the main model is forced to ignore the easy shortcuts and find the true, robust features.

Identifying Shortcuts through Training Dynamics

Beyond using separate models, researchers have found that the training process itself leaves "fingerprints" of bias. Yining Wang and colleagues in Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse utilize a geometric phenomenon called Neural Collapse. They observe that models quickly map biased samples into tight clusters in the latent space. By identifying these prematurely collapsed regions, the ETF-Debias framework can proactively steer the model away from these easy paths.

Similarly, Miaoyun Zhao and Qiang Zhang demonstrate in Fine-Grained Bias Exploration and Mitigation for Group-Robust Classification that a model’s internal loss signals are sufficient to distinguish between "core" features and "spurious" ones. This bias-agnostic approach removes the need for expensive human-labeled metadata, allowing the model to self-correct by reweighting its own training data based on how "suspiciously easy" certain samples are.

Generative Intervention and Counterfactuals

If a model relies on a shortcut—like associating "water" with "bird"—the most direct way to break that link is to show the model a bird in a desert. However, collecting these "counter-shortcut" images is difficult.

Recent work uses generative models to create these scenarios synthetically. Eungyeup Kim and Jaegul Choo introduced BiaSwap (BiaSwap: Removing Dataset Bias with Bias-Tailored Swapping Augmentation), which automatically identifies bias attributes (like texture or background) and uses image-to-image translation to swap them. This forces the classifier to look at the object itself rather than its surroundings.

In the realm of interpretability, Matteo Pennisi and Giovanni Bellitto developed DiffExplainer (Diffexplainer: Towards Cross-modal Global Explanations with Diffusion Models), which uses Latent Diffusion Models to visualize what a model has learned. By generating images that maximize specific neuron activations, it can reveal hidden biases—such as a "bee" detector that is actually a "flower" detector—without any manual inspection.

From Post-hoc Audits to Autonomous Debugging

The field is shifting from merely explaining why a model failed to automatically fixing it. Yael Konforti and Mateo Espinosa Zarlenga propose CB-SLICE (CB-SLICE: Concept-Based Interpretable Error Slice Discovery), which uses Concept Bottleneck Models to link failures to specific semantic concepts. This allows for a "closed-loop" system where a model discovers its own systematic errors and triggers targeted retraining.

By automating the discovery of spurious correlations, researchers are moving toward a future where AI systems are not just accurate on a test set, but robust in the messy, unpredictable real world. As Enzo Tartaglione argues in his synthesis of bias correction (De la réduction de la complexité des modèles à la sélection des caractéristiques dans l'apprentissage profond), the ultimate goal is bias-agnostic training: creating models that can sense and ignore the "Clever Hans" shortcuts entirely on their own.

Papers in this version

Showing 60 of 62 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_automated_discovery_of_spurious_correlat_518b2d49,
  title = {Automated Discovery of Spurious Correlations in Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-discovery-of-spurious-correlations-in-neural-networks/rdr_518b2d49d6694c93b9fce431a28cb36e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_automated_discovery_of_spurious_correlat_518b2d49_v1,
  title = {Automated Discovery of Spurious Correlations in Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-discovery-of-spurious-correlations-in-neural-networks/rdr_518b2d49d6694c93b9fce431a28cb36e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}