Deep learning models in Natural Language Processing (NLP) are notorious for exploiting "spurious correlations" or shortcuts. For instance, a model might learn that the presence of the word "not" always implies a "contradiction" label in a logic task, bypassing any actual linguistic understanding of negation. To combat this, researchers typically "balance" datasets by adding counter-examples or using Adversarial Filtering (AF) to remove instances that simple models solve easily.
This position paper argues that such dataset balancing is a losing battle. The authors demonstrate that aggressive filtering eventually removes all learnable signals and that many so-called spurious correlations are actually essential "fallback" meanings that humans use to navigate an underspecified world.
The Futile Arms Race
The current paradigm in NLP benchmark development creates a cycle of diminishing returns. As researchers filter out simple correlations, they build larger models that simply find subtler, more complex shortcuts. This creates an "arms race" where the datasets become increasingly sanitized yet remain unlearnable in a way that reflects true linguistic competence.
This cycle is evident in the evolution of benchmarks like SWAG and HellaSWAG. While baselines on SWAG were initially low (~59%), larger models like BERT quickly reached human-level performance (~86%). This led to the creation of HellaSWAG, which employed more aggressive filtering, only for the cycle to repeat. Filtering does not necessarily improve generalization; it often just makes the shortcuts harder for humans to spot.
Theoretical Limits and Model Overconfidence
The paper provides a theoretical "XOR Argument" to show the limits of filtering. While a dataset can be balanced for single features (unigrams), it remains unbalanced for feature combinations (bigrams). If one were to truly balance for all possible $n$-gram combinations, all statistical signal would eventually be removed, leaving the model with nothing to learn.
Qualitative probing further highlights how models fail when forced to choose. In an experiment with RoBERTa-large fine-tuned on sentiment analysis (SST-2), the model assigned a 99.99% confidence score to a positive sentiment label for the ambiguous sentence: "To my great surprise, the movie turned out different than what I thought."
Gradient-based interpretations showed the model was almost entirely focused on the words "great" and "surprise," ignoring the structural ambiguity. This "forced choice" inductive bias encourages models to latch onto any available correlation to reduce loss, even when the context is insufficient.
Rethinking "Spurious" Correlations
The authors suggest that some correlations labeled as "spurious" are actually valuable world knowledge. For example, if a model learns that "The President of the U.S." usually refers to the current incumbent, that is a useful fallback for an underspecified question. Drawing on Grice’s Maxims, the paper reminds us that humans only explicitly mention a small fraction of the information needed for understanding; the rest is filled in via common-sense correlations.
Forcing a model to "unlearn" these via balancing can make it less useful in real-world communication. While social-bias correlations (e.g., gender stereotypes) must be mitigated, general world-knowledge fallbacks are a feature of human communication, not a bug.
Proposed Alternatives
Instead of the endless cycle of filtering, the authors propose several shifts in how we train and evaluate models:
- Abstention: Developing models that can say "I don't know" or ask for more context when an input is ambiguous rather than forcing a guess.
- Richer Contexts: Explicitly training models to handle complex linguistic phenomena like negation, sarcasm, and humor.
- Few-Shot Learning: Moving away from massive fine-tuning on biased datasets toward zero- or few-shot setups where models rely more on their pre-trained general knowledge.
Summary updated