Abstract
The term "spurious correlations" is frequently used in Natural Language Processing (NLP) to describe undesirable feature-label correlations that models exploit as shortcuts. However, this broad categorization fails to distinguish between features that are truly irrelevant to a task and those that are necessary but insufficient for prediction. This paper introduces a causal framework using the probabilities of necessity (PN) and sufficiency (PS) to delineate these relationships. We demonstrate that standard debiasing methods often fail because they treat these distinct feature types identically. Our analysis reveals that while models can become invariant to irrelevant features, they must retain and properly contextualize necessary features to achieve robust performance.
Introduction
NLP models are notorious for relying on "shortcuts"—superficial cues that correlate with labels in training data but fail to generalize. For instance, in Natural Language Inference (NLI), a model might learn that the presence of the word "not" always indicates a contradiction. Traditionally, the research community has grouped all such patterns under the umbrella of spurious correlations.
However, not all shortcuts are created equal. There is a fundamental difference between a model relying on a specific punctuation mark (an irrelevant feature) and a model relying on negation (a necessary but insufficient feature). This paper argues that a more fine-grained treatment is required to specify desired model behavior and evaluate robustness accurately.
A Causal Taxonomy of Features
To formalize the distinction between different types of features, we utilize a Structural Causal Model (SCM) and two key metrics from causal inference:
- Probability of Necessity (PN): This measures how likely the label is to change if the feature is removed or altered. For example, removing a negation word often flips the sentiment of a sentence, indicating high necessity.
- Probability of Sufficiency (PS): This measures how likely the feature is to produce the label on its own in a neutral context. Adding "not" to a random sentence does not guarantee a specific label without the surrounding context, indicating low sufficiency.
Based on these metrics, we categorize features into two primary groups:
- Irrelevant Features (Low PN, Low PS): These are truly spurious. They are neither necessary nor sufficient for the task (e.g., a specific font or punctuation mark correlated with a label).
- Necessary but Insufficient Features (High PN, Low PS): These features are essential building blocks of meaning. They are "spurious" only when a model relies on them exclusively without considering the broader context.
Experimental Methodology
We evaluated how standard debiasing methods interact with these feature types across several benchmarks, including MNLI, the HANS challenge set (which targets lexical overlap heuristics), and synthetic datasets designed to isolate specific biases.
The debiasing methods tested include:
- Subsampling: Balancing the dataset so the feature no longer correlates with the label.
- Product-of-Experts (POE): Training a model to focus on examples that a "bias-only" model gets wrong.
- Iterative Null-space Projection (INLP): Linearly removing feature information from the model's internal representations.
Key Insights and Results
The Destructive Nature of Feature Removal
Our findings indicate that methods like INLP, which attempt to physically remove feature representations, are highly effective for irrelevant features (low PN) but catastrophic for necessary features (high PN). For example, attempting to "delete" the representation of negation words prevents a model from ever performing logical reasoning correctly. You cannot remove the concept of negation and expect a model to understand contradiction.
The Persistence of Necessary Features
Interestingly, data balancing (subsampling) does not erase necessary features from a model's representation. In fact, for high PN features like lexical overlap, the model often encodes the feature more clearly after debiasing. This suggests that the model is learning to use the feature more precisely as part of a complex logical chain rather than as a simple shortcut.
Performance Trade-offs
On the HANS dataset, we observed a significant performance gap (43.7%) in RoBERTa-large between entailment and non-entailment cases, confirming a heavy reliance on lexical overlap as a shortcut. While debiasing improved performance on challenge sets, it often came at the cost of "in-distribution" accuracy when the features involved were high PN.
Discussion and Conclusion
The primary takeaway is that the goal of debiasing should not always be invariance. While we want models to be invariant to irrelevant features (like punctuation), we want them to be sensitive to necessary features (like negation) in a context-dependent way.
Future work should move away from trying to "remove" spurious features and instead focus on training models to match the true causal distribution of language. This involves ensuring that necessary features are only utilized when the surrounding context justifies their application. As NLP moves toward larger models and more complex reasoning, distinguishing between noise and the essential building blocks of logic will be critical for building truly robust systems.
Summary updated