Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens

Nitish Joshi, Xiang Pan, He He

Introduction

In Natural Language Processing (NLP), "spurious correlations" are often viewed as monolithic shortcuts—superficial features like the word "not" or a specific punctuation mark that models exploit to achieve high accuracy without true understanding. Standard debiasing techniques aim to force models to ignore these features. However, this paper argues that this broad definition overlooks a critical distinction: some features are truly irrelevant, while others are necessary but insufficient components of the task's logic.

A Causal Framework for Bias

The authors formalize the distinction between different types of bias using a causal lens. They identify two primary types of dependence:

  1. Type 1 (Irrelevant): Features with no causal link to the label (e.g., a director's name in sentiment analysis).
  2. Type 2 (Necessary): Features that are causally linked but insufficient on their own (e.g., negation words in inference).
Causal graphs distinguishing between Type 1 (irrelevant) and Type 2 (necessary) dependencies.

To quantify these, the paper introduces two metrics from causality theory:

  • Probability of Necessity (PN): Measures how likely the label is to change if the feature is modified. High PN indicates a feature is essential.
  • Probability of Sufficiency (PS): Measures how likely a feature is to produce a specific label when added to a neutral context. High PS indicates a "smoking gun" feature.

A Taxonomy of Spuriousness

By combining PN and PS, the authors categorize features into four distinct buckets. This taxonomy reveals why treating all biases the same way is problematic.

Taxonomy of features based on Probability of Necessity (PN) and Probability of Sufficiency (PS).
  • Irrelevant (Low PN, Low PS): Features like punctuation that have no logical bearing on the task.
  • Incomplete/Necessary (High PN, Low PS): Features like "not" that are essential for logic but don't guarantee a label.
  • Redundant (Low PN, High PS): Features that strongly signal a label but aren't strictly necessary.
  • Robust (High PN, High PS): Features that are both necessary and sufficient.

Key Findings and Implications

The study demonstrates that debiasing is a "double-edged sword." For truly irrelevant features (Low PN), debiasing improves robustness without harming performance. However, for necessary features (High PN), attempting to remove the feature's influence leads to a significant drop in task accuracy. The model requires this information to construct the complex, legitimate logic needed for the task.

Furthermore, the authors show that even after rigorous debiasing or data balancing, High PN features remain extractable from the model's internal representations. This suggests that bias persistence isn't always a failure of the debiasing method; rather, it reflects the model's use of these features as building blocks for higher-level reasoning.

Conclusion

The paper concludes that invariance is not always a desirable goal. For features with high necessity, an ideal model—much like a human—should not be invariant. Future work in NLP debiasing must move beyond "one-size-fits-all" approaches and account for the causal role features play in linguistic logic.

Summary updated

Abstract

The term `spurious correlations' has been used in NLP to informally denote any undesirable feature-label correlations. However, a correlation can be undesirable because (i) the feature is irrelevant to the label (e.g. punctuation in a review), or (ii) the feature's effect on the label depends on the context (e.g. negation words in a review), which is ubiquitous in language tasks. In case (i), we want the model to be invariant to the feature, which is neither necessary nor sufficient for prediction. But in case (ii), even an ideal model (e.g. humans) must rely on the feature, since it is necessary (but not sufficient) for prediction. Therefore, a more fine-grained treatment of spurious features is needed to specify the desired model behavior. We formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates the causal relations between a feature and a label. We then show that this distinction helps explain results of existing debiasing methods on different spurious features, and demystifies surprising results such as the encoding of spurious features in model representations after debiasing.
BibTeX
@inproceedings{joshi2022are,
  author = {Nitish Joshi and Xiang Pan and He He},
  title = {{Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=g45-duDigA},
}
Figure