Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers

Parikshit Bansal, Amit Sharma

Abstract

To address the problem of NLP classifiers learning spurious correlations between training features and target labels, a common approach is to make the model's predictions invariant to these features. However, this can be counter-productive when the features have a non-zero causal effect on the target label and thus are important for prediction. Therefore, using methods from the causal inference literature, we propose an algorithm to regularize the learnt effect of the features on the model's prediction to the estimated effect of feature on label. This results in an automated augmentation method that leverages the estimated effect of a feature to appropriately change the labels for new augmented inputs. On toxicity and IMDB review datasets, the proposed algorithm minimises spurious correlations and improves the minority group (i.e., samples breaking spurious correlations) accuracy, while also improving the total accuracy compared to standard training.

Introduction

Machine learning models are notorious "lazy learners." When a specific word or pattern in a dataset correlates strongly with a label—even if it is not the actual cause—the model tends to latch onto it as a shortcut. For instance, in the IMDB sentiment dataset, a reviewer might mention a numerical rating like "8/10" within a negative review to contrast it with a better film. A model might see "8/10" and immediately classify the review as positive, ignoring the surrounding text. This phenomenon is known as a spurious correlation.

The standard remedy in NLP has been to force the model to be invariant to these features, essentially instructing the model to ignore them entirely. However, this is often a "sledgehammer" approach. In many cases, these features carry legitimate information; an "8/10" rating usually is associated with a positive sentiment. Completely removing the feature's influence can degrade the model's overall accuracy. The challenge lies in finding a middle ground: allowing a model to use a feature just enough to be helpful without letting it become a biased shortcut.

The FEAG Framework

This paper introduces Feature Effect Augmentation (FEAG), a two-phase framework designed to regularize rather than eliminate feature effects. Instead of aiming for "zero effect," FEAG aims for the "correct effect."

1. Causal Effect Estimation via Riesz Representers

The first phase treats specific text features (such as the word "kill" or a rating "8/10") as a "treatment" in a causal graph. To determine the true impact of this treatment on the label, the authors utilize a Riesz Representer (RR) Estimator.

Traditional causal tools, such as propensity-score methods, often fail in high-dimensional text data. They involve dividing by the probability of a feature appearing in a specific context; in text, these probabilities are often near zero, leading to high variance and instability (the "positivity" problem). The RR estimator bypasses this by directly modeling the weights needed to balance the data, providing a much more stable estimation of the true causal effect ($\tau$) of a feature.

2. Automated Augmentation and Label Softening

Once the "true" effect ($\tau$) is estimated, the framework generates counterfactual data. For a given sentence containing the feature, FEAG creates a version without it (and vice versa).

Crucially, the labels for these augmented samples are not simply copied from the original. Instead, the estimated $\tau$ is used to "soften" or adjust the label. This forces the classifier to learn a specific, controlled relationship between the feature and the output, ensuring the model's learned behavior matches the real-world causal effect identified in the first phase.

Experimental Evaluation

The authors evaluated FEAG across three distinct setups:

  • CivilComments Semi-Synthetic (SS): A toxicity dataset where a known feature effect was manually injected to test if the RR estimator could accurately recover it.
  • CivilComments Subsampled: A real-world toxicity dataset where the word "kill" acts as a spurious feature (frequently appearing in toxic contexts, but not inherently toxic).
  • IMDB Reviews: A sentiment analysis dataset where numeric ratings act as highly correlated but occasionally misleading features.

Key Results

FEAG consistently outperformed established baselines such as INLP (latent space removal) and Product of Experts (PoE).

  • Robustness vs. Accuracy: FEAG maintained high total accuracy while significantly boosting performance on "minority groups"—those specific samples where spurious correlations typically lead models astray.
  • Estimation Precision: On semi-synthetic data, the Riesz estimator proved significantly more accurate at identifying the true effect than standard direct or propensity-based methods.
  • IMDB Performance: On the IMDB dataset, FEAG achieved an Average Group Accuracy of 63.36%, compared to 56.25% for standard training, demonstrating its ability to handle features that are helpful but potentially misleading.

Discussion and Future Work

A secondary benefit of the FEAG approach is its utility in detecting annotator bias. By comparing the estimated effect of a word (e.g., identity terms like "gay" or "black") to its expected effect (which should be zero in a fair toxicity classifier), researchers can identify systematic biases in the humans who labeled the original training data.

Despite its success, the method currently relies on relatively simple counterfactual generation, such as adding or removing specific tokens. This is effective for discrete keywords but more difficult to apply to abstract concepts or stylistic features. Future research may integrate "neuro-counterfactual" generation—using Large Language Models to rewrite sentences—to address more complex spurious correlations beyond simple vocabulary.

Conclusion

FEAG represents a shift in how we approach model bias. By moving from a philosophy of feature invariance to one of causal regularization, we can build models that are both robust to shortcuts and sensitive to the genuine nuances of language.

Summary updated

Abstract

To address the problem of NLP classifiers learning spurious correlations between training features and target labels, a common approach is to make the model's predictions invariant to these features. However, this can be counter-productive when the features have a non-zero causal effect on the target label and thus are important for prediction. Therefore, using methods from the causal inference literature, we propose an algorithm to regularize the learnt effect of the features on the model's prediction to the estimated effect of feature on label. This results in an automated augmentation method that leverages the estimated effect of a feature to appropriately change the labels for new augmented inputs. On toxicity and IMDB review datasets, the proposed algorithm minimises spurious correlations and improves the minority group (i.e., samples breaking spurious correlations) accuracy, while also improving the total accuracy compared to standard training.
BibTeX
@inproceedings{bansal2023controlling,
  author = {Parikshit Bansal and Amit Sharma},
  title = {{Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=Pj0nzC9JCg},
}
Figure