Finding Dataset Shortcuts with Grammar Induction

Dan Friedman, Alexander Wettig, Danqi Chen

Abstract

Many NLP datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy. However, it is difficult to discover shortcuts automatically. Prior work on automatic shortcut detection has focused on enumerating features like unigrams or bigrams, which can find only low-level shortcuts, or relied on post-hoc model interpretability methods like saliency maps, which reveal qualitative patterns without a clear statistical interpretation. In this work, we propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets. Specifically, we use a context-free grammar to model patterns in sentence classification datasets and use a synchronous context-free grammar to model datasets involving sentence pairs. The resulting grammars reveal interesting shortcut features in a number of datasets, including both simple and high-level features, and automatically identify groups of test examples on which conventional classifiers fail. Finally, we show that the features we discover can be used to generate diagnostic contrast examples and incorporated into standard robust optimization methods to improve worst-group accuracy.

Introduction

Modern NLP models are frequently "right for the wrong reasons." Despite achieving state-of-the-art performance on benchmarks, they often rely on shortcuts—spurious correlations within the training data that do not represent the underlying task logic. For example, a sentiment analysis model might learn that any sentence containing the word "not" is negative, or a Natural Language Inference (NLI) model might assume that high word overlap between two sentences always implies a paraphrase.

Identifying these shortcuts is currently a manual, intuition-driven process. Researchers typically hypothesize a potential shortcut (such as lexical overlap) and then design experiments to confirm its presence. Automated tools exist but are limited: n-gram analysis only identifies low-level surface features, while saliency maps provide qualitative insights for individual examples without offering a statistical overview of the entire dataset. This paper introduces a framework to automatically discover complex, high-level shortcuts using grammar induction.

Methodology: Grammar Induction for Shortcut Discovery

The core of this approach involves treating the dataset as a language to be learned through probabilistic grammars. By modeling the data distribution, the authors can extract recurring structural patterns that correlate with specific labels.

1. Inducing the Grammar

The researchers utilize Probabilistic Context-Free Grammars (PCFGs) for single-sentence tasks and Synchronous PCFGs (SCFGs) for sentence-pair tasks. These grammars are parameterized using neural networks, enabling the system to learn the underlying structure of the data without requiring human-labeled parse trees. For sentence pairs, the SCFG explicitly models the transformation from one sentence to another, capturing operations like insertions, deletions, and substitutions.

2. Feature Extraction and Ranking

Once the grammar is trained, the dataset is parsed to extract all "complete subtrees." These subtrees represent recurring patterns ranging from single words to complex phrases. To identify which patterns constitute shortcuts, the authors calculate the Mutual Information (MI) between these subtrees and the dataset labels. A subtree with high MI is a strong candidate for a shortcut, as it serves as a highly predictive (but potentially spurious) signal for a specific class.

3. Diagnostic Testing

The discovered patterns are used to create "contrastive examples"—modified data points designed to test if a model's performance collapses when the shortcut is removed or subverted. This allows for a quantitative assessment of how much a model like BERT relies on these grammatical "crutches."

Experimental Evaluation

The method was evaluated on four major NLP benchmarks:

  • IMDb: Binary sentiment analysis of movie reviews.
  • SUBJ: Subjectivity vs. objectivity classification.
  • SNLI: Stanford Natural Language Inference.
  • QQP: Quora Question Pairs.

Discovered Shortcuts

The grammar induction method successfully rediscovered known shortcuts and identified new, subtle patterns:

  • IMDb: The system identified that negative reviews are three times more likely to mention a film's duration (e.g., "90 minutes"). It also isolated a specific non-terminal node representing numerical ratings (e.g., "8/10"), a well-documented shortcut in sentiment datasets.
  • SNLI: The grammar found that "Contradiction" labels are highly correlated with the presence of antonyms (e.g., man vs. woman), while "Entailment" often relies on hypernyms (e.g., replacing "man" with "person").
  • QQP: The system identified specific question structures and word-overlap patterns that models use to bypass actual semantic understanding.

Improving Model Robustness

Beyond discovery, the authors demonstrated that these grammar-based features can be used to train more reliable systems. By incorporating the discovered features into a robust optimization framework called DRiFt, they significantly improved "worst-group accuracy."

On a diagnostic set involving hypernyms in SNLI, the error rate of a standard BERT model was 21.8%. After applying the grammar-informed DRiFt method, the error rate dropped to 13.0%. This suggests that grammar-based features are more discriminative and better predictors of model failure than simple n-gram pairs.

Limitations and Conclusion

While powerful, the method faces computational challenges. The algorithm for parsing sentence pairs (bitext inside-outside) has a complexity of $O(N^6)$, which limits the length of sentences that can be processed. Additionally, while PCFGs are expressive, they may struggle with highly divergent or exceptionally complex sentence structures. Future work will focus on approximate inference methods to scale the system to larger datasets and longer sequences.

In conclusion, grammar induction provides a structured, human-readable, and statistically sound framework for uncovering the hidden shortcuts that modern NLP models exploit. By moving beyond simple word counts to structural patterns, researchers can better understand model behavior and build systems that are truly right for the right reasons.

Summary updated

Abstract

Many NLP datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy. However, it is difficult to discover shortcuts automatically. Prior work on automatic shortcut detection has focused on enumerating features like unigrams or bigrams, which can find only low-level shortcuts, or relied on post-hoc model interpretability methods like saliency maps, which reveal qualitative patterns without a clear statistical interpretation. In this work, we propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets. Specifically, we use a context-free grammar to model patterns in sentence classification datasets and use a synchronous context-free grammar to model datasets involving sentence pairs. The resulting grammars reveal interesting shortcut features in a number of datasets, including both simple and high-level features, and automatically identify groups of test examples on which conventional classifiers fail. Finally, we show that the features we discover can be used to generate diagnostic contrast examples and incorporated into standard robust optimization methods to improve worst-group accuracy.
BibTeX
@inproceedings{friedman2022finding,
  author = {Dan Friedman and Alexander Wettig and Danqi Chen},
  title = {{Finding Dataset Shortcuts with Grammar Induction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=3vLb99gHPy},
}
Figure