Finding Dataset Shortcuts with Grammar Induction

Dan Friedman, Alexander Wettig, Danqi Chen

Introduction

Machine learning models in Natural Language Processing (NLP) often achieve high performance by exploiting "shortcuts"—spurious correlations within a dataset that do not reflect true linguistic understanding. For instance, a model might learn to classify a movie review as negative simply because it mentions the word "length," or assume two sentences are paraphrases based solely on high lexical overlap. While these heuristics work within specific benchmarks, they fail in real-world applications. Existing diagnostic tools, such as unigram analysis or saliency maps, often fail to capture the complex structural patterns that models use as crutches. This paper introduces an automated method using grammar induction to identify these high-level, structural shortcuts across large-scale datasets.

Methodology

The core of the proposed approach is the use of Grammar Induction to model dataset patterns without relying on predefined linguistic rules.

  1. Inducing the Grammar: The researchers train Probabilistic Context-Free Grammars (PCFG) for single-sentence tasks and Synchronous PCFGs (SCFG) for sentence-pair tasks like Natural Language Inference (NLI). These grammars are induced directly from the data, allowing them to capture the specific structures most prevalent in the benchmark.
  2. Feature Extraction: Sentences are parsed into trees, and specific subtrees are identified. The authors calculate the Mutual Information between these subtrees and the ground-truth labels to find highly predictive structural features.
  3. Weakness Identification: The dataset is partitioned into "supporting examples" (where the shortcut aligns with the label) and "counter-examples" (where the shortcut is present but the label differs). A significant performance drop on counter-examples indicates that the model has latched onto the shortcut.
Grammar induction identifies structural subtrees (left) that correlate with specific labels in NLI tasks (right).

Empirical Analysis and Insights

The method was evaluated on four major benchmarks: IMDb (sentiment), SUBJ (subjectivity), SNLI (inference), and QQP (duplicate detection).

Shortcuts are Structural

The grammar induction revealed that shortcuts are often more complex than simple word correlations. In the SNLI dataset, the "contradiction" label was found to be highly correlated with structural patterns involving antonym replacement (e.g., swapping "man" for "woman" or "jumps" for "sits").

Model Vulnerability

State-of-the-art models like BERT and RoBERTa consistently exploit these structural cues. In the case of the "Verb Antonym" shortcut in SNLI, BERT’s accuracy plummeted from nearly 100% on supporting examples to just 55.6% on counter-examples. This suggests that the model is "lazy," prioritizing simple structural templates over deep semantic reasoning.

Grammars as Diagnostic Tools

Because the induced grammar provides a formal template, it allows for "contrastive edits." Researchers can systematically modify sentences (e.g., replacing a hypernym like "man" with "human") to pinpoint exactly where a model's logic breaks down. These grammar-based features proved to be significantly more discriminative than standard n-gram features.

Results and Robustness

The method successfully rediscovered known shortcuts, such as numerical ratings in IMDb reviews (e.g., "8/10" predicting positive sentiment), while also uncovering previously undocumented structural biases.

To mitigate these issues, the discovered grammar features were integrated into the DRiFt robust optimization algorithm. This approach forced the models to ignore the identified shortcuts, leading to measurable improvements in robustness. On hard "Antonym" test sets, the error rate was reduced from 71.1% to 68.7%, and on "Hypernym" sets, it dropped significantly from 21.8% to 13.0%.

Conclusion and Future Work

While effective, the current method faces scalability challenges due to the $O(n^6)$ computational complexity of SCFG parsing, limiting its use to shorter sentence pairs. Additionally, the assumption that a single parse tree can represent complex relationships may be too simplistic for tasks like document summarization. Future research will focus on extending this framework to more diverse languages and more complex NLP tasks to further enhance model reliability and transparency.

Summary updated

Abstract

Many NLP datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy. However, it is difficult to discover shortcuts automatically. Prior work on automatic shortcut detection has focused on enumerating features like unigrams or bigrams, which can find only low-level shortcuts, or relied on post-hoc model interpretability methods like saliency maps, which reveal qualitative patterns without a clear statistical interpretation. In this work, we propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets. Specifically, we use a context-free grammar to model patterns in sentence classification datasets and use a synchronous context-free grammar to model datasets involving sentence pairs. The resulting grammars reveal interesting shortcut features in a number of datasets, including both simple and high-level features, and automatically identify groups of test examples on which conventional classifiers fail. Finally, we show that the features we discover can be used to generate diagnostic contrast examples and incorporated into standard robust optimization methods to improve worst-group accuracy.
BibTeX
@inproceedings{friedman2022finding,
  author = {Dan Friedman and Alexander Wettig and Danqi Chen},
  title = {{Finding Dataset Shortcuts with Grammar Induction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=3vLb99gHPy},
}
Figure