Principles from Clinical Research for NLP Model Generalization
Generalization failure in NLP is frequently attributed to "out-of-distribution" (OOD) data. However, this paper argues that the root cause is often a lack of internal validity—the model never truly learned the underlying task, but instead exploited "shortcuts" or spurious correlations within the training data. By borrowing principles from clinical research, the authors propose a framework to distinguish between true task mastery and surface-level pattern matching.
The Hierarchy of Validity
In clinical trials, internal validity ensures that an observed effect is caused by the treatment rather than confounding factors. Applying this to NLP, the authors argue that internal validity—establishing intra-test set model literacy—is a prerequisite for any form of generalization.
As shown in Figure 1, internal validity serves as the foundation. Without ensuring the model has learned the correct linguistic capabilities to produce its performance, claims of cross-domain or cross-task generalization are premature. The community often ignores this "internal" check, jumping straight to external validity (generalization).
Methodology: Detecting Shortcuts via Surrogate Modeling
To investigate model behavior, the authors use Interpretable Surrogate Modeling. They train simple, transparent models (e.g., Multinomial Naive Bayes) using only "surface patterns" like entity distance or sentence length. They then measure the Cohen’s Kappa ($\kappa$) correlation between these surrogates and complex models like BERT. A high correlation indicates the complex model is likely relying on the same simple shortcuts rather than linguistic logic.
Key Results and Shortcut Reliance
The empirical results reveal significant shortcut reliance across multiple benchmarks. As detailed in Figure 3, in the PTM-PPI task, the surrogate model (NB-T) achieved a correlation of $\kappa = 0.73$ with the complex model’s actual predictions on the test set (TS (MP)), which is significantly higher than its correlation with the ground truth ($\kappa = 0.55$). On the generalization set (GH), this correlation rose to a staggering $\kappa = 0.85$. This provides a "smoking gun" that the model is more "in sync" with surface shortcuts (like the distance between entities) than with the actual biological truth.
Similarly, in the SNLI benchmark, surrogate models reached a $\kappa$ of 0.34 by merely observing sentence length and negations, confirming that these heuristics remain a major factor in NLI performance.
Causal Matching and Contrast Sets
To move beyond correlation, the authors suggest adapting "matching" from randomized controlled trials using contrast sets. These are pairs of examples where a minor, label-flipping change is made (e.g., changing "good" to "bad"). If a model's prediction remains unchanged despite a linguistically significant perturbation, it lacks the internal validity required for robust performance. Expert verification on the generalization set showed that only 20% of the model's "correct" predictions were actually right for the right reasons.
Conclusion
The NLP community must move beyond the "OOD excuse." High benchmark scores do not equate to task literacy. The authors call for the standard adoption of internal validity checks, such as surrogate correlation analysis and contrast set evaluation, to ensure models are truly learning linguistic capabilities rather than memorizing dataset-specific signatures.
Summary updated