Many NLP models now come with "rationales": the handful of input words the model says it relied on. New rationale methods are usually judged by whether deleting those words changes the prediction, and are often reported as beating older methods by a few points. We suspect that many of these reported gains are not about the method at all. They may come from three other sources. First, one method received more tuning of its loss weights and training settings than the others. Second, the methods returned rationales of different lengths and shapes. Third, the tuning itself rewarded models that behave strangely on text with words deleted, which is a known way to game the deletion test. We propose a controlled study that gives every rationale method the same tuning budget, forces every method to output rationales under the same exactly enforced length and shape rules, and measures how much of any faithfulness gain survives once the gaming route is closed.
The payoff is a clearer scientific record for explainable NLP. A comparable correction has already happened for accuracy. When character-level Transformers and LSTMs were both tuned thoroughly, the claimed Transformer advantage largely disappeared (Quantifying the Hyperparameter Sensitivity of Neural Networks for Character-level Sequence-to-Sequence Tasks). No one has run that correction for explanation quality. The risk may be larger there, because the objectives involved can be gamed.
Core Claim
Our hypothesis has two parts. First, under equal tuning and equal rationale shape, the ranking of rationale methods on faithfulness and human agreement will change on a substantial fraction of the benchmark tasks drawn from ERASER: A Benchmark to Evaluate Rationalized NLP Models. Second, when hyperparameter search optimizes the deletion-based faithfulness scores (defined below), part of the improvement it finds will be explained by how easily the tuned model can tell that words have been deleted. That is the same weakness that Goodhart's Law Applies to NLP's Explanation Benchmarks exploited deliberately. We call this second part "tuning-induced Goodharting": search over loss weights can rediscover the gaming strategy without anyone intending it.
The new element is the combination of the two, not either one alone. Sensitivity metrics for hyperparameters exist but have only been applied to accuracy. The gaming of deletion metrics has been shown, but only with a hand-built wrapper, never as something that tuning finds by itself.
Background and Where This Sits
The faithfulness scores come from ERASER. Its comprehensiveness score is the drop in the predicted class's probability when the rationale is removed. Its sufficiency score is the drop when only the rationale is kept. ERASER also supplies the plausibility metrics, which measure token and span overlap with human-marked rationales.
The strongest method we test is REFER. It trains a token selector jointly with the classifier, using an estimator called AIMLE to pass gradients through the discrete keep-or-drop choices. It weighs four losses: task, sufficiency, comprehensiveness, and human agreement. It reports gains on e-SNLI and CoS-E, but gives no systematic search over those four weights. That absence is the opening this proposal targets.
e-SNLI motivates the project from a different angle. It showed that a hypothesis-only model reaches 67% label accuracy but writes correct explanations only about 7% of the time. Explanation quality therefore carries information that accuracy does not, and it deserves its own tuning analysis.
Two papers constrain our design. The Goodhart paper shows that deleting words produces unnatural text, which models can detect. With a simple wrapper, comprehensiveness on Movies rose from 0.18 to 0.89, and random highlights reached a combined faithfulness score of 1.00 on BoolQ. Any faithfulness number we report must therefore be paired with a check for this detection route. Robust Explanation for Free or At the Cost of Faithfulness shows that smoothing a model to stabilize its explanations eventually erodes faithfulness. We therefore do not use smoothing as the fix.
Mixed-integer linear programming (MILP) supplies a tool rather than the headline. MILP is optimization in which some variables must be whole numbers, here 0 or 1 for "keep this token." In tabular recourse work, OrdCE and DACE use such formulations to enforce hard rules exactly. OrdCE reports more than 50% lower cost than greedy ordering. We borrow that property for a narrower purpose. Whatever token scores a method produces, we turn them into a rationale with the same exact rules for every method. Those rules are a token budget, a cap on the number of contiguous spans, and coverage rules such as "at least one span in the premise and one in the hypothesis" for e-SNLI, or "at least one span per cited sentence" for FEVER and MultiRC. A standard integer-programming solver handles these rules. Unlike REFER's gradient approximation, it guarantees they hold.
The hyperparameter-sensitivity metrics come from Wiemerslage et al. We reuse relative performance at the first $k$ trials, the percentage of best-equivalent runs, the number of local maxima, and the rank correlation between hyperparameter distance and performance. We apply them for the first time to comprehensiveness, sufficiency, and plausibility.
Research Program
Tasks. We use ERASER's Movies, BoolQ, FEVER, MultiRC, e-SNLI, CoS-E, and Evidence Inference. These span short sentence pairs and full clinical trial reports. For language diversity we add the English and Chinese sentiment, textual similarity, and reading-comprehension tasks from A Fine-grained Interpretability Evaluation Benchmark for Neural NLP, whose perturbation-based consistency score is a faithfulness signal that does not rely on deletion.
Methods compared. We compare three families:
- a post-hoc family: attention or gradient saliency, thresholded into a rationale;
- a two-stage extract-then-predict pipeline;
- REFER.
Each family is trained on one shared mid-sized pretrained encoder.
Tuning. Each method on each task receives the same random-search budget of 60 configurations. The search space covers learning rate, dropout, and every loss weight the method exposes. We record the full grid of results, so any smaller budget can be simulated after the fact and the ranking of methods can be traced as a function of budget.
Rationale selection. Every method's rationales are produced twice. The first uses its native selection rule. The second uses the shared exact selector with fixed length and shape rules. This separates what the method scores from how its scores become a rationale.
Detecting gaming. For every tuned configuration we measure a "masked-text detectability" score: how accurately a small probe on the model's internal representations separates rationale-deleted inputs from natural inputs. We also recompute faithfulness under a reference evaluator. This is a copy of the classifier fine-tuned with random word deletions, so that deleted-word text is no longer unusual to it. The difference between standard and reference faithfulness is each configuration's "Goodhart gap." Tuning-induced Goodharting predicts that configurations chosen by searching standard comprehensiveness will show larger Goodhart gaps and higher detectability than configurations chosen at random.
Baselines
Reported numbers. The naive reference point is each method as published, with default or reported hyperparameters and native rationale selection. For REFER, the number to reproduce is its reported 11% improvement in composite Normalized Relative Gain on e-SNLI and 3% on CoS-E (REFER). We first rerun these settings to confirm we are in the same regime. The file contains no per-task ERASER numbers for the original baselines ("no reported result found on Lacuna for ERASER baseline comprehensiveness per task"), so we rerun those baselines ourselves under identical data splits.
REFER, fully tuned. This is the strongest competitor because it optimizes faithfulness and plausibility directly and holds the best reported gains. It receives the same 60-trial budget as everyone else, with its four loss weights searched. It "wins" the question if its margin over the tuned pipeline and tuned post-hoc methods satisfies three conditions:
- it stays positive under equal tuning and exact shape equalization;
- it exceeds the spread between best-equivalent runs;
- it survives the reference evaluator.
Tuned post-hoc saliency. This is the simplest serious competitor. It needs no extra training objective, and our hypothesis predicts that it closes much of the gap once rationale shape is equalized.
Ablation 1: selection rule. We compare native selection against exact shared selection while holding tuning fixed. This isolates how much of a method's advantage is simply producing different-shaped rationales.
Ablation 2: tuning objective. We compare tuning on standard comprehensiveness against tuning on reference-evaluator comprehensiveness. This isolates the Goodharting effect.
For every comparison, the proposal counts as having found something if two things hold:
- the Kendall rank correlation between method rankings at the published setting and at the equal-tuned, equal-shape setting falls clearly below 1 on several tasks;
- selecting configurations for standard faithfulness raises detectability by more than the between-seed spread.
Why not just…?
Why not just report several random seeds instead of full hyperparameter search? Seeds measure noise at one configuration. They do not measure how far that configuration sits from the best one. The Wiemerslage et al. study found that Transformers had more local maxima and fewer best-equivalent runs than LSTMs, and that effect only shows up across configurations. We report seed spread as well, and use it as the threshold that tuning effects must exceed.
Why not just use top-k thresholding, or dynamic programming, instead of an integer-programming solver? This is a real threat to the MILP ingredient, and we accept it. For a single budget plus contiguity along one sequence, dynamic programming is exact and faster. We will verify that it returns identical rationales to the solver in those cases. The solver is needed only for the coupled rules: multiple coverage requirements across premise and hypothesis, or across evidence sentences, together with a span cap. Writing a custom dynamic program for each task's rule set is brittle. If the exact selector never changes any method ranking relative to plain top-k, we will report that shape equalization is unnecessary and drop the ingredient.
Why not just default hyperparameters, which work well in modern libraries? Defaults were chosen for task accuracy, and nobody chose them for faithfulness. REFER's four loss weights also have no library default. The first ablation directly measures how far the published settings sit from the tuned optimum, so this proposal answers the question empirically either way.
Why not just switch to a faithfulness metric that does not delete words? We do include one: the perturbation-consistency score from the fine-grained benchmark. However, deletion-based ERASER scores remain the field's standard. Showing that tuning pressure alone drives models toward gaming those scores is itself the finding that justifies moving away from them. The reference evaluator gives a deletion-based number that is harder to game, and it stays comparable to published results.
Decisive Checks
The study succeeds if three results hold. First, method rankings under equal tuning and equal shape differ from published rankings on at least two ERASER tasks. Second, configurations selected for standard comprehensiveness show higher masked-text detectability and a larger Goodhart gap than random configurations. Third, the sensitivity metrics show that faithfulness is more hyperparameter-sensitive than accuracy for the same model, meaning fewer best-equivalent runs and more local maxima.
The hypothesis fails if REFER's margin persists across all three controls. That outcome would be equally useful to publish: it would be the first tuning-controlled confirmation that explanation-regularized training produces rationales that are both more faithful and harder to game.
References
- Quantifying the Hyperparameter Sensitivity of Neural Networks for Character-level Sequence-to-Sequence Tasks
- ERASER: A Benchmark to Evaluate Rationalized NLP Models
- Goodhart's Law Applies to NLP's Explanation Benchmarks
- REFER: An End-to-end Rationale Extraction Framework for Explanation Regularization
- e-SNLI: Natural Language Inference with Natural Language Explanations
- Robust Explanation for Free or At the Cost of Faithfulness
- Ordered Counterfactual Explanation by Mixed-Integer Linear Optimization
- DACE: Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization
- A Fine-grained Interpretability Evaluation Benchmark for Neural NLP