Abstract
We study few-shot debugging of transformer-based natural language understanding (NLU) models, using recently popularized test suites to not just diagnose but correct a problem. Given a few debugging examples of a certain phenomenon, and a held-out test set of the same phenomenon, we aim to maximize accuracy on the phenomenon at a minimal cost of accuracy on the original test set. We examine several methods that are faster than full epoch retraining. We introduce a new fast method, which samples a few "in-danger" examples from the original training set. Compared to fast methods using parameter distance constraints or Kullback-Leibler divergence, we achieve superior original accuracy for comparable debugging accuracy.
Introduction
Deep learning models for Natural Language Understanding (NLU) often exhibit systematic failures despite high aggregate accuracy. A sentiment analysis model might consistently fail on negation (e.g., "not good"), or a question-answering system might struggle with the passive voice. When these "bugs" are identified via diagnostic test suites like CheckList or HANS, practitioners face a difficult choice: accept the error, or undergo the expensive process of collecting thousands of new examples and retraining the entire model.
This research addresses the need for few-shot debugging—a method to fix specific linguistic failures using only a handful of examples (e.g., 10) in a matter of seconds, without triggering "catastrophic forgetting" of the model's original capabilities.
The Challenge of Fast Debugging
Traditional fine-tuning on a small set of error cases (the "Debug Only" approach) typically causes the model to overfit to the new examples, leading to a sharp decline in performance on the original task. To prevent this, researchers have explored several "fast" constraints:
- Weight Constraints ($L^2$ and $L^\infty$): Forcing the model's weights to remain close to their original values during the update.
- Kullback-Leibler (KL) Divergence: Adding a penalty to ensure the model's output distribution does not drift significantly from the original model's predictions on random samples.
While these methods are faster than full retraining, they often struggle to balance the fix with the preservation of original knowledge.
Proposed Method: In-Danger Sampling
We propose a novel sampling technique designed for targeted recovery. Rather than using random samples to preserve original performance, we identify the specific parts of the original training set most likely to be corrupted by the debug update.
- Dry Run: We perform an initial debugging update on the 10 new examples.
- Identify "In-Danger" Examples: We scan the original training set to find samples that the model correctly classified before the update but misclassified after the update.
- Final Update: We perform the actual debugging update using a combination of the 10 debugging examples and a small set (typically 20) of these "in-danger" examples.
This approach ensures that the model receives a corrective signal exactly where its internal representations are most volatile.
Experimental Framework
We evaluated our method across three major NLU tasks and their corresponding diagnostic suites:
- MNLI (Natural Language Inference): Evaluated using the HANS suite to test for lexical overlap biases.
- SST-2 (Sentiment Analysis): Evaluated using CheckList to test negation and neutral phrase handling.
- QQP (Quora Question Pairs): Evaluated using CheckList to test active/passive voice swaps.
Results and Key Insights
1. Superior Accuracy Retention
Our "in-danger" sampling method achieved the highest original accuracy retention in 11 out of 13 test cases compared to other fast methods. By focusing on the most vulnerable original samples, we effectively regularized the model against catastrophic forgetting.
2. Debugging as Generalization
Interestingly, training on just 10 examples of a specific phenomenon often improved the model's performance on hundreds of unseen examples of that same phenomenon. This suggests that few-shot debugging is not merely memorization; it nudges the transformer toward a more robust internal representation of linguistic rules.
3. Massive Efficiency Gains
Speed is the primary advantage of this approach. While "slow" methods like oversampling require hours of computation (12,000 to 25,000 seconds), our proposed method completes the update in approximately 25 seconds—a 720x speedup. This makes it feasible for interactive debugging workflows.
Conclusion and Future Work
This work demonstrates that NLU models can be patched quickly and effectively using a targeted sampling strategy. While the current study utilizes template-based test suites (HANS and CheckList), the underlying principle of "in-danger" sampling is broadly applicable. Future research will explore the application of this method to "wild" errors found in production logs and its scalability to even larger transformer architectures where full retraining is computationally prohibitive.
Summary updated