TL;DR
As standard NLP benchmarks reach saturation, simple accuracy scores fail to capture how models truly differ. This paper introduces SCoPE, a framework that measures "shared invariances"—how similarly two models react to specific linguistic changes like synonyms or typos. It reveals that larger models possess a more diverse set of behavioral patterns that smaller models simply haven't "unlocked" yet.
What Problem Does This Paper Address?
We are currently in an era of "benchmark saturation." When multiple models all score 90%+ on a task like sentiment analysis, how do we know which one is actually better for real-world use?
Standard metrics (like IID accuracy) tell us what a model gets right, but they don't tell us why or how the model perceives the underlying language. Two models might have the same accuracy but arrive at their answers using completely different logic. One might be robust to typos, while the other collapses if a single letter is swapped. To truly compare models—especially when one is a "black box" API—we need to look at their shared invariances: the specific perturbations to an input that leave the model’s judgment unchanged.
Methods: The SCoPE Framework
The authors propose a framework called SCoPE (SHared-Capabilities-thrOugh-Perturbed-Examples). The process works in three steps:
- Define a Capability: Choose a linguistic trait to test, such as
Synonym-Invariance(replacing words with synonyms) orTypo-Invariance(swapping internal characters in a word). - Generate Invariant Perturbations: Using a "Reference Model" (e.g., BERT), the framework uses a greedy search to find specific changes to a sentence that do not change the Reference Model's output.
- Measure the Target: Take those same perturbed sentences and feed them to a "Target Model."
The authors introduce two specific metrics to quantify this relationship:
- Hard-SCoPE: A binary measure. Does the target model's top prediction (the argmax) stay the same as its original prediction when given the perturbation?
- Soft-SCoPE: A more granular measure. It looks at the change in the entire probability distribution. Even if the top prediction stays the same, does the "confidence" shift in the same direction and magnitude as the reference model?
Datasets
The study primarily utilizes standard benchmarks to validate its approach:
- SST2 (Stanford Sentiment Treebank): A standard binary sentiment classification dataset.
- AG’s News: A multi-class topic classification dataset used to validate results across different tasks.
- WordNet: Used as the engine for generating synonym-based perturbations.
- TextAttack: A library used to implement the search and transformation constraints.
Key Insights and Ideas
- The Asymmetry of Scale: One of the most striking findings is that larger models are "behavioral supersets" of smaller ones. Large models share almost all the invariances found in smaller models, but smaller models do not share the unique invariances of the giants. This suggests that "emergent properties" in LLMs might actually be the accumulation of a vast, diverse library of linguistic invariances.
- Distillation is a Leaky Bucket: When we "distill" a large model (like BERT) into a smaller one (like DistilBERT), we don't just lose a bit of accuracy. We lose specific robustness. The authors found that DistilBERT loses significantly more
Typo-InvariancethanSynonym-Invariance, likely because the pre-training objective (Masked Language Modeling) reinforces word-level meaning more than character-level robustness. - RLHF Retains More "Soul": When comparing OpenAI’s models,
text-davinci-003(trained with Reinforcement Learning from Human Feedback) shared more invariances with the base GPT-2 than its supervised-only counterpart (text-davinci-002). This suggests RLHF might be better at preserving the underlying behavioral "logic" of language than standard supervised finetuning.
Key Results
- Accuracy vs. Invariance: BERT and DistilBERT have a small accuracy gap (~3.5%), but their
Typo-Invariancealignment is much lower (84%) compared to theirSynonym-Invariance(96%). This highlights how standard accuracy masks deeper behavioral differences.
- Scale Trends: In the BERT family, as model size increases (from Tiny to Base), the models become more similar to each other in terms of shared invariances, regardless of which one is used as the reference.
- Black-Box Insights: Using GPT-2 as a reference, the authors showed that larger InstructGPT models (like
davinci) are significantly more "agreeable" to GPT-2's invariances than smaller versions likeada, providing a way to "fingerprint" the behavior of closed-source models.
Limitations and Future Work
- Search Efficiency: The current method uses a greedy search to find perturbations. This is computationally expensive and requires many queries to the model, which can be costly when testing commercial APIs.
- Semantic Uncertainty: The paper notes that estimating the output probabilities of instruction-tuned models (which often output text rather than log-probs) is an open challenge. They used a "multiple sample" approach to estimate probabilities, but more robust methods are needed.
- Expanding Capabilities: The study focused on synonyms, typos, and fairness. Future work could expand this to more complex linguistic features like negation, passive voice, or cross-lingual invariances.
Summary updated