TL;DR
Minimal-pair paradigm (MPP) datasets are widely used to evaluate language models' linguistic knowledge, but this paper reveals a significant "token-length bias." This bias, stemming from subword tokenization, causes models to favor shorter token sequences, leading to inaccurate evaluations. The authors demonstrate that common normalization techniques fail to fix this, highlighting the need for future MPP datasets to strictly control token lengths.
What Problem Does This Paper Address?
Evaluating the linguistic competence of large language models (LLMs) is crucial. One popular method uses "minimal-pair paradigm" (MPP) datasets. These datasets consist of pairs of sentences that are identical except for one word, where one sentence is grammatically acceptable and the other is not. Models are evaluated by how often they assign a higher "acceptability score" (typically sentence log-likelihood) to the acceptable sentence.
A key challenge in creating these datasets is controlling for sentence length, as longer sentences naturally have lower log-likelihoods. While MPP datasets typically ensure the word count is similar, modern LLMs use subword tokenization (e.g., breaking "tokenization" into "token" and "ization"). This means sentences with the same word count can have different token counts. This paper argues that this difference in token length introduces a "token-length bias," where models might unfairly favor shorter token sequences, regardless of their actual grammatical acceptability. This bias could lead to misinterpreting a model's true linguistic understanding.
Methods
The researchers investigated two main research questions:
- RQ1: Does token-length bias affect MPP dataset evaluations?
- They analyzed existing MPP datasets (BLIMP, CrowS-Pairs, COMPS) by splitting them based on whether the acceptable sentence had more, fewer, or the same number of tokens as the unacceptable sentence.
- They then compared the models' accuracy across these splits. If token-length bias were absent, accuracy should be consistent.
- They also examined the correlation between sentence log-likelihood and token length for various models (GPT-2, BERT, RoBERTa, ELECTRA).
- RQ2: Is normalized log-likelihood effective in mitigating token-length bias?
- They applied two normalization techniques, MeanLP and PenLP, to the sentence log-likelihood scores.
- They analyzed how these normalized scores correlated with token length and how they affected accuracy across different token length differences.
Datasets
The study primarily used three English MPP datasets:
- BLIMP (Benchmark of Linguistic Minimal Pairs): A large dataset covering 67 linguistic paradigms.
- CrowS-Pairs: A dataset focused on measuring social biases and stereotypes.
- COMPS (Conceptual Minimal Pair Sentences): A dataset designed to test robust property knowledge and its inheritance.
The authors also generated a debiased version of the BLIMP dataset, named FairBLIMP, using their proposed method to control for token length.
Key Insights and Ideas
- Token length, not just word length, matters: Modern subword tokenization means that even sentences with identical word counts can have different token lengths, and this difference significantly impacts model evaluations.
- Shorter token sequences are unfairly favored: The study found a consistent trend where models assigned higher log-likelihoods to sentences with fewer tokens, regardless of their grammatical acceptability. This is the core of the "token-length bias."
- Existing normalization methods are insufficient: Techniques like MeanLP and PenLP, designed to account for sentence length, did not effectively eliminate the token-length bias. In some cases, they even introduced new complexities or failed to generalize across datasets.
- RoBERTa shows some robustness: While most models were susceptible, RoBERTa exhibited a smaller regression coefficient between log-likelihood and token length, suggesting it was less affected by the bias. The paper leaves a deeper investigation of this as future work.
Key Results
- RQ1 Confirmed: The study definitively showed that token-length bias does affect MPP dataset evaluations. Accuracy decreased when acceptable sentences were longer than unacceptable ones (A>U) and increased when they were shorter (A<U) compared to pairs with equal token lengths (A=U). This effect was observed across multiple datasets and models.
- RQ2 Answered: Normalizing log-likelihood using MeanLP and PenLP was found to be ineffective in mitigating token-length bias. While PenLP showed some promise, it didn't consistently work across all datasets, and the overall conclusion is that these methods do not solve the problem and can even add confusion.
- FairBLIMP Impact: Re-evaluating models on the debiased FairBLIMP dataset showed that the performance gaps between models narrowed, suggesting that the original BLIMP dataset led to potentially incorrect conclusions about relative model performance. For instance, the difference between the best and worst performing models was significantly reduced.
Limitations and Future Work
- Dataset Reconstruction Challenge: The proposed method for generating debiased minimal pairs (ensuring equal token lengths) becomes challenging when comparing a large number of models with different tokenizers and vocabulary sizes. This means MPP datasets might need to be reconstructed for specific sets of models.
- Future Work:
- Detailed RoBERTa Analysis: Further investigation into why RoBERTa appears more robust to token-length bias is suggested.
- Unsupervised Acceptability Scoring: Developing unsupervised methods for calculating acceptability scores that do not rely on dataset reconstruction and can inherently mitigate token-length bias is a key future direction.
- Future MPP Dataset Design: The paper strongly recommends that future MPP datasets strictly control token lengths to be equal between acceptable and unacceptable sentences, potentially using templates and generation tools that can accommodate multiple models and tokenizers.
Summary updated