Introduction
Probing has emerged as a standard diagnostic tool in Natural Language Processing (NLP) to interpret the internal representations of neural language models. By training a small classifier (a "probe") on frozen model states, researchers aim to determine if specific linguistic features, such as verb tense or part-of-speech, are encoded within the model. However, a significant methodological gap remains: how much data is actually required to make these claims reliably?
Currently, the field suffers from a "Goldilocks" problem. Probing datasets vary wildly in size, often chosen without a principled statistical basis. If a dataset is too small, results lack the power to distinguish between models; if it is too large, resources are wasted on redundant labeling. This paper introduces a formal framework to determine the optimal dataset size for probing, ensuring experiments are both statistically sound and resource-efficient.
The Landscape of Probing Datasets
A meta-analysis of existing probing suites reveals the extent of the inconsistency in dataset selection. Suites like CATS and LAMA utilize a few thousand samples, while others like UD POS and SentEval scale up to 100,000. This arbitrary selection process can lead to underpowered studies or inefficient use of compute and human labeling effort.
Methodology: Probing as a Comparison Problem
The authors argue that probing should be framed as a comparison between two configurations (e.g., comparing BERT to GloVe, or a linear probe to an MLP). They propose a four-step "recipe" for researchers:
- Pilot Study: Conduct a small-scale experiment to measure the performance difference ($|R_1 - R_2|$) between configurations.
- Generalization Bounds: Apply machine learning theory—specifically finite function space bounds—to relate training sample size ($n$) to "excess risk."
- Heuristic Application: Set the generalization bound to half the observed difference ($|R_1 - R_2| / 2$). This ensures the measurement noise is significantly smaller than the signal being detected.
- Power Analysis: Use McNemar’s test and simulations to verify that the recommended size achieves a statistical power of at least 0.8.
Key Insights and Results
A central finding is that probing is distinct from standard classification. The goal is not maximum accuracy but quantifying the "easiness to decode" information. Therefore, probes must have low capacity to avoid learning the task independently of the model's representations.
The framework's validity is demonstrated by comparing empirical results against theoretical predictions. As shown in the analysis of tense prediction (past vs. present), empirical probing results across varying data sizes consistently fall within the margins predicted by the theoretical generalization bounds.
Case Study: BERT vs. GloVe
In a practical application, the authors examined the data required to prove BERT's superiority over GloVe in tense prediction. While 1,024 samples might appear sufficient based on raw accuracy, the power analysis revealed that at least 4,096 samples are necessary to reliably reject the null hypothesis with 80% confidence. This highlights how intuitive estimates of dataset size can often fall short of statistical requirements.
Limitations and Future Directions
While providing a robust foundation, the framework has limitations. It currently assumes data is independent and identically distributed (I.I.D.), which may not hold for all linguistic tasks. Furthermore, the math focuses on comparing two models on a single task rather than cross-task performance. Future work should explore extending these bounds to unsupervised probing methods and incorporating label difficulty into the data requirement calculations.
Conclusion
By moving from ad-hoc dataset selection to a systematic engineering process (Pilot → Estimate → Collect → Verify), this framework allows NLP researchers to conduct more rigorous and efficient probing experiments. It identifies "collapsed" comparisons where no amount of data will yield a significant difference, saving valuable research effort and improving the reproducibility of linguistic insights.
Summary updated