Calibration of Pre-trained Transformers

Shrey Desai, Greg Durrett

Abstract

Large-scale pre-trained language models like BERT and RoBERTa have revolutionized NLP, but their reliability in terms of confidence estimation is less understood. This study investigates the calibration of these models, finding that they are surprisingly well-calibrated out-of-the-box, particularly in out-of-domain settings. We further demonstrate that techniques like temperature scaling and label smoothing can significantly enhance this reliability, making these models more suitable for high-stakes deployment.

Introduction

In machine learning, calibration refers to the alignment between a model’s predicted probability (confidence) and its actual likelihood of being correct. A well-calibrated model that assigns a 70% probability to a prediction should be correct exactly 70% of the time. While modern neural networks are often criticized for being overconfident, this research explores whether massive pre-trained Transformers share this flaw, especially when encountering "out-of-domain" (OOD) data they weren't specifically trained on.

Methodology

The study evaluates BERT and RoBERTa against two non-pre-trained baselines: DA (Decomposable Attention) and ESIM (Enhanced Sequential Inference Model). Calibration is measured using Expected Calibration Error (ECE), which involves grouping predictions into bins based on confidence and calculating the average difference between confidence and accuracy within those bins.

The authors also test two primary methods to improve calibration:

  1. Temperature Scaling (TS): A post-hoc technique where the model's output scores (logits) are divided by a single learned scalar ($T$) to "soften" the probability distribution.
  2. Label Smoothing (LS): A training-time regularization technique that prevents the model from becoming too certain by slightly distributing the target probability mass across all possible labels.

Evaluation is performed across three task pairs to test both In-Domain (ID) and Out-of-Domain (OOD) performance: SNLI/MNLI (Inference), QQP/TwitterPPDB (Paraphrase), and SWAG/HellaSWAG (Reasoning).

Key Insights

The Complexity Paradox

Usually, as models get deeper and more complex, their calibration worsens. Surprisingly, the authors found the opposite: massive pre-trained Transformers were better calibrated than simpler LSTM-based models, especially when facing out-of-domain data.

Model Comparison: RoBERTa vs. BERT

RoBERTa consistently outperformed BERT in calibration. The authors suggest that more robust pre-training—including larger corpora, more steps, and dynamic masking—leads to more "honest" posterior probabilities.

Calibration of BERT and RoBERTa across SNLI, QQP, and SWAG datasets.

Optimization Strategies

The "best" way to fix a model depends on the target domain. Temperature scaling acts as a "silver bullet" for in-domain data, providing significant improvements with minimal effort. However, label smoothing is far more effective at maintaining calibration when the model encounters a distribution shift (OOD).

Results

Pre-trained models achieved calibration errors up to 3.5× lower than non-pre-trained baselines in out-of-domain settings. On the adversarial HellaSWAG dataset, RoBERTa reduced ECE by a factor of 3.4 compared to the DA model.

Post-processing with temperature scaling further refines these results. For instance, applying TS to BERT on the SNLI task reduced the ECE from 2.54 to 1.14. Label smoothing also showed dramatic results on challenging tasks; on HellaSWAG, RoBERTa trained with label smoothing achieved an ECE of 2.14, compared to 11.93 for the standard model.

Calibration of BERT and RoBERTa after applying Temperature Scaling (+TS).

Conclusion

Pre-trained Transformers are inherently more robust and "honest" about their uncertainty than previous architectures. While this study focuses on classification, it sets a baseline for future research into generative tasks and newer architectures like T5 or GPT-3. Understanding the underlying mechanisms—whether it's the self-attention mechanism or the sheer volume of data—remains a key area for future investigation.

Summary updated

Abstract

Pre-trained Transformers are now ubiquitous in natural language processing, but despite their high end-task performance, little is known empirically about whether they are calibrated. Specifically, do these models' posterior probabilities provide an accurate empirical measure of how likely the model is to be correct on a given example? We focus on BERT and RoBERTa in this work, and analyze their calibration across three tasks: natural language inference, paraphrase detection, and commonsense reasoning. For each task, we consider in-domain as well as challenging out-of-domain settings, where models face more examples they should be uncertain about. We show that: (1) when used out-of-the-box, pre-trained models are calibrated in-domain, and compared to baselines, their calibration error out-of-domain can be as much as 3.5x lower; (2) temperature scaling is effective at further reducing calibration error in-domain, and using label smoothing to deliberately increase empirical uncertainty helps calibrate posteriors out-of-domain.
BibTeX
@inproceedings{desai2020calibration,
  author = {Shrey Desai and Greg Durrett},
  title = {{Calibration of Pre-trained Transformers}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=iMzSXnamZcQ},
}
Figure