On the Evaluation of Neural Selective Prediction Methods for Natural Language Processing

Zhengyao Gu, Mark HopkinsACL 2023 · OpenReview

Abstract

We provide a survey and empirical comparison of the state-of-the-art in neural selective classification for NLP tasks. We also provide a methodological blueprint, including a novel metric called refinement that provides a calibrated evaluation of confidence functions for selective prediction. Finally, we supply documented, open-source code to support the future development of selective prediction techniques.

Introduction: Knowing When to Say "I Don't Know"

Deep learning models are notoriously overconfident, often asserting incorrect answers with high certainty. In high-stakes Natural Language Processing (NLP) applications, it is crucial for a model to "know what it doesn't know." Selective prediction (or selective classification) addresses this by allowing a model to abstain from making a prediction when its uncertainty exceeds a certain threshold.

A neural network processing a sentiment analysis task. The model outputs probabilities for different classes; the maximum softmax probability (MaxProb) is a common baseline for selective prediction.

The primary challenge in this field is evaluation. Traditionally, researchers have relied on the Area Under the Precision-Recall curve (AUPR) to assess how well a model's confidence scores distinguish between correct and incorrect predictions. However, AUPR is inherently "coupled" with the model's underlying accuracy. A high AUPR might simply reflect an easy task rather than a superior ability to self-assess, making it difficult to compare models with different baseline performances.

The Interpretability Gap and the Refinement Metric

To address the lack of interpretability in traditional metrics, we propose Refinement, a calibrated metric based on the Kendall-tau distance. Refinement counts "ranking errors"—instances where a model is more confident in an incorrect prediction than a correct one.

Unlike AUPR, Refinement is independent of task performance. It provides a fixed baseline where a score of 1.0 represents a perfect ranking of predictions, 0.5 represents random guessing, and 0.0 represents a worst-case scenario. This calibration allows researchers to compare the "honesty" of different models even when their baseline accuracies differ significantly.

Precision-Recall curves illustrating the "interpretability gap." Models with similar AUPR scores (e.g., 0.863 vs 0.865) can exhibit vastly different behaviors. Furthermore, the "worst case scenario" AUPR baseline varies significantly with model accuracy (0.386 vs 0.736), complicating direct comparisons.

Benchmarking Selective Prediction in NLP

We conducted an extensive empirical study using the GLUE benchmark, focusing on tasks such as sentiment analysis (SST-2), natural language inference (MNLI, QNLI, RTE), and linguistic acceptability (CoLA). We evaluated several state-of-the-art techniques categorized into post-hoc and training-time methods:

Post-hoc Methods (No Retraining Required)

  • MaxProb: The simplest baseline, using the maximum softmax probability as the confidence score.
  • Monte Carlo Dropout (MCDM/MCDV): Estimates uncertainty by performing multiple forward passes with dropout enabled to observe output fluctuation.
  • TrustScore: Measures the distance between a model's internal vector representations and the training data distribution.

Training-time Methods

  • Error Regularization (EREG): Incorporates a penalty for ranking errors directly into the loss function during training.
  • Deep Abstaining Classifiers (DAC): Introduces a dedicated "abstain" output class to the neural network architecture.

Key Insights and Results

Our results highlight several critical findings for the NLP community:

  • MaxProb is a "Strong Ghost": Despite its simplicity, MaxProb remains a remarkably high bar to clear. Many complex methods struggle to significantly outperform it across standard benchmarks.
  • MCD Superiority: Monte Carlo Dropout (MCDM) emerged as the most consistent winner, outperforming MaxProb in 62% to 85% of trials. However, its computational cost—requiring multiple forward passes—remains a hurdle for production environments.
  • TrustScore Limitations: While popular in computer vision, TrustScore was less effective for NLP. The high-dimensional nature of BERT embeddings likely makes nearest-neighbor distances a noisy signal for uncertainty.
  • Batch Size Sensitivity: Training-time methods like EREG are highly sensitive to minibatch size. Because EREG penalizes pairwise ranking errors within a batch, small batches do not provide enough "pairs" for the model to learn effectively.

Conclusion

This work provides a rigorous framework for evaluating selective prediction in NLP. By introducing the Refinement metric and the open-source spred package, we aim to standardize how researchers measure model reliability. While current methods show promise, future work must address the computational overhead of techniques like MC Dropout and extend these evaluations to generative tasks and out-of-distribution scenarios.

Summary updated

Abstract

We provide a survey and empirical comparison of the state-of-the-art in neural selective classification for NLP tasks. We also provide a methodological blueprint, including a novel metric called refinement that provides a calibrated evaluation of confidence functions for selective prediction. Finally, we supply documented, open-source code to support the future development of selective prediction techniques.
BibTeX
@inproceedings{gu2023evaluation,
  author = {Zhengyao Gu and Mark Hopkins},
  title = {{On the Evaluation of Neural Selective Prediction Methods for Natural Language Processing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=qaTvTnTGNW},
}
Figure