The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail

Samuel R. Bowman

Introduction

For years, the Natural Language Processing (NLP) community has focused on debunking "hype" surrounding artificial intelligence. However, this paper argues that the field has overcorrected into a state of underclaiming—systematically downplaying the actual capabilities of state-of-the-art (SOTA) models. This trend of scholarly pessimism is not merely a matter of scientific modesty; it poses significant risks to researcher credibility and our collective ability to prepare for transformative AI developments.

The Four Archetypes of Underclaiming

Samuel Bowman identifies four primary ways in which researchers inadvertently underclaim model capabilities:

  1. Reliance on Outdated Baselines: Researchers often cite the failures of older models (e.g., from 2017) to imply that current SOTA models share the same fundamental flaws, ignoring the rapid pace of improvement.
  2. The Analysis Lag: There is often a 2-3 year delay between the release of a new model and the publication of deep analysis papers. Consequently, many critiques focus on models like BERT, even after newer architectures like DeBERTa have resolved the specific weaknesses being analyzed.
  3. Misapplication of Philosophical Limits: Arguments regarding the theoretical limits of learning from "form" alone (e.g., the "Octopus" thought experiment) are sometimes incorrectly used to predict that models will fail at practical, empirical tasks they are already performing.
  4. Misinterpretation of Adversarial Datasets: Low scores on datasets specifically designed to be "tricks" (like ANLI) are often reported as absolute measures of general capability, rather than as stress tests for specific edge cases.

Empirical Evidence of Progress

To counter the narrative of stagnation, the paper highlights quantitative leaps in model performance. A key example is the progress made on the Stanford Question Answering Dataset (SQuAD) and its adversarial variants (AS and AOS).

Table 1: Performance improvements on SQuAD and adversarial variants (AS/AOS) from 2017 to 2019.

As shown in the table above, error rates on adversarial sets were significantly reduced in just two years. Between the ReasoNet ensemble (2017) and XLNet-Base (2019), accuracy on the AS metric jumped from 39 to 69, effectively cutting the error rate nearly in half (from 61% to 31%).

The Risks of Scholarly Pessimism

Underclaiming creates a "credibility trap." If researchers insist that a system "cannot understand language" while industry stakeholders observe that same system successfully automating complex linguistic tasks, the research community loses its influence. This disconnect makes it harder to advocate for safety measures or address subtle harms like algorithmic bias.

Furthermore, underclaiming leaves the community unprepared for rapid scaling. If we convince ourselves that certain capabilities are impossible, we risk being blindsided by the arrival of transformative, and potentially catastrophic, AI risks.

The 'Universal Paperclips' thought experiment highlights the shift toward considering existential risks as models become more capable.

Conclusion and The "Bender Rule"

To improve reporting, Bowman proposes a new heuristic: when reporting a model failure, researchers must specify the model type, its relation to current SOTA, and whether the test was an adversarial "trick." Moving forward, the NLP community must balance its skepticism with empirical forecasting to ensure we are ready for the reality of increasingly capable AI systems.

Summary updated

Abstract

Researchers in NLP often frame and discuss research results in ways that serve to deemphasize the field's successes, often in response to the field's widespread hype. Though well-meaning, this has yielded many misleading or false claims about the limits of our best technology. This is a problem, and it may be more serious than it looks: It harms our credibility in ways that can make it harder to mitigate present-day harms, like those involving biased systems for content moderation or resume screening. It also limits our ability to prepare for the potentially enormous impacts of more distant future advances. This paper urges researchers to be careful about these claims and suggests some research directions and communication strategies that will make it easier to avoid or rebut them.
BibTeX
@inproceedings{bowman2022dangers,
  author = {Samuel R. Bowman},
  title = {{The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=zlT1lL4Fl7w},
}
Figure