Verbosity ≠ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models

Yusen Zhang, Sarkar Snigdha Sarathi Das, Rui Zhang

Introduction

When Large Language Models (LLMs) are uncertain, they often exhibit a behavior termed Verbosity Compensation (VC). Instead of providing a direct answer or admitting ignorance, these models generate excessive, low-density text—essentially "babbling" as a proxy for hesitation. This paper investigates the relationship between output length and accuracy, revealing that verbose responses are significantly less reliable than concise ones.

Comparison between concise responses and two types of Verbosity Compensation: repeating the question and introducing ambiguity.

Defining Verbosity Compensation

The authors define VC as the tendency of an LLM to generate responses that could be significantly compressed without information loss. This behavior acts as a defense mechanism, similar to a human "hedging" during an exam. Common forms of VC include repeating the prompt, adding vague fillers, or over-enumerating. This is problematic not only for accuracy but also for increasing API costs and latency.

Methodology and Datasets

To formalize VC, researchers prompted models to answer "as concisely as possible" (ideally under three words) across five benchmarks: Qasper, LongBench, NarrativeQA, NaturalQuestions 30 (NQ30), and a modified MMLU. If a model exceeded the length limit for a question with a short "gold" answer, it was flagged for VC.

The study introduced several metrics to analyze this behavior:

  • Performance Difference (Δ): Measuring the accuracy gap between concise and verbose responses.
  • Uncertainty Quantification: Using perplexity and Laplacian scores to correlate verbosity with model confidence.
  • CaSel (Cascade Model Selection): A routing algorithm that re-queries a stronger model if the initial output is verbose.

Key Results

VC was pervasive across all 14 models tested, including state-of-the-art closed-source models. For instance, Mistral-7b showed a VC frequency of approximately 74%, while even GPT-4 exceeded 50% on certain tasks.

Frequency of Verbosity Compensation (VC) across various open-source and closed-source models on five benchmarks.

The "accuracy gap" is stark: on the Qasper dataset, verbose responses showed a 27.61% performance drop compared to concise ones. This confirms that for LLMs, talking more usually indicates knowing less. The researchers found a sharp spike in uncertainty (perplexity) right around the third token of a response; if a model cannot reach the point immediately, it likely won't reach it at all.

Mitigation via CaSel

The proposed CaSel algorithm leverages verbosity as a signal of impending failure. By routing verbose outputs from a "weak" model (like Mistral) to a "strong" one (like GPT-4), the researchers reduced VC frequency on Qasper from 63.81% to 16.16%. This approach proved more efficient and cost-effective than uncertainty-based routing or using the most powerful model for every query.

Conclusion

Verbosity is a measurable signature of model uncertainty. While scaling helps, it does not eliminate the tendency to babble. Future work may involve RLHF specifically targeted at conciseness or dynamic token density measurements to trigger model switching mid-generation.

Summary updated

Abstract

Although Large Language Models (LLMs) have demonstrated their strong capabilities in various tasks, recent work has revealed LLMs also exhibit undesirable behaviors, such as hallucination and toxicity, limiting their reliability and broader adoption. In this paper, we discover an understudied type of undesirable behavior of LLMs, which we term **Verbosity Compensation (VC)** — similar to the hesitation behavior of humans under uncertainty — where they respond with excessive words such as repeating questions, introducing ambiguity, or providing excessive enumeration. We present the first work that defines and analyzes Verbosity Compensation, explores its causes, and proposes a simple mitigating approach. Our experiments, conducted on five datasets of knowledge and reasoning-based QA tasks with 14 newly developed LLMs, reveal three conclusions. 1) We reveal a pervasive presence of VC across all models and all datasets. Notably, GPT-4 exhibits a VC frequency of 50.40\%. 2) We reveal the large performance gap between verbose and concise responses, with a notable difference of 27.61\% on the Qasper dataset. We also demonstrate that this difference does not naturally diminish as LLM capability increases. Both 1) and 2) highlight the urgent need to mitigate the frequency of VC behavior and disentangle verbosity with veracity. We propose a simple yet effective cascade algorithm that replaces the verbose responses with the other model-generated responses. The results show that our approach effectively alleviates the VC of the Mistral model from 63.81\% to 16.16\% on the Qasper dataset. 3) We also find that verbose responses exhibit higher uncertainty across all five datasets, suggesting a strong connection between verbosity and model uncertainty. We will release our code and dataset upon acceptance.
BibTeX
@misc{zhang2024verbosity,
  author = {Yusen Zhang and Sarkar Snigdha Sarathi Das and Rui Zhang},
  title = {{Verbosity ≠ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models}},
  year = {2024},
  eprint = {2411.07858},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.07858},
}
Figure