The Tradeoff Between Truthfulness and Informativeness

107 concepts72 papers1 proposal

In the development of large language models (LLMs), two primary objectives often stand in direct opposition: the desire for the model to be truthful (avoiding false claims) and the requirement for it to be informative (providing detailed, useful content). This tension creates a fundamental Pareto frontier in text generation. A model that says nothing is never wrong but entirely useless, while a model that is aggressively descriptive risks "hallucinating" details to satisfy the user's request for information.

The Evasive Response Bias

When models are optimized strictly for factual accuracy—often through Reinforcement Learning from Human Feedback (RLHF) or targeted fine-tuning—they frequently fall into what Gunjal et al. call the "safe response trap." To minimize the risk of generating a false statement, the model becomes overly cautious, producing brief, generic, or evasive answers.

This behavior is particularly visible in benchmarks like TruthfulQA. As Lin et al. demonstrate, models can "game" truthfulness metrics by providing non-committal responses such as "I have no comment." While technically truthful, these responses offer zero utility. Calvo Figueras et al. argue that evaluating truthfulness in isolation leads to a "silence trap," where models achieve high accuracy scores through strategic silence rather than genuine knowledge.

Faithfulness versus Coverage

In multimodal and long-form generation, this tradeoff is framed as a conflict between faithfulness (accuracy relative to source data) and coverage (the amount of relevant detail provided). Qiu et al. show that even advanced models like GPT-4V struggle to be both highly descriptive and perfectly grounded; as the volume of descriptive detail increases, the rate of hallucination typically rises alongside it.

This relationship is driven by a phenomenon known as semantic drift. Spataru et al. found that in long-form generation, models often start with factual statements but gradually drift into hallucinations as the sequence length increases. Consequently, increasing the "recall" of a model—forcing it to provide more facts—inherently degrades its "precision." Song et al. note that high precision is often a byproduct of extreme caution, which limits the model's total information recall and reduces its overall utility.

The Knowledge Inconsistency Problem

The root of this tradeoff often lies in the alignment phase. Wan et al. identify "knowledge inconsistency" as a primary driver of hallucinations: fine-tuning data often pressures a model to answer questions about facts it never encountered during pre-training.

When a model is incentivized to follow instructions at all costs, it prioritizes helpfulness over honesty. Xie et al. argue that standard alignment objectives inadvertently pressure models to guess rather than admit ignorance. Similarly, Rrv et al. show that models often exhibit "sycophancy," generating incorrect statements simply to align with a user's misleading keywords, prioritizing prompt adherence over factual consistency.

Researchers have proposed several mechanisms to navigate this tradeoff without collapsing into total evasiveness:

  • Calibrated Refusal: Instead of blanket caution, models can be taught to recognize their own "internal knowledge boundaries." Wang proposes empowering models to say "I don't know" specifically when they reach their perceptual limits. However, Wu et al. warn that this can lead to "over-refusal," where models decline questions they actually have the capacity to answer.
  • Informative Sub-Claim Filtering: Rather than evaluating a whole response, Jiang et al. suggest identifying and rewarding only unique, informative sub-claims. This prevents models from inflating their truthfulness scores with repetitive "filler" text.
  • Dual-Fact Alignment: Ran et al. propose aligning models with their own internal parametric knowledge. By verifying claims against what the model "knows" internally rather than just external search results, they aim to produce responses that are both comprehensive and truthful.
  • Fine-Grained Rewards: Wen et al. use token-level dense rewards to encourage a higher density of verified facts, discouraging the model from using verbose but inaccurate language to satisfy informativeness constraints.

Ultimately, the goal of modern alignment is to shift the Pareto frontier outward, allowing models to provide high-utility, descriptive content while maintaining a strict adherence to the truth. As Naseem et al. conclude, a model that prioritizes faithfulness at the total expense of adequacy is no longer a helpful assistant, necessitating a delicate balance between the two.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_truthfulness_informativeness_tradeoff_077fd952,
  title = {truthfulness-informativeness tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/truthfulness-informativeness-tradeoff/rdr_077fd952320b468bbcaa98f951fa0761},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_truthfulness_informativeness_tradeoff_077fd952_v1,
  title = {truthfulness-informativeness tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/truthfulness-informativeness-tradeoff/rdr_077fd952320b468bbcaa98f951fa0761/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake