The Silent Cost of Conversation History in LLM Reasoning

Ali ŞenolarXiv 2026 · arXiv

TL;DR

Retaining full conversation history in multi-turn LLM interactions causes a massive collapse in reasoning accuracy—dropping from 60% to 10% within just twenty turns—because models "anchor" on their own earlier mistakes. Crucially, this failure is "silent": standard quality metrics like logical coherence and consistency remain high or even improve as the model becomes confidently wrong, making the degradation nearly impossible to detect in real-time without ground-truth labels.

What Problem Does This Paper Address?

As LLMs are integrated into complex decision-support systems (like logistics assistants or procurement trackers), they must maintain state over many turns. Traditionally, developers simply feed the entire transcript back into the model's context.

The author identifies two major gaps in our understanding of this practice:

  1. The Measurement Gap: Most benchmarks only test a few turns. We don't know how reasoning holds up over long horizons (100+ turns) when the model is forced to build upon its own previous reasoning.
  2. The Monitoring Gap: In a deployed system, you don't have the "gold" (correct) answer. Can we use label-free metrics (like how consistent the model is with itself) to detect that a model is starting to "drift" or fail before the final answer is wrong?

The core concern is a reliability trap: if a model makes an error at turn 5, it might generate turn 6 perfectly consistently with that error. To an observer, the model looks fluent and logical, but it is actually hallucinating a false task state.

Methods

The paper employs two distinct studies to isolate the "cost" of history:

  • Study 1 (Short Horizon): Uses "sharded" math problems where information is revealed turn-by-turn. It compares a Dialogue History condition (full transcript) against a Fresh Prefix condition (only the user's information shards, no previous model replies). This isolates the effect of the model seeing its own previous words.
  • Study 2 (Long Horizon): Introduces a 100-turn state-tracking challenge. It compares Dialogue History against a Verified State Summary (where a symbolic solver provides a ground-truth "ledger" of the current state, replacing the messy transcript).
  • Monitoring Framework: The author tested six "RQEval" dimensions (Accuracy, Consistency, Robustness, Logical Coherence, Efficiency, and Stability) alongside five standard statistical drift detectors (like CUSUM and ADWIN) to see if any could serve as an "early warning" system.

Datasets

  • Lost in Conversation (Math subset): Derived from GSM8K, these are arithmetic word problems split into shards.
  • CDR-Stream (New Benchmark): A purpose-built benchmark of 100-turn dialogues across four domains: budget ledgers, warehouse inventory, lab protocols, and project milestones. It features an exact "gold" answer for every turn and a 12-level difficulty ladder that is calibrated specifically for each model tested (GPT-4o-mini, Llama-3 70B, and Gemma 3 12B).

Key Insights and Ideas

  • The Anchoring Effect: Models are highly susceptible to "self-conditioning." Once an LLM commits to a wrong state in the transcript, it treats that error as a factual anchor for all future reasoning.
  • The Stability Paradox: Perhaps the most surprising finding is that as accuracy collapses, stability actually rises. Because the model is anchoring on its own previous context, its repeated samples become more similar to each other. A "stable" model is not necessarily a correct one; it might just be consistently wrong.
  • Coherence is Not Correctness: Logical coherence (the absence of contradictions between steps) fell far less than accuracy. A model can be perfectly "logical" while operating on a completely false premise inherited from turn 10.
  • The Power Problem: Detecting drift in a single 100-turn conversation is statistically difficult. The author shows that a single stream only has enough "power" to detect massive accuracy drops; smaller, insidious degradations require pooling data across many sessions.

Key Results

  • Accuracy Collapse: In Study 2, accuracy under full dialogue history plummeted from ~0.6 to ~0.1 within the first twenty turns.
  • The Mitigation Works: Replacing the transcript with a verified state summary kept accuracy 0.52–0.59 higher and allowed models to recover from difficult turns—a feat the "history" models never achieved.
  • Study 1 Impact: Even in short math tasks (median 6 turns), simply seeing its own previous replies lowered success by 0.21 for GPT-4o-mini and 0.29 for Mistral-7B.
  • Silent Failure: None of the label-free metrics (Consistency, Coherence, Stability) provided a robust early warning. They often moved in the "wrong" direction or lagged behind the actual collapse in accuracy.

Limitations and Future Work

  • Task Narrowness: The study focused on arithmetic and state-tracking with exact answers. It remains to be seen if these "silent costs" apply similarly to creative or open-ended prose generation.
  • Model Selection: The models tested were capable but not the absolute "frontier" (e.g., GPT-4o-mini vs. GPT-4o). Stronger models might have a longer "grace period" before the collapse.
  • Future Directions: The author suggests moving away from raw transcripts as memory. Instead, systems should maintain explicit "state files" (like a JSON ledger) that are updated and verified outside the LLM's primary reasoning loop. They also suggest exploring "counterfactual re-queries"—asking the model the same question without the history to see if the answer changes.

Summary updated

Abstract

Large language models (LLMs) in multi-turn information systems accumulate their own earlier replies and a growing task state. We asked whether this conversation history lowers accuracy, and whether process-level quality metrics warn before it does. Each analysis was fixed before its outcomes were computed. In Study 1, on 48 sharded math tasks, retaining its replies lowered final success by 0.21 for GPT-4o-mini and 0.29 for Mistral-7B (Holm-adjusted p = 0.026 and 0.004); Llama-4 Scout showed the same direction, not significant. Study 2 introduces CDR-Stream, 100-turn state-tracking dialogues with an exact answer per turn and per-model difficulty calibration. With the full history, accuracy fell from about 0.6 to 0.1 within twenty turns at matched difficulty, whereas a solver-verified state summary kept it 0.52-0.59 higher (Holm-adjusted p less than 0.001). Consistency and logical coherence fell far less than accuracy, and stability rose. None of these metrics, a paired readiness measure or calibrated single-stream drift detectors gave robust, model-general early warning, although in Study 2 the repeat-based metrics were complete only for GPT-4o-mini. In the monitors we tested, a single 100-turn stream had power only for very large drops. Systems built on LLMs should keep task state outside the transcript and monitor across sessions.
BibTeX
@misc{senol2026silent,
  author = {Ali Şenol},
  title = {{The Silent Cost of Conversation History in LLM Reasoning}},
  year = {2026},
  eprint = {2611.20071},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20071},
}
Figure