In conversational AI, there is a persistent gap between how humans and machines handle long interactions. While humans generally become more attuned to a partner's needs as a conversation progresses, AI models often exhibit the opposite behavior: a phenomenon known as performance decay or turn fatigue.
As the number of interaction turns increases, the accuracy of dialogue state tracking, retrieval, and reasoning often drops precipitously. Nikos Voskarides and colleagues at the University of Amsterdam found that conversational search systems can see significant declines in retrieval accuracy as interaction history grows, suggesting that the "noise" of accumulated context eventually outweighs the "signal" of the user's current intent.
The Signal-to-Noise Problem
The primary reason multi-turn dialogue is difficult is not just a lack of memory, but an inability to manage it. In retrieval-based systems, Chao Xiong and others have noted that the accumulation of vocabulary across many turns creates significant noise. When a model relies on word-level matching, it becomes increasingly difficult to distinguish between the relevant current query and distracting keywords from five turns ago.
This noise leads to several specific failure modes:
- Contextual Drift: Models lose track of established goals. Peng Wu 0037 observes that in multi-domain tasks (like booking a hotel then a restaurant), models often fail to carry over "hard" constraints while dropping obsolete ones.
- State Inertia: As dialogue history grows, models struggle to prioritize recent corrections over earlier, now-invalid statements. The paper Oh My Mistake! highlights that models find it difficult to "overwrite" established context when a user changes their mind late in an interaction.
- Positional Bias: Even large language models (LLMs) suffer from what Ayesha Qamar describes as a failure to link related utterances if they are separated by more than a few turns, despite having large context windows.
Fragile Reasoning Chains
In task-oriented and reasoning-heavy dialogues, the problem is structural. Zhiyu Chen and the creators of ConvFinQA demonstrate that multi-turn reasoning is "fragile." If a question at turn 5 depends on a numerical result calculated at turn 2, any slight error in the intervening turns propagates. This creates a "goldfish memory" effect where the model might follow the immediate instruction but contradicts the broader logical sequence established earlier.
Similarly, in conversational Text-to-SQL, Hanchu Zhang found that accuracy can drop by nearly half between the first and fourth turns. This occurs because the complexity of maintaining the "state" of a database query grows exponentially as the user adds, removes, or modifies filters across multiple turns.
Structural Solutions to Context Decay
To combat this decay, researchers are moving away from treating dialogue history as a flat sequence of text.
1. Dynamic Graphs: Mirella Lapata and Parag Jain proposed using Dynamic Context Graphs. Instead of a list of sentences, the history is represented as an evolving subgraph of a Knowledge Graph. This allows the model to "hop" directly to relevant entities regardless of how many turns ago they were mentioned.
2. Auxiliary Tracking Tasks: Rather than just predicting the next response, models are being trained on "meta" tasks. Hanchu Zhang introduced tasks that predict how SQL logic "switches" between turns, while Deyi Xiong used utterance tagging to help models explicitly distinguish between speakers and functional roles.
3. Recursive Summarization: To handle truly long-form dialogue, Liang Ding suggests that models should recursively summarize their own history. By compressing past turns into high-level themes, the model preserves the "gist" of the conversation without filling its context window with the noise of verbatim transcripts.
4. Test-Time Adaptation: Recent work like AdaRewriter and Test-Time Reranking by Hung-Chun Hsu suggests that the model should spend extra computation at inference time to evaluate how well a candidate response aligns with the entire history, effectively "double-checking" its work to ensure long-term consistency.