Abstract
Large Language Models (LLMs) often experience "context drift" during multi-turn interactions, where initial instructions and constraints are gradually lost. Contrary to the prevailing belief that drift accumulates indefinitely, this study demonstrates that drift stabilizes at a predictable equilibrium. By modeling drift as a stochastic recurrence process, the authors show that simple interventions, such as periodic goal reminders, can effectively lower this equilibrium, significantly improving model alignment over long conversations.
Introduction
While LLMs excel at single-turn tasks, real-world applications like virtual assistants or autonomous agents require maintaining a consistent persona and set of goals over many turns. A common failure mode is context drift, where the model's adherence to initial constraints erodes as the conversation lengthens. Traditionally, this was thought to be an unbounded decay due to memory limits or compounding errors. This research challenges that notion, proposing that drift reaches a stable equilibrium that can be actively managed.
Theoretical Framework: Stochastic Recurrence
The authors treat context drift as a dynamical system, measuring the Kullback–Leibler (KL) divergence ($D_t$) between a test model and a high-performing reference model (GPT-4.1) at each turn $t$. They propose a Stochastic Recurrence Model:
$$D_{t+1} = D_t + g_t(D_t) + \eta_t - \delta_t$$
In this model, $g_t(D_t)$ represents systematic bias or memory decay, $\eta_t$ is random noise, and $\delta_t$ represents "restoring forces" or interventions. This framework predicts that drift naturally settles into an equilibrium ($D^*$) where divergent forces are balanced by the model's inherent tendency to return to its baseline behavior.
Experimental Setup
The study utilized two primary benchmarks to validate this theory:
- Synthetic Controllable Drift Task: Models are given strict constraints (e.g., "academic tone") and then subjected to conflicting instructions in subsequent turns to measure their resistance to drift.
- τ-bench: A realistic tool-use benchmark in retail and airline domains, where LLMs act as user simulators with specific goals interacting with task-oriented agents.
Results: The Equilibrium of Drift
A key finding is that LLM divergence does not grow linearly but plateaus. Larger models, such as LLaMA-3.1-70B, reach lower (better) equilibrium points than smaller counterparts like LLaMA-3.1-8B, though the stable pattern remains consistent across scales.
As shown in Figure 3, injecting "goal reminders" mid-conversation acts as a powerful restoring force, shifting the equilibrium point lower and keeping the model significantly more aligned for the duration of the interaction.
Quantitative Impact of Interventions
In the τ-bench retail domain, reminders at turns 4 and 7 reduced KL divergence by up to 11.8% for LLaMA-3.1-70B. Furthermore, qualitative alignment scores from an "LLM-as-Judge" (OpenAI o1) improved by 16-27% across all tested models, as detailed in the table below.
Conclusion
Context drift in LLMs is a manageable phenomenon characterized by stable equilibria rather than infinite decay. By understanding the "restoring forces" within these models, developers can use simple strategies like periodic prompting to maintain high alignment in long-running agentic workflows. Future work will explore more sophisticated interventions like adaptive prompting and Retrieval-Augmented Generation (RAG) to further stabilize these equilibria in complex, multimodal environments.
Summary updated