Treating User Intent Drift as a Common-Mode Failure in Agent Pipelines

When an AI system splits a user's request into subtasks handled by several agents, reliability tools estimate the chance that the whole pipeline fails catastrophically by assuming each agent fails on its own. Reliability engineers know this assumption breaks whenever components share a hidden cause, such as a power supply or a common software bug. We argue that in agent pipelines serving people, the most important shared cause is the user changing their mind. Suppose a user sees a proposed flight and decides they now want a different city. That one shift invalidates the hotel agent, the restaurant agent, and every other downstream agent that relied on the original intent, all at once. The failures are strongly correlated, and the correlation is caused by the system's own intermediate outputs.

This note proposes to measure this effect, model it as a common-mode failure, and plan around it. Concretely, we treat a hidden "intent state" as a shared variable. Given that state, agent failures are roughly independent, and the chance that the state shifts depends on which intermediate results have been shown to the user. The tail-risk planner then works over a mixture of intent states rather than assuming one global independence. It also decides where to place a small number of intent-confirmation checkpoints so that the worst-case pipeline loss stays within budget. The claim is falsifiable. Independence-based tail-risk estimates should be too optimistic under realistic preference drift, a drift-conditioned estimator should close most of that gap at a fraction of the simulation cost, and checkpoint placement guided by that estimator should reduce worst-case failures more than uniform or reactive clarification.

What exists and what is missing

The tail-risk side supplies the method we extend. Risk-Sensitive Agent Compositions represents workflows as directed acyclic graphs of agents. Its dynamic program, BucketedVaR, chooses the composition that minimizes Value-at-Risk (the loss level exceeded only with small probability α) and Conditional Value-at-Risk (the average loss beyond that level). It splits the risk budget across discretized buckets using a union bound, which is valid only when agent losses are independent. The authors tried synthetic Gaussian correlation at ρ = 0.5, reported "reasonable approximations", and gave no error figure. They kept the loss function fixed. We keep their dynamic program and change what it is conditioned on.

The reliability literature motivates the fix. Bayesian software certification work, The Unnecessity of Assuming Statistically Independent Tests in Bayesian Software Reliability Assessments and Demonstrating software reliability using possibly correlated tests, shows that clustered failures make independence-based certificates optimistic. The Load, Capacity, and Availability of Quorum Systems and Reliable Service Allocation in Clouds with Memory and Capacity Constraints show the same thing for voting and replica placement, where redundancy that shares a failure point is not really redundancy. All of these treat the shared cause as external to the system, such as a switch or a power unit. What is new here is a shared cause that the system itself triggers by showing outputs to the user.

The preference-drift side supplies the phenomenon. AcCoRD models user-agent collaboration as a partially observed decision process and identifies "triggered preferences", which only appear once an intermediate option is shown. Models score as low as 0.09 on these, and even Claude-4.5 satisfies every preference less than 30% of the time. AcCoRD assumes tools never fail and uses a single agent, so it never asks how one triggered shift spreads through a multi-agent pipeline. Right Now, Wrong Then: Non-Stationary DPO handles drift by down-weighting old preference data with a discount factor γ. It optimizes expected performance of one model, and choosing γ requires knowing how much the preferences will vary in total. ResiliBench stress-tests workflows against tool failures, but its failures are independent (base success rate 0.8), which the authors flag as a limitation, and the task goal is fixed. The gap is therefore specific: no work models intent drift as a correlation-inducing cause in workflow tail risk, and no work lets a risk planner act on it.

Core Claim

Let $z_t$ be the user's intent state at step $t$, and let $L_i$ be the loss of agent $i$. We assume $L_i$ are conditionally independent given the intent state that holds while agent $i$'s output is still needed. The intent state changes with a hazard $h(z_{t+1} \ne z_t \mid o_{1:t})$, where $o_{1:t}$ are the intermediate outputs shown to the user so far. The pipeline's maximum loss then has the mixture distribution

$$ P(\max_i L_i > \ell) = \sum_{z} P(z \mid \text{surfacing plan}) \, P(\max_i L_i > \ell \mid z), $$

where the inner term is exactly what BucketedVaR already computes under independence, and the outer weights come from the drift hazard. This structure is the beta-factor idea from common-cause reliability, but the shared cause is a person reacting to what the system shows them. It yields two consequences we can test. Independence-based VaR is biased low, and the bias grows with the number of agents downstream of a trigger point. The surfacing plan, meaning which results are shown and when intent is confirmed, becomes a control variable for tail risk rather than a dialogue-design detail.

Research Program

The first component is a benchmark, which we call DriftFlow for now. It uses AcCoRD-Travel because its flight, hotel, and restaurant bookings form a natural graph of dependent subtasks that share one intent, and it averages 11.25 preferences per scenario. We add ResiliBench-style tool failures in two modes. In the first, each call fails independently at a success rate of 0.8. In the second, outages are cluster-correlated through shared simulated services, so we can separate infrastructure correlation from intent correlation. We run each pipeline many times with AcCoRD's user simulator configured for triggered preferences. From these runs we measure pairwise failure correlations between agents, conditioned on whether a drift event occurred. A secondary instance on AcCoRD-Shop, built on WebShop, tests generality.

The second component is the drift-conditioned estimator. We fit the hazard $h$ with a small classifier over features of the surfaced outputs, such as how far a surfaced option is from the stated constraints. We then run BucketedVaR within each of a few discretized intent states and combine the results using the mixture above. The number of intent states is kept small, typically two to four, so that the dynamic program stays polynomial.

The third component is checkpoint planning. A checkpoint asks the user to confirm intent before a group of agents commits. Each checkpoint costs dialogue budget, and AcCoRD shows that over-asking consumes budgets and hurts outcomes. We therefore treat checkpoints as a constrained resource. A second dynamic program places at most $k$ checkpoints to minimize pipeline CVaR, placing them before agents that have many dependents and follow high-hazard surfacing points.

Baselines

The naive reference is the plain agent: ReAct on the same pipeline with no risk planning and no checkpoints. It shows how bad triggered-drift failures are, and AcCoRD's under-30% perfect-outcome rate for Claude-4.5 and 0.09 floor on triggered preferences set the expected range.

The strongest existing method for tail-risk planning is BucketedVaR from Risk-Sensitive Agent Compositions, run under its independence assumption on the same graph, the same per-agent loss samples, and the same risk level α. Its reported accuracy is that estimated quantiles fall "within 1-2 percentage points of the true empirical quantiles" on 16-Rooms and Fetch. There is no reported result on Lacuna for BucketedVaR under preference drift or on any travel or shopping workflow. Beating it means two things. The quantile error against a 10,000-trajectory empirical ground truth must be clearly smaller under triggered drift. And the chosen compositions must have lower realized CVaR, without being worse under the no-drift control, where our estimator should reduce to BucketedVaR.

For interactive handling of drift, the strongest baseline is AcCoRD's UncReAct, which prompts the agent to mark uncertainty and ask for clarification. We give it the same question budget as our checkpoint planner. Beating it means a higher triggered-preference score and perfect-outcome rate at equal or lower question count.

For drift-aware weighting, we include an NS-DPO-style exponential discount on past user signals, following Right Now, Wrong Then, with γ tuned on a validation split using the same budget as our hazard model. This tests whether simply trusting recent signals more is enough.

Two ablations isolate the key ingredient. The first uses a constant hazard, so drift is shared across agents but does not depend on surfaced outputs. This asks whether endogeneity matters. The second places the same number of checkpoints uniformly. This asks whether risk-guided placement matters.

Why not just…?

Why not just prompt a strong model to notice when the user changes their mind and replan? This is the most common answer in practice. The file's evidence goes against it. On ResiliBench, chain-of-thought prompting reached 50.8% success, below structured MDP-optimal workflows at 62.1% (ResiliBench). On AcCoRD, uncertainty-aware prompting still left frontier models below 30% perfect outcomes. Reactive replanning also acts only after downstream agents have already committed, which is exactly the correlated-failure case. The ReAct and UncReAct baselines test this directly.

Why not just simulate the full pipeline with drift many times and read off the empirical tail? This is a real threat. With enough samples, Monte Carlo is unbiased and needs no model. The file notes that tail estimation often needs 10,000 or more trajectories, and each trajectory here involves LLM agents and a simulated user, which makes it expensive. Our experiments state the comparison explicitly. We plot quantile error against the number of trajectories for raw Monte Carlo and for our estimator. If Monte Carlo matches our accuracy at comparable cost, the estimator's contribution shrinks, and what remains is its use in checkpoint planning, where brute-force simulation would have to be rerun for every candidate placement.

Why not just ask the user to confirm before every agent? Confirming everywhere removes drift risk and turns the problem into dialogue cost. AcCoRD reports that over-asking consumes dialogue budgets and degrades outcomes. We therefore sweep the question budget and report the tradeoff curve between CVaR and questions. The uniform-checkpoint ablation shows whether informed placement beats frequency alone.

Why not just discount stale preferences, as NS-DPO does? Discounting reweights evidence for a single model's expected performance. It does not tell a planner which agents will fail together, and its γ assumes a known total amount of drift. It is included as a baseline. If it matches our planner on CVaR, the correlation-structure argument fails.

Proposed Experiments

The decisive first check is diagnostic. On DriftFlow with independent tool failures, we test whether agent failures become correlated when triggered drift is switched on, and whether independence-based BucketedVaR underestimates the empirical 95th and 99th percentile loss by an amount that grows with downstream fan-out. If the correlation is weak, the premise fails and we stop there. The second check is estimator accuracy. We compare quantile error against Monte Carlo ground truth for independence-based and drift-conditioned estimates, using both independent and cluster-correlated tool outages, to show the two correlation sources can be separated. The third check is planning. We compare realized CVaR, triggered-preference score, perfect-outcome rate, and questions asked against all baselines and ablations, with the same agent models, user simulator, and tuning budget across methods. We also check generalization by fitting the hazard on AcCoRD-Travel and evaluating it on AcCoRD-Shop. The key result would be that the output-conditioned hazard beats the constant-hazard ablation. That would show that the system's own intermediate outputs are a measurable, controllable source of correlated failure.