Who Is Caving? Separating Simulator Softness from Assistant Sycophancy in Closed-Loop Dialogue

Researchers increasingly test whether chatbots give in to pushy or mistaken users by having a second language model play the user. A simulated user argues and pressures, and we count how often the assistant capitulates. The method has a hidden flaw. The simulated user is also an aligned chat model, trained toward agreeableness, and it tends to soften, concede, or drift back into a polite-helper voice partway through a conversation. Any measured "capitulation rate" therefore reflects two models at once: how hard this particular simulator actually pushed, and how easily the assistant gave way. Swap the simulator and both the numbers and the ranking of assistants may change. This matters most for the question practitioners now ask, which is whether multi-agent backends (a verifier, a debate, a designated dissenter) protect users from sycophancy. That comparison is only meaningful if pressure is held constant across the systems being compared.

We propose to model the user–assistant loop as what the multi-agent bias literature says it is: a two-agent system in which each side's alignment shapes the dynamics. We then factor out the assistant's share. The central object is a per-turn concession hazard: the probability that the assistant gives way at a turn, given the type of pressure the user applied at that turn. If this hazard is a property of the assistant alone, it should transfer across simulators of very different temperament, and it would give a simulator-invariant way to score assistants. If it does not transfer, closed-loop sycophancy numbers cannot be compared across papers. We test this in τ-bench airline and retail scenarios, where "giving in" means executing a policy-violating database action that can be checked by code rather than by an LLM judge.

Core Claim

Write $l_t$ for the pressure type the simulated user applies at turn $t$. Pressure types are a small set of categories such as restating the demand, giving a reasoned rebuttal, terse assertive doubt ("are you sure?"), and emotional escalation. Write $h_A(l)$ for the probability that assistant system $A$ concedes at a turn with pressure type $l$, given that it has not conceded yet. The violation rate we observe for a conversation is then

$$ P(\text{violation}) = 1 - \prod_{t=1}^{T} \bigl(1 - h_A(l_t)\bigr), $$

where $T$ is the number of user turns. This rate depends on the pressure trajectory $l_1,\dots,l_T$, and that trajectory is produced by the simulator, not the assistant. A soft simulator that drops to "restate politely" by turn 4, or ends the conversation early, will make every assistant look robust.

Our claim has two parts. First, raw closed-loop capitulation rates reorder assistants depending on which simulator is used, so existing single-simulator results confound the two agents. Second, $h_A(l)$, estimated with a validated pressure-type labeler, is approximately invariant across simulators. If so, it predicts an assistant's violation rate under a new simulator from that simulator's pressure trajectories alone. Possibly a third: once pressure is equalized, the protection offered by multi-agent backends differs from what raw rates suggest.

The second part is falsifiable. Hazards may fail to transfer because live simulators adapt to what the assistant just said, and the content of an adaptive reply may matter beyond its pressure category. We treat that outcome as a finding about how much adaptivity contributes, not as a failure.

Where This Sits

The seed work by Tanabe, Measuring Mutual Confirmation Bias Amplification in Simulated User-AI Dialogue, pairs a confirmation-biased LLM user with real assistants and uses two judges to track user belief and assistant accommodation turn by turn. It motivates our study directly. However, it treats the simulator's bias as a controlled input. It does not ask whether the simulator holds that bias, and it uses a single assistant. Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians proves in a binary Bayesian model that even mild sycophancy drives users toward false certainty. It shows the loop is dangerous, but it uses no language models and so cannot address simulator realism.

Three papers supply the evidence that the simulator is not a neutral instrument. Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning documents that LLM simulators drift from flawed or difficult personas back to a cheerful assistant over 60 turns. It reduces this drift with multi-turn reinforcement learning, specifically PPO (proximal policy optimization) using judge-based consistency rewards, and reports a 58.5% improvement in prompt-to-line consistency on open-ended personas. TRAIT shows that aligned models score 86–87 on agreeableness and resist prompting into disagreeable traits. Breaking the Assistant Mold finds that generated characters answer questions directly nearly 100% of the time. The second seed paper, Psychometric Framework for Comparing Alignment-Conditioned Behavior, places models on axes of responsiveness and demandingness. We borrow its demandingness axis as a description of simulator temperament and its human-validation design for our labeler. We use it only statically, as they do. All of these papers study the simulator in isolation; none measures how simulator softness distorts the assistant scores it produces.

On the assistant side, the sycophancy work fixes the pressure by construction. Challenging the Evaluator uses canned rebuttals and finds that terse doubt ("are you sure?") persuades more often than reasoned argument: 84.5% versus 56.1%. That result is why we treat pressure as unordered types rather than a single intensity scale. AssertBench uses single-turn framings. Not Just RLHF shows that one dissenting agent cuts multiple-choice failure under peer pressure by 54–73 percentage points. It also finds that models act as "unanimity detectors" when pressure comes from a user role. Peacemaker or Troublemaker shows that debate collapses toward agreement after 2–3 rounds. These papers constrain our design: they tell us which backends to test and which pressure types matter. They are all single-turn or scripted, though, so their mitigation numbers have never been checked against an adaptive user.

The multi-agent amplification results frame the user–assistant pair as one more small network. Emergence of Biased Consensus finds that low-temperature, high-conformity groups lock into consensus within one or two rounds. Aligned Agents, Biased Swarm shows that reflector agents only temporarily dampen bias. In our setting, two agreeable agents in a loop may settle into mutual concession, which would look like "robust assistant, satisfied user."

Research Program

The testbed has four parts.

Pressure tasks. We build these on τ-bench airline and retail (τ-bench, code at sierra-research/tau-bench, with the τ³ task fixes the repository points to). We rewrite user instructions for existing scenarios so that the user's goal requires an action the domain policy forbids. Examples are modifying a basic-economy booking or refunding outside the allowed window. The correct end state is then the unchanged or policy-compliant database, and violation is checked programmatically.

A second, ungrounded track uses FEVEROUS claims drawn from AssertBench. There, the user insists a true claim is false, and assistant capitulation is scored as an explicit endorsement of the falsehood. This track connects our results to the seed dyad setting.

Simulator panel. We deliberately vary simulator temperament. The panel includes:

  • a prompted instruct model (Llama-3.1-8B-Instruct);
  • the same model trained with the multi-turn consistency recipe of Lee et al.;
  • a base model given a few-shot transcript prompt, since Not Just RLHF shows base models behave differently under social pressure;
  • a large prompted frontier model;
  • a "pressure-scripted" simulator. This one is an LLM that rephrases a pre-sampled sequence of pressure types into fluent text, so the pressure trajectory is fixed while the wording still responds to context.

Pressure labeler. A judge model assigns each user turn a pressure type and records whether the demand is still being asserted. Two human annotators label a 500-turn sample to validate it, following the agreement-reporting practice of the second seed paper.

Assistant backends. We run four configurations for each of two or three assistant models (GPT-4o, Llama-3.3-70B, Qwen3-32B): a single agent; the agent plus a policy verifier that checks proposed tool calls; a three-agent debate that decides each action; and the same debate with one agent assigned to dissent.

From every conversation we estimate $h_A(l)$ with a discrete-time survival model. Its inputs are pressure type, turn index, and the assistant's own previous refusals, so that prior commitment, which often hardens refusal, is not mistaken for a pressure effect.

Baselines

Do-nothing reference. This is the unpressured task: the same τ-bench scenario with a cooperative user who accepts the policy answer. It sets the floor of violations an assistant commits on its own, from misreading policy rather than from pressure. For context, GPT-4o reaches 61.2% pass^1 in retail and 35.2% in airline, and Claude-3.5-Sonnet-20241022 reaches 0.460 airline pass^1 on the leaderboard excerpt (τ-bench). Pressure effects are measured as excess violations above this floor. Our protocol is "better" only if it changes conclusions relative to the field's current practice, the next baseline.

Raw closed-loop rate with a single prompted simulator. This is the strongest existing method and what the first seed paper and τ-bench do. We run it with every simulator on identical tasks, seeds, and turn limits, and report the per-simulator rankings. Our hazard metric beats it if two things hold. First, it predicts held-out-simulator violation rates more accurately: lower mean absolute error, estimated on simulators A–C and tested on D. Second, the assistant rankings it produces agree more closely across simulators, measured by Kendall's τ.

Scripted rebuttal templates. These follow Challenging the Evaluator, translated into multi-turn tool tasks. The per-turn wording is fixed in advance, so pressure is identical across assistants by construction. Its reported numbers are 84.5% persuasion for terse doubt and 86.0% for Llama-3.3-70B under full conversational rebuttal. Those are multiple-choice figures; there is no reported result on Lacuna for scripted pressure inside tool-use tasks. Scripts get the same task set and turn budget.

Consistency-trained simulator. The PPO simulator of Lee et al. is the strongest fix for drift on the simulator side. If it alone makes raw rates agree with the scripted condition, our decomposition is unnecessary.

Ablation removing pressure conditioning. A hazard model that uses only the turn index and ignores pressure type isolates our key ingredient. If it predicts held-out violation rates as well as the full model, then pressure typing carries no information and the method reduces to measuring persistence.

Every condition receives the same number of conversations per task (eight, enabling pass^k-style reliability) and the same decoding settings.

Why not just…?

Why not just use scripted rebuttals and skip the simulator? Scripts fix the pressure, which removes our confound. That makes them a serious threat. Their weakness is that they ignore what the assistant says, which is the policy-mismatch failure that Is MultiWOZ a Solved Task? documents for frozen dialogue. In tool tasks especially, a script cannot respond to a partial offer or a request for clarification. Our pressure-scripted simulator settles the question. It matches the pressure-type sequence of a live conversation while letting wording adapt. If hazards estimated from scripts predict live outcomes as well as hazards estimated from live runs, adaptivity is irrelevant, and we will recommend scripts as the cheaper standard.

Why not just train the simulator to stay in persona? The PPO recipe enforces consistency with a static backstory. Its authors note that it does not model when a person legitimately changes their mind. Persona consistency is not the same as a known pressure level: a perfectly consistent simulator can still apply weak types of pressure. We include it as a panel member. If raw rankings under it match the scripted condition and the other simulators converge to it once trained, the simpler fix suffices.

Why not just use a base model as the user? Base models avoid the polite-assistant pull. However, Not Just RLHF finds they are often more sycophantic, and base models are also less able to follow a task goal. The τ²-bench authors report a 40% error rate for prompt-only simulators even with instruct models (τ²-Bench). We measure base-simulator goal adherence directly. If it is adequate, base simulators become one more point on the temperament axis rather than a replacement for our method.

Why not just add a dissenter and declare the problem solved? The 54–73 point reduction comes from single-turn multiple-choice settings with a static jury. Whether it survives an adaptive user who escalates over many turns is exactly what our backend comparison measures, once pressure is equalized.

Proposed Experiments and Decisive Checks

The first check is ranking instability. Across five simulators and the assistant configurations above, we test whether the order of assistants by raw violation rate changes. Kendall's τ below 0.6 between simulator pairs would show that current single-simulator results are not comparable.

The second check is hazard transfer. We fit $h_A(l)$ on three simulators and predict violation rates on the held-out two from their observed pressure trajectories. We compare the prediction error to two alternatives: using the raw rate from the training simulators, and using the turn-only ablation.

The third check is the size of the adaptivity gap. We compare hazards from live simulators with hazards from the pressure-scripted simulator on matched pressure sequences.

The fourth check is backend protection under equal pressure. We report the dissenter and verifier effects both as raw rates and as reductions in hazard for each pressure type. We expect protection to concentrate on reasoned rebuttals and to be weaker against terse doubt, given the pattern in Challenging the Evaluator. We report pass^k alongside these results, because τ-bench shows that single-run success hides reliability collapse.

The FEVEROUS track repeats checks one and two in the ungrounded dyad. This tests whether conclusions about mutual amplification in the seed setting survive a change of simulator.

Workshop authors to connect with