Planted Persona Shifts as Ground Truth for Explaining Diagnostic Dialogue Models

When a language model diagnoses a patient through conversation, its conclusion depends on three different things: which symptoms the patient reports, when in the conversation they report them, and how the patient is behaving at that moment, for example anxious, confused, or downplaying. Real patients change along that third axis during a single encounter. Current explanation tools mix the three together, and nobody can check whether an explanation is right, because real transcripts come with no record of what actually drove the model's decision. We propose to build patient simulators whose behavioral state changes on a known schedule, such as confusion starting at turn 6 or downplaying starting after a test is ordered. Each planted change can be re-rendered counterfactually with the clinical facts held fixed. That gives exact, known causes against which explanation methods for diagnostic models can be scored.

The opportunity comes from putting two literatures side by side. Character-modeling work has built machinery for representing a persona that evolves over time, so the change itself can be controlled. Clinical interpretability work has shown that standard attribution confuses a measurement's value with its timing (Interpreting Sepsis Prediction through Positional Explanation), but it can only check its explanations against clinician opinion, never against known causes. We use the first to supply what the second lacks.

Core Claim

Our claim is that attributing a conversational diagnostic model's output to dialogue turns requires a three-way split rather than PaX's two-way split. Each patient turn $u_t$ is treated as a triple $(c_t, s_t, p_t)$:

  • $c_t$ is the clinical content stated in the turn.
  • $s_t$ is the patient's behavioral state while saying it.
  • $p_t$ is the turn's position in the dialogue.

A simulator with scheduled persona evolution can intervene on each component separately:

  • swapping positions, as PaX does with its position-swap perturbation;
  • paraphrasing content away;
  • re-rendering the same facts under a different behavioral state (a "state swap").

Together these interventions produce counterfactual effects $\Delta_c, \Delta_s, \Delta_p$ on the doctor model's diagnosis probability. These effects serve as ground truth. We predict that existing explainers will systematically assign state effects to content. Concretely, when a downplaying patient causes a missed myocardial infarction, current methods will blame the chest-pain turn's wording rather than the behavioral shift. We also predict that an explainer which estimates state effects explicitly can be validated on the simulator and then trusted more on real transcripts, where no counterfactual exists.

Research Program

Building the evolving simulator. We start from PatientSim, which crosses 170 MIMIC-derived clinical profiles across five acute conditions with personality, language-proficiency, recall, and confusion axes. Those axes are fixed for the whole encounter. PatientSim also showed that "overly positive" patients sharply reduce doctor-model diagnostic accuracy. We make the axes time-varying with explicit schedules, in the style of the temporal state annotations of Large Language Models Meet Harry Potter, where character attributes update at known story points. We also add a persistence constraint on transitions, borrowed from MECoT. MECoT showed that a Markov chain over emotional states prevents implausible mood swings that free generation produces.

Transitions are tied to plausible clinical triggers. Examples include confusion onset in stroke, downplaying after an invasive test is proposed, and rising distrust after a dismissive doctor turn. AnnaAgent evolves seeker emotions and complaint chains across sessions. Its goal is realism, not causal control, and its internal state is not exposed for intervention. Our simulator differs in that every state change is logged and can be swapped.

Checking that planted states are actually rendered. Safety tuning tends to push simulated patients toward compliance, as PatientSim documents. A planted "distrustful" state may therefore come out bland. Every state-swapped utterance is checked in two ways:

  • Clinical facts must still be preserved, measured with PatientSim's entailment check (natural-language inference, NLI).
  • The state must be recognizably present, measured by a held-out state classifier plus clinician spot ratings.

Counterfactual pairs that fail either check are discarded. We will report the fraction discarded as a measure of how faithfully the simulator renders states.

Defining ground truth. In the offline setting, doctor turns are replayed fixed while patient turns are re-rendered. The ground-truth effect of component $k$ at turn $t$ is

$$\Delta_k(t) = f(D) - \mathbb{E}\left[f(D^{(k,t)})\right],$$

where:

  • $f$ is the doctor model's probability for the true diagnosis;
  • $D$ is the original dialogue;
  • $D^{(k,t)}$ is the dialogue with component $k$ of turn $t$ swapped;
  • the expectation runs over several independent re-renderings, which averages out paraphrase noise.

In a second, interactive setting the doctor responds live. There we measure only end-to-end effects, because the doctor's questions change after each swap.

The proposed explainer. The explainer estimates $\alpha_c, \alpha_s, \alpha_p$ on real transcripts, where a counterfactual simulator is unavailable. It extends PaX's position swap with two further perturbations:

  • a content-ablation swap, in which a turn is replaced by a paraphrase that drops its clinical claims;
  • a state-swap re-rendering, produced by a separately trained style-transfer model conditioned on the target behavioral state, which works on any transcript.

The simulator's role is to tell us whether these cheap estimates agree with the ground-truth $\Delta$s.

Baselines

Naive reference: leave-one-turn-out occlusion. Each turn is deleted and the change in diagnosis probability is recorded. This is the reference any method must beat. It confounds all three components by construction.

Turn-level SHAP and LIME. These treat turns as features. They are the standard tools in clinical attribution. PaX showed that on ICU sequences they mistake position effects for content, for instance inflating admission gender through primacy bias.

PaX applied to dialogue turns. This is the strongest existing method: the two-way what/when decomposition by position swap from Interpreting Sepsis Prediction through Positional Explanation. It reports an insertion-AUC improvement of up to 0.03 on MC-MED over feature-only explainers. There is no reported result on Lacuna for PaX or any decomposed attribution on dialogue or simulated-patient transcripts, so we will establish that number ourselves.

Doctor-model self-explanation. The doctor model is asked which turns drove its diagnosis. DT-GPT and AICare both offer free-text rationales as the explanation interface.

Fairness of comparison. All explainers run on the same transcripts and the same doctor models (several open-weight models, to be fixed in advance). Each perturbation-based method gets the same budget of forward passes per transcript. Any hyperparameters are tuned on a held-out profile split.

What counts as beating PaX. Our explainer must do both of the following on held-out conditions:

  • achieve higher rank correlation with ground-truth $\Delta$s than PaX;
  • reduce the rate at which large state effects ($|\Delta_s|$ above a preset threshold) are assigned to content.

Key ablation. Our explainer is run with the state swap removed, which reduces it to PaX plus content ablation. This isolates whether the state component is what produces any gain.

Why not just…?

Why not just measure accuracy drops under persona variants, as PatientSim does? For simulated data, that measurement is essentially our ground truth, and we use it that way. Its limitation is that it requires the simulator. It cannot say which turn in a real DAIC-WOZ interview, or a real emergency-department transcript, caused a model's error. The whole point is to certify an explainer that works without the simulator. If simulator-validated rankings of explainers fail to carry over to clinician-annotated real transcripts (see the decisive checks below), this objection wins and the contribution shrinks to a benchmark.

Why not just ask the doctor model for its rationale? Free-text rationales are not tied to the computation that produced the answer. AICare found that senior clinicians engaged in adversarial verification of narrative claims and demanded drill-downs into the underlying data. Our setup tests this directly: self-explanations are scored against the same $\Delta$s. We predict they will name content turns and rarely name behavioral shifts. If they turn out to track $\Delta_s$ well, that would be a useful and cheaper result, and we would report it.

Why not just apply PaX unchanged? This is the real threat. PaX's "what" score might already absorb state effects well enough for practical purposes. The ablation above settles it. If adding the state swap does not improve agreement with $\Delta_s$, or does not reduce state-to-content misattribution, the three-way split is unnecessary and we will say so.

Proposed Experiments and Decisive Checks

Dataset. The main dataset is the evolving PatientSim variant: 170 profiles, each with several transition schedules, plus stable-persona controls. Rendering fidelity is reported against PatientSim's own reference points: 98.1% factual entailment for Llama 3.3 70B, and a clinician realism score of 3.89 out of 4.

Transition plausibility is checked with the Emotional Trend Correlation Coefficient (ETCC), MECoT's measure of whether emotional trajectories follow expected trends, on the RAPD benchmark. MECoT reports 0.94, against 0.84 for zero-shot models. We use these figures as a sanity bar for our persistence constraint; we are not trying to compete with them.

Metrics. We report:

  • Spearman correlation between attributions and the $\Delta$s;
  • top-1 localization of the planted transition turn;
  • the state-to-content misattribution rate;
  • insertion and deletion AUC.

Check 1: downplaying cases. On downplaying trajectories where the doctor model misses the diagnosis, do SHAP, PaX, and self-explanation attribute the miss to content while the ground truth places it in $\Delta_s$? This tests the motivating claim directly.

Check 2: positional artifacts. On stable-persona controls, do position effects $\Delta_p$ show primacy or recency artifacts? This asks whether PaX's sequence-model finding reappears in dialogue models.

Check 3: transfer to real transcripts. Clinicians annotate which turns mattered on a small set of DAIC-WOZ interviews and held-out real encounters. We then ask whether the ranking of explainers on the simulator predicts the ranking by agreement with those annotations. If the rankings disagree, the simulator is not a valid proxy, and the negative result is itself reported as a constraint on simulation-based explainer evaluation.