Clinician Edits as a Training Signal for Conversational Medical Models

Medical language models are mostly adapted to their domain by continued training on formal writing such as PubMed abstracts. They then struggle with the informal, back-and-forth language of real patient conversations. Real conversational data is scarce, and paying clinicians to write it is slow. A common workaround is to let a large model draft synthetic doctor–patient dialogues and have clinicians correct them. Existing pipelines keep only the corrected final text and throw the corrections away. This proposal argues that the corrections themselves are the most valuable data in the loop. The spans a clinician changes are exactly where the model's medical or conversational knowledge is wrong.

The core idea fits in two sentences. Record each clinician correction as a span-level edit, then continue training a general model with an objective that masks exactly the edited spans and asks the model to reproduce the clinician's version, rather than masking random spans. We claim that, for a fixed budget of clinician hours, this edit-targeted objective gives better conversational history-taking ability than the obvious alternatives, especially on medical specialties absent from the training data. Those alternatives are fine-tuning on the cleaned dialogues, preference training on draft-versus-edit pairs, and fully automated synthesis.

Why this gap exists

The three directions in the file each stop short of this point. On the domain-adaptation side, BioBART continues training BART on 41GB of PubMed abstracts. It reports a gap on informal clinical dialogue such as iCliniq and names patient–doctor interaction corpora as the missing ingredient. That paper motivates our work. MediTOD builds 213 staged history-taking interviews with a schema that links symptoms to clinical attributes such as onset and severity. It reports two findings that constrain us. First, a fully fine-tuned general Llama3 8B beat the medical-specific OpenBioLLM in-domain, with natural-language-understanding F1 of 0.7139. Second, every model dropped to roughly 0.52 F1 on musculoskeletal cases it had not trained on. The first finding contradicts the code-domain evidence that domain pre-training dominates (MonoCoder; A Comparative Study of PEFT Methods for Python Code Generation). We address the contradiction directly. OpenBioLLM's domain text was formal literature, not dialogue. So the question is not whether domain training helps in general, but whether it helps when the domain data is conversational and concentrated on the model's own errors.

On the data-synthesis side, three papers establish that drafting by a model followed by human review is efficient. Bae et al. found that filtering synthetic dialogues was 13.3 times faster than writing them from scratch. Their pipeline cut persona and policy violations from 35.8% to 2.0%. DIALOCONAN scaled 222 expert dialogues to over 3,000 through post-editing, with edit rates (the fraction of a draft a human must change) between 0.175 and 0.448. NaturalCodeBench reports a pipeline more than 4 times faster than manual authoring. These papers supply the method we reuse. All of them, however, use the human only to produce a clean final dataset or a binary reject flag. The closest use of the signal is Bae et al.'s unlikelihood training, which penalizes the model on rejected turns. That works at whole-turn granularity and never learns what the correct replacement should be.

On the human–computer interaction side, Narrative Datasets through the Lenses of NLP and HCI argues that rich context about how annotators work is usually stripped from NLP datasets. The Dou et al. tutorial calls for revision that preserves meaning while changing form, and notes that no pipeline ties interaction traces to base-model training. We treat this direction as a design lens. The editing interface is built so that each edit is captured with its span and a short reason code: factual error, wrong register, missing attribute question, or wrong question order. The interface is part of the method, not just tooling.

Core Claim

Masking and reconstructing clinician-edited spans should yield more useful learning per expert-hour than the same clinician effort spent on any of three alternatives: authoring seeds, filtering whole dialogues, or producing cleaned text for fine-tuning. The reason is that edits put the training loss on tokens where the base model is demonstrably wrong. Random masking spends most of its loss on tokens the model already predicts well. We further predict the advantage will be largest on unseen specialties. Edits for register ("my chest feels tight" versus "dyspnea") and for history-taking structure (asking about onset before severity) should transfer across specialties. Edits that fix specialty facts should not.

Research Program

We first build a synthesis pipeline grounded in MediTOD's attribute schema. Seed dialogues come only from MediTOD's in-domain training split. A general instruction model drafts multi-turn history-taking dialogues across several specialties. Musculoskeletal is deliberately excluded and reserved for out-of-domain testing. Clinicians post-edit the drafts in an interface that logs span edits, reason codes, time per dialogue, and edit rate.

The training objective is simple. For an edited dialogue, let $E$ be the set of token positions a clinician changed in the final text $x$. We train with

$$\mathcal{L} = -\sum_{t \in E} \log p_\theta(x_t \mid x_{\setminus E}) \;-\; \lambda \sum_{t \notin E} \log p_\theta(x_t \mid x_{<t}),$$

Here $p_\theta$ is the model, $x_{\setminus E}$ is the dialogue with edited spans masked, and $\lambda$ weights ordinary next-token prediction on the untouched text. We include that second term so the model does not lose fluency on unedited text. Optionally, an unlikelihood term on the original draft span reuses Bae et al.'s negative signal at span level.

Clinician edits will run out long before a pre-training budget does. A second stage therefore trains a small classifier, using the clinician data, to predict which spans of a draft a clinician would edit. That classifier then marks likely-error spans in a pool of unedited drafts roughly ten times larger, and those spans receive the same masked objective with model-generated corrections. This stage is explicitly a scaling hypothesis. We report it separately from the clean, clinician-only result.

All arms start from Llama3 8B, the model that won in MediTOD. All arms use the same number of training tokens and the same downstream fine-tuning on MediTOD's training split. Arms are budget-matched in clinician hours, measured from interface logs, rather than in number of dialogues.

Baselines

The naive reference point is Llama3 8B fully fine-tuned on MediTOD's training split with no synthetic data. MediTOD reports this as the strongest model: in-domain natural-language-understanding F1 0.7139 and roughly 0.52 on musculoskeletal cases. We rerun it under our seeds and splits so our numbers are directly comparable. Beating it means a statistically reliable gain in-domain and, more importantly, out of domain.

The strongest data-centric competitor is supervised fine-tuning on the same clinician-cleaned dialogues. These are the exact same tokens our method sees, used the standard way. Matching tokens, epochs, and learning-rate search budget isolates the objective rather than the data.

A second strong competitor is direct preference optimization on (draft, edited) pairs. This method trains the model to prefer one response over another using paired examples. It uses the edit signal, but as a preference rather than a reconstruction target, and receives the same tuning budget.

The general-model prompting reference is a large instruction model given the attribute schema and few-shot MediTOD examples. MediTOD's comparison included GPT-4 among its evaluated models.

The medical-model reference is OpenBioLLM fine-tuned identically.

The key ablation applies the same objective with random spans of matched count and length on the same final text. If edit-targeted masking does not beat random masking, the central claim fails.

A further ablation uses fully automated synthesis with no clinicians, filtered by a model-based judge, at matched compute.

Why not just…?

Why not just fine-tune on the cleaned dialogues? This is the real threat. It uses the same clinician effort, and MediTOD shows fine-tuned Llama3 is already the best model. If supervised fine-tuning on cleaned data matches our method in and out of domain, then the edit locations add nothing and we will say so. The random-span ablation separates two effects: the benefit of the data from the benefit of where the loss is concentrated.

Why not just do preference training on draft-versus-edit pairs? Preference training learns that the edit is better. It does not directly teach the replacement text, and it operates on whole responses. If it matches our objective, the honest conclusion is that edits are useful as a preference signal and the reconstruction framing is unnecessary. The head-to-head comparison settles this.

Why not just prompt a strong general model with the schema? EHRSQL (EHRSQL) shows general models collapsing on realistic clinical structure, with a zero-shot GAP model reaching 4.7% execution accuracy. However, that was text-to-SQL, not dialogue. MediTOD's prompted general models were beaten by fine-tuning in-domain. We include the prompting baseline rather than assume it loses.

Why not just skip clinicians and synthesize automatically? Bae et al.'s 35.8% violation rate for raw model dialogues and the sycophancy failures documented by BrokenMath suggest unchecked drafts teach errors. The automated-synthesis ablation at matched compute measures how much of our gain needs humans at all.

Proposed Experiments

The primary evaluation uses MediTOD's understanding, dialogue-policy, and generation tasks, trained on respiratory cases and tested both in-domain and on held-out musculoskeletal cases. The decisive check has two parts. First, the edit-targeted objective must beat both cleaned-data fine-tuning and random-span masking on musculoskeletal F1, above the ~0.52 MediTOD reports. Second, it must do so at equal clinician hours. A per-reason-code analysis tests the transfer hypothesis: retraining with only register and structure edits should keep most of the out-of-domain gain, and retraining with only specialty-fact edits should not.

Secondary evaluations probe the formal–informal gap that BioBART identified. We use iCliniq and HealthCareMagic summarization, for which no reported result was found on Lacuna, and MeQSum, where BioBART-Large reports Rouge-L 53.15 against 51.05 for vanilla BART-Large. These check that gains are not confined to MediTOD's schema. We also run a general-dialogue check to detect whether the general model forgets broad conversational ability during continued training.

On the interaction side, we report clinician time per dialogue, edit rate, and self-reported workload across three budget allocations: authoring seeds, filtering whole dialogues, and span post-editing. This lets the study state which use of an expert-hour buys the most model improvement. That statement is useful even if the training objective itself turns out not to matter.