When a Symptom Doesn't Fit: Second Disease or Misreport?

Automated diagnostic agents that choose which test to order next are judged on two things: getting the diagnosis right and keeping testing costs down. Almost all of them assume two things that real patients routinely violate. They assume one disease explains everything, and they assume patients report their symptoms accurately. This proposal starts from the observation that these two assumptions break together. When a patient reports a symptom that the leading diagnosis cannot explain, there are two competing stories. Either a second, concurrent condition is present, or the report is wrong because the patient forgot, minimized something embarrassing, or agreed with a leading question. An agent that trusts reports and allows multiple diseases will invent comorbidities from misreports. An agent that discounts noisy reports and keeps one disease will explain away real second conditions as noise. Neither error is fixed by ordering more tests at random, because the cheap action that settles one story, such as re-asking the question differently or checking a passive proxy, is usually different from the expensive action that settles the other, such as an objective lab or imaging test.

We propose to build a cost-aware diagnostic agent that treats the patient's reporting channel as a component that can itself fail, with specific, typed failure modes. In this agent, "misreport of symptom $s$" and "concurrent disease $d$" are rival hypotheses in the same search, and the agent spends its budget on the action that best separates them per unit cost. We will build a controlled benchmark in which second diseases and biased reports are injected independently. That benchmark lets us measure, for the first time, how often current agents confuse one for the other, and whether resolving the confusion explicitly beats simply training an end-to-end agent on noisy data.

Core Claim

The claim is narrow and testable. Unexplained evidence in sequential diagnosis has two confounded sources, a concurrent fault and a reporting fault. A policy that represents both explicitly and chooses actions by their power to discriminate between them per unit cost will reach a better accuracy–cost tradeoff on multi-condition, biased-report patients than three alternatives: agents that model only one source, agents that model the noise as generic random flips, and cost-penalized reinforcement-learning agents trained end to end on the same noisy data. We expect the advantage to be concentrated in one place. It should appear on held-out shifts in how often and how patients misreport, because there an implicitly learned robustness has nothing structural to transfer.

Formally, let $F$ be the set of active diseases (capped at small cardinality) and $R$ the set of corrupted self-reports. Each corrupted report carries a type, which is underreporting, duration/recall drift, or acquiescence to confirmatory phrasing. The agent maintains a posterior $p(F, R \mid e)$ over evidence $e$ and picks the next action $a$ with cost $c(a)$ to maximize

$$ \frac{\mathbb{E}_{o \sim p(o \mid a, e)}\left[ H(F \mid e) - H(F \mid e, o) \right]}{c(a)}, $$

where $o$ is the action's outcome and $H$ is entropy over disease sets only. Report-error hypotheses are nuisance variables that the agent marginalizes out rather than tries to resolve for their own sake. This ratio is ordinary cost-weighted information gain. What changes is the hypothesis space it ranges over. Because $R$ is in the model, a re-phrased question that is immune to acquiescence, or a passive measurement, gains value exactly when the posterior is split between "misreport" and "second disease." It gains no value otherwise.

Positioning Against Prior Work

The closest systems motivate the gap without filling it. LA-CDM trains a hypothesis agent and a test-ordering agent with real USD costs in the reward. It restricts diagnosis to four mutually exclusive abdominal conditions and takes histories as given. We keep its cost-in-the-reward design but drop both restrictions. Salus fights premature closure with a controller that keeps testing. It omits costs and scores against one primary diagnosis. SM-DDPO maps the cost–F1 Pareto front for panel ordering on isolated single tasks from lab values alone, with no self-report anywhere. DDXPlus provides a large synthetic patient simulator and shows that training on full differential lists greatly improves differential recall. Every one of its patients has exactly one disease and answers questions perfectly, which is why we use it as the substrate to perturb rather than as a finished benchmark.

Taken together, these papers show the tension the file's Gap 2 describes. Cost penalties push agents to stop early, which is exactly the regime in which a second condition hidden behind a plausible-looking misreport is missed.

Other work constrains the design. The Optimization Paradox study found that composing individually best agents dropped accuracy from 77.4% to 67.7% and produced hallucinated test results in 13.87% of cases. We therefore keep the core agent a single model-based policy with one shared posterior, rather than a pipeline of specialist language agents passing messages. MLCI shows that disease combinations interact non-additively, with mortality distance correlation of 54.80 against 19.98 for the additive Elixhauser index on MIMIC-IV. This warns us that a simple union of two diseases' symptom profiles understates real multimorbidity, so our simulator includes masking rules and treats additive composition as only the easy case.

The self-report literature supplies the bias types rather than a method. We take underreporting of stigmatized behavior from work like the Bergen scale invariance study. We take recall-driven misclassification from RECOVER-Adult and the ultra-processed food metabolomics study. We take demand characteristics from Tag2Risk, which also demonstrates passive proxies as a near-zero-cost but weak (66.4% accuracy) alternative to asking. The Intern Health Study pairs daily self-reported mood with wearable data. It shows that subjective scales interact with the respondent's state, which supports modeling bias as state- and item-dependent rather than as uniform noise.

Research Program

The first component is a simulator, which we call DDXPlus-MR. It starts from DDXPlus patients and composes multi-condition patients by merging the evidence of two or three single-disease cases under a fixed set of rules. Some merges are additive unions. Others are masking merges, where one condition suppresses or alters a symptom of the other, for example a dominant pain localization overriding a secondary one.

Independently of the disease composition, a reporting model corrupts the answers the simulated patient gives. We will tag a subset of evidence items as stigmatized (substance use and sexual-history antecedents in the DDXPlus vocabulary are natural candidates) and underreport them with a set probability. Duration and onset answers get recall drift. Questions phrased confirmatorily get a yes-bias.

The action set has three cost tiers. The cheapest is asking a question. Re-asking in a neutral or indirect form costs slightly more and is immune to acquiescence but not to underreporting. The most expensive tier is an objective measurement attached to a subset of physically verifiable evidence, unbiased but costly. Because disease composition and reporting bias are sampled independently, every case carries a label saying which of the two caused each unexplained finding.

The second component is the agent. It uses DDXPlus's knowledge base to give likelihoods of evidence given diseases, extended with the typed reporting model. It searches only small disease sets and small report-error sets, keeping the hypothesis space tractable by pruning to minimal explanations of the current evidence, in the spirit of conflict-set diagnosis. It ranks actions by the ratio above, with a one-step lookahead. It stops when the expected cost of a wrong final answer falls below the cost of the cheapest useful action. Costs and misdiagnosis penalties follow LA-CDM's structure so that stopping is not tuned separately.

The third component tests transfer to real clinical text on MIMIC-CDM. We will perturb the free-text histories in a controlled way, for example removing or softening alcohol-use statements in pancreatitis cases, or shifting reported pain duration. We then measure whether agents recover through tests. Here the method is implemented by adding explicit report-reliability hypotheses to an LA-CDM-style hypothesis agent's output and training on perturbed histories. MIMIC-CDM stays single-label, so this setting tests only the bias half of the claim.

Baselines

The naive reference points are two. One is an agent that trusts every report and assumes a single disease, which is the DDXPlus AARLC setting. The other is an order-everything policy that acquires all objective measurements upfront. The first defines the error floor and the second the cost ceiling. On clean single-disease DDXPlus, AARLC trained on differentials reports differential diagnosis recall of 97.73%, compared with 21.96% when trained on single labels (DDXPlus). No reported result was found on Lacuna for any agent on multi-condition or biased-report variants of DDXPlus, so all numbers on DDXPlus-MR will be new and produced under one protocol.

The strongest existing competitor is an LA-CDM-style cost-penalized agent trained with group relative policy optimization (LA-CDM). Its target is changed from one class to a disease set, and it is trained on DDXPlus-MR with the same noisy patients our agent sees. It is the real threat because end-to-end reinforcement learning can, in principle, learn to distrust stigmatized answers without being told the bias exists. On clean MIMIC-CDM it reports 81.3% accuracy at roughly $1295 average cost per patient. It will get the same cost table, the same action set, the same number of simulator episodes, and a hyperparameter search budget matched to ours. Beating it means a better accuracy–cost Pareto front on the in-distribution test set and a clearly larger margin under held-out bias shifts. If the margin is only in-distribution and small, we do not count it as a win.

Two ablations isolate the key ingredient. In the first, our agent keeps the same posterior machinery but models report noise as untyped, symmetric random flips. In the second, report error is removed entirely while multiple diseases are still allowed. A third ablation keeps typed noise but forces a single disease. Together these pin down whether gains come from representing both confounded sources or from either one alone. A Salus-style always-continue controller (Salus) with costs added will also be run, to separate the benefit of discriminating actions from the benefit of simply testing longer.

Why not just…?

Why not just train the RL agent on noisy data and let it learn robustness? This is the most serious alternative, and it may well match us in distribution. Our bet is that implicitly learned distrust is tied to the training bias rates and items. The experiment that settles it trains both agents at one underreporting rate and set of stigmatized items, then tests at different rates and on newly stigmatized items. If the end-to-end agent holds up as well, the structured representation is not earning its keep, and we will report that.

Why not just verify everything objectively? Verification removes the confound, but at a price. SM-DDPO found that full observation on sepsis mortality gave F1 0.517 against 0.567 for selective acquisition, at roughly 85% higher cost. In our setting, many discrepancies can be resolved by a neutral re-ask. The order-everything baseline will show directly how much budget that cheap resolution saves.

Why not just add generic input noise and calibrate outputs afterward? Calibration rescales final probabilities. It cannot undo a wrong test sequence chosen early because of a false answer. Generic noise also cannot tell an embarrassing omission from a leading-question yes. The untyped-noise ablation measures this directly. If it performs as well as typed bias, the typing claim fails.

Why not just use a large prompted language model? Monolithic frontier models show premature closure on complex cases. DeepSeek-V3.2 reaches 71.38% against 83.64% for Salus-7B on CompDiag-Bench (Salus), and ReAct reaches 74.9% against 81.3% for LA-CDM on MIMIC-CDM. We will still include a prompted frontier-model baseline on the MIMIC-CDM perturbation set, since it costs little to run.

Proposed Experiments and Decisive Checks

The central measurement is a confusion analysis on DDXPlus-MR. Each unexplained finding is labeled as caused by a second disease, by a misreport, or by both. For every agent we report two error rates. The missed-second-disease rate counts real conditions explained away as noise. The phantom-comorbidity rate counts misreports turned into extra diagnoses. Both are reported alongside exact disease-set match, per-disease F1, and mean cost, traced as Pareto curves over the misdiagnosis penalty.

The prediction is specific. Trust-report multi-disease agents should show high phantom-comorbidity rates. Noise-aware single-disease agents should show high missed-second-disease rates. Our agent should lower both at a fixed budget. It should also do so mainly by choosing cheap re-asks on misreport-driven discrepancies and objective tests on disease-driven ones, which we can check from its action logs.

The decisive checks are three. The first is the held-out bias shift against the end-to-end reinforcement-learning agent. The second is the masking-merge subset, where additive likelihoods are wrong and our model-based agent may lose its edge; we will report that subset honestly. The third is the MIMIC-CDM history perturbation. There we measure accuracy and cost on perturbed versus clean histories, and the fraction of perturbed pancreatitis cases in which the agent orders the disambiguating test, against an LA-CDM-style agent trained on the same perturbed data.

If the structured agent wins only on additive, in-distribution DDXPlus-MR cases, the result is a useful benchmark plus a negative finding about structure. If it wins under shift and on real text, it shows that diagnostic agents should model the patient's reports as something that can fail, the same way they model the patient's organs.