When Silence Is Not Recovery: Correcting Note-Derived Symptom Trajectories for How Hospitals Follow Patients

Many outcomes that matter to patients never reach the structured parts of a medical record. Incontinence after prostate surgery, fatigue, and low mood rarely get billing codes. They show up only as sentences in clinicians' notes. At Stanford, billing codes flagged urinary incontinence after prostatectomy in 4 of 5,349 patients, while text mining of the notes found it in 450 of the 1,485 patients who had post-operative notes (Hernandez‐Boussard et al.). The natural next step is to read these notes over time and rebuild each patient's recovery curve, then compare recovery across hospitals or treatments. That step has a trap. A symptom that stops being mentioned may have gone away, or the patient may have stopped coming in, or the clinician may have stopped writing about it. How often patients are seen, and whether clinicians record that a symptom is absent, are local habits of each hospital. A curve built from notes at one center therefore partly measures that center's follow-up schedule and documentation style.

We propose to treat note-derived symptom trajectories as a measurement problem with two parts: what the note says, and whether the patient was seen and the symptom was asked about at all. The method has three steps. First, a per-visit extractor labels each symptom as reported (with severity), explicitly denied, or not addressed. Second, a model of the visit and documentation process is fitted separately for each site. Third, recovery time is estimated with visits reweighted by how likely they were to occur and to mention the symptom. The claim is that most of the disagreement between hospitals in note-derived recovery curves comes from treating silence as resolution under different follow-up regimes, not from extraction errors. If that is right, correcting for the observation process should bring note-derived curves into agreement with curves built from patient-completed surveys. It should also remove between-site differences that the surveys do not show.

Core Claim and What Is New

The prior work sits on three separate shelves. On the patient-outcome side, Hernandez‐Boussard et al. built a lexicon and negation-rule pipeline that reduces the 90 days after surgery to one present/absent flag at one center, and checks it against chart review rather than patient surveys. Fine-tuning Zero-shot Large Language Models for Patient-reported Outcomes fine-tunes Llama 3.1 8B with low-rank adapters to flag severe depression in individual prostate-cancer survey records, with each survey treated as an isolated snapshot. These two papers motivate the work: the outcome is real and recoverable from text, but nobody has followed it over time or across sites.

On the longitudinal side, DeltaNet and BioViL-T condition on a prior scan or report so the model stops claiming "unchanged" or "resolved" without evidence. ClinicalMamba reads up to 16,000 tokens of a patient's notes, scaling linearly with length. These papers supply the framing that "resolved" is a claim that needs evidence. We carry that framing from images to symptom narratives. We do not build another delta architecture, because a symptom's state at each visit can be read directly from that visit's note.

On the bias side, Stewart et al. found that 87% of emergency-medicine machine-learning studies were at high risk of bias, largely because they rested on single-center retrospective data. A Cross-institutional Evaluation on Breast Cancer Phenotyping NLP Algorithms shows that clinical extractors lose accuracy when moved between hospitals but recover with modest local fine-tuning (micro-F1 0.925 against 0.932 for a model trained entirely at the target site). That result constrains the design. It suggests the per-note extraction gap between sites is fixable with known tools, which is why we place the open problem at the trajectory level instead. FLMD models hidden confounders in structured longitudinal records but assumes the timing of encounters carries no information. CliniCARE-Bench shows that language-model agents auditing patient timelines rarely answer "lack of data" when they should. Our three-way label (reported, denied, not addressed) is that abstention made explicit at the level of a single visit.

What is new is the target and the correction. The target is a recovery curve for a patient-centered outcome, estimated from notes and checked against direct patient surveys at more than one hospital. The correction explicitly models each site's visit and documentation process.

Research Program

The extractor is a domain-tuned language model fine-tuned with low-rank adapters, following the recipe that worked best in the survey study. For each note and each target symptom, it outputs one of three states and, when the symptom is reported, an ordinal severity. The first application is post-prostatectomy urinary incontinence, which allows direct comparison with the Stanford pipeline, with bowel and sexual function added afterward. Distinguishing "not addressed" from "denied" is the part rule-based negation handles badly. We will label it in a chart-review sample at every site.

The observation model treats a symptom observation as the product of two events: the patient has a visit at time $t$, and the note addresses the symptom. We fit a visit-intensity model $\lambda_s(t \mid H_t)$, the rate at which a patient at site $s$ returns given observed history $H_t$ (time since surgery, last extracted state, age, stage, scheduled-visit protocol). We fit a separate documentation-propensity model $\pi_s(t \mid H_t)$, the probability that a visit note addresses the symptom at all. Recovery is then estimated with an interval-censored survival model. Here "recovered" is known only to lie between the last reported visit and the first explicit denial. Each informative visit is weighted by

$$w_{it} = \frac{1}{\hat\lambda_s(t \mid H_{it})\,\hat\pi_s(t \mid H_{it})},$$

where $i$ indexes patients and the hats denote fitted models. This is standard inverse-intensity weighting: it removes the dependence of the curve on when observations happen, provided visit timing depends only on what is recorded in $H_t$. That assumption fails if patients return because of worsening that no note records. We will report sensitivity analyses that vary an unrecorded-return effect and show how far the conclusions move.

Proposed Experiments

The data come in three tiers. The development site is the Stanford STRIDE prostatectomy corpus described by Hernandez‐Boussard et al.. The evaluation adds at least two partner prostatectomy cohorts, each with clinical notes and protocol-timed prostate-cancer quality-of-life questionnaires covering the urinary, bowel, and sexual domains used in the survey study. Survey answers at fixed protocol timepoints serve as the reference trajectory. We restrict to those timepoints because they are less driven by symptoms than ad hoc clinic visits. The file names no existing public corpus with both notes and direct patient surveys at multiple sites, so assembling this paired cohort is itself a deliverable. Following the transfer protocol in the breast cancer cross-institutional study, we report zero-shot transfer and limited-target-data fine-tuning for the extractor.

There are three levels of metrics. At the visit level we report macro-F1 over the three states, with separate recall for "not addressed," on chart-reviewed samples at each site. At the trajectory level we report the absolute difference between the note-derived and survey-derived proportion recovered at 3, 6, and 12 months, and the difference in median recovery time. At the comparison level we report the between-site difference in note-derived recovery curves minus the between-site difference in survey-derived curves. That residual is the part of the apparent site difference that comes from measurement.

Baselines

The naive reference is structured data: billing codes for incontinence, which captured under 0.1% of affected patients at Stanford. Alongside it we run the single 90-day present/absent flag, which shows how much the trajectory framing adds at all.

The rule-based reference is the NCBO Annotator plus extended-negation pipeline of Hernandez‐Boussard et al.. It reported a positive predictive value of 0.73 and sensitivity of 0.84 on 200 chart-reviewed snippets. We re-run it unchanged at every site and add its outputs to the same survival estimator. Beating it at the visit level means higher three-state macro-F1 at each external site. The site-to-site drop for both systems is reported side by side.

The strongest practical competitor is the obvious pipeline: run our same fine-tuned per-note extractor, line the outputs up by date, and call recovery at the last positive mention. It uses identical extraction weights, data, and tuning budget, so any difference comes only from the trajectory step. A second strong competitor reads the whole timeline in one pass and outputs recovery time directly. We run it two ways: with a long-context clinical model in the style of ClinicalMamba, fine-tuned on the same labeled patients, and with a frontier language-model agent given note-search tools, as in CliniCARE-Bench. There is no reported result on Lacuna for note-derived recovery-curve error against patient surveys for any of these systems. The relevant evidence is indirect: CliniCARE-Bench reports that accuracy drops by 4.8 to 14.8 points once shortcut reasoning is penalized, and that agents under-abstain. Beating these baselines means a smaller recovery-curve error than survey-derived curves at every site, and a smaller residual between-site gap.

The key ablation is interval-censored survival on the three-state extraction without the visit and documentation weights. It isolates the observation model. If it matches the full method, the contribution reduces to "label silence correctly and use interval censoring," and we will report it that way.

Why not just…?

Why not just extract per note and line the results up in time? This is the main threat, and we use it as our backbone rather than argue against it. The "last positive mention" rule converts every unaddressed or missed visit into recovery. Recovery then looks fastest at the site with the sparsest follow-up. The direct comparison between this baseline and the full method, which share the same extractor, settles whether the extra step matters.

Why not just use the patient surveys and ignore the notes? Where surveys are dense and complete, they should win, and we use them as the reference. In most retrospective cohorts, though, surveys are sparse and often absent outside dedicated programs. We will report survey coverage per site. If coverage proves high enough that notes add little, that is itself a finding about where note mining is worth the effort.

Why not just interval-censored survival on explicit denials? This is a strong and principled baseline, and we expect it to close much of the gap. It is weak in one specific case. Patients with ongoing symptoms come back more often, and clinicians write explicit denials less often at some sites than others. In that case, when observations happen depends on the outcome, and interval censoring assumes it does not. The weights-off ablation and the resampling test below measure how much this matters in practice.

Why not just give a long-context model or agent the whole record? Models that read long timelines inherit the silence problem unless they are told to separate "denied" from "not addressed." CliniCARE-Bench found that agents commit to yes/no answers when the data are missing. We test this directly by counting how often these systems assert recovery for patients whose last relevant note is a positive mention.

Decisive Checks

The first check is controlled resampling. Take densely followed patients at the development site and thin their visits and their symptom mentions to mimic each partner site's observed visit intensity and documentation rate. The true curve is fixed by construction. The last-positive-mention baseline should shift with the imposed schedule, and the weighted estimator should not. If the weighted estimator also shifts by a similar amount, the observation-model claim is wrong.

The second check uses real data. At each external site, the full method should move note-derived curves closer to survey-derived curves than the weights-off ablation does. Between-site differences in note-derived recovery that the surveys do not reproduce should shrink toward zero. If they persist, either institutional differences enter through channels other than visit timing and documentation, or clinicians' notes are too filtered a stand-in for what patients report. Either outcome would be an important negative result for retrospective outcome research that relies on notes.