Studies that compare the risk of rare complications after COVID-19 vaccination with the risk after infection mostly count outcomes using billing codes. Those codes miss mild, outpatient, and slowly developing conditions such as postural orthostatic tachycardia syndrome (POTS), a disorder in which the heart races on standing. The obvious fix is to read the clinical notes as well. But the words that describe these conditions show up in notes for very different reasons. After vaccination, a note may say "you may experience chest pain or palpitations; return if so." After infection, a note may say "I have had palpitations since my COVID." A keyword or entity extractor counts both the same way. That error does not affect the two exposure groups equally: vaccine visits are full of counseling language, while infection visits are full of symptom reports.
This proposal makes the outcome definition sensitive to discourse, meaning the role a sentence plays: first-hand report, instruction, or background information. A text mention counts as an adverse-event case only when it is a first-hand experiential report. Each case is then dated by the symptom onset the patient describes, not by the date the note was written. The linguistic cues come from work on detecting personal narratives: first-person singular pronouns and past-tense event verbs mark experience, while second-person and modal or future forms mark advice. The test bed is the self-controlled design already used to compare POTS after infection and after vaccination. The claim is that both fixes change the estimated relative risks in a measurable, differential way. The first removes counseling contamination. The second removes symptoms that began before the exposure but were only recorded at a visit the exposure prompted.
Core Claim
Text-derived outcomes in infection-versus-vaccine cohorts carry two biases that depend on the exposure. Both can be corrected with narrative linguistic markers. The first is discourse contamination: adverse-event vocabulary shows up in instructional and informational text, and it does so disproportionately in vaccine-window notes. The second is timing displacement. Exposure-triggered healthcare contact records symptoms that already existed, and this pushes events into the post-exposure window. That is exactly the detection bias Kwan et al. flag in their POTS study. The prediction is directional and can be falsified. Relative to a discourse-blind text outcome, the discourse-aware and onset-anchored outcome should lower the post-vaccination odds ratio more than the post-infection odds ratio. It should also keep the myocarditis positive control elevated and pull a negative-control outcome toward 1.
How This Differs From Prior Work
The comparative-risk literature supplies the design and the benchmark. Patone et al. used self-controlled case series on linked English national data, counting ICD-coded admissions, and found about 40 extra myocarditis events per million after a positive test versus 1–10 after vaccine doses. Hippisley-Cox et al. and Patone et al. on neurological outcomes did the same for thromboembolic and neurological events. Kwan et al. applied sequence-symmetry analysis to Cedars-Sinai records. Sequence-symmetry analysis asks whether a diagnosis is more often recorded after an exposure than before it, within the same person and a fixed window. All of these studies define outcomes by structured codes and name that choice as a limitation. None uses note text. This proposal keeps their self-controlled designs and changes only how outcomes are defined and dated. Self-controlled designs remove fixed personal confounders, so the propensity-score machinery of Tewari et al. is not the main tool here. Those cohort studies serve instead as a reminder that time-varying bias does not vanish with scale, and detection bias is time-varying.
Clinical-note adverse-event extraction supplies the text machinery but not the comparison. The IBD pilot study used a hierarchical clinical BERT to extract serious adverse events from 928 outpatient notes. It reached 61–68% macro F1 and found that restricting attention to text within two sentences of a hospitalization mention raised macro F1 from 62% to 68%, which suggests local context matters. Belousov et al. showed that separating an adverse drug event from a reason for treatment is hard even with contextual models: ADE F1 rose only from 28.17% to 40.50%. Neither study compares two exposures, and neither asks who is speaking or in what mode.
The narrative-marker literature supplies the linguistic signal. Jijkoun et al. and Dhanwal et al. describe the pronoun and tense footprint of experiential versus instructional discourse. Dirkson et al. show that character 3-grams detect patient narratives with F1 0.815. Antoniak et al. provide span-level story annotations. Falk and Lapesa find that 27% of COVID-subreddit stories concern vaccine and infection experiences, with "disclosure of harm" as a distinct function. That work motivates the idea that harm disclosure has a recognizable form, but it never links to outcomes. Wang et al. aligned vaccine adverse-event reports with tweets but covered only vaccination and relied on keyword dictionaries. The step here is to move these markers into patient-written portal messages and the patient-reported parts of clinician notes, inside a cohort where the counts become risk estimates.
Research Program
The setting is a single health system's electronic record, with structured vaccination and test data plus clinical notes, triage notes, and patient portal messages. This mirrors the Kwan setup, which makes the codes-only result a direct reference point. The file does not identify a site that has agreed to participate, so securing one is the first dependency.
Three linked pieces get built.
The first is a mention-level labeling scheme and gold standard. Every sentence that mentions a target outcome (POTS or dysautonomia symptoms, myocarditis or chest pain with troponin context, and a negative-control outcome chosen for no plausible biological link) gets three labels. The discourse mode is experiential report, instruction or counseling, informational or background, or negated or screening. The speaker is patient voice or clinician observation. The onset expression is any stated time anchor, such as "since my second shot" or "for two years." Clinicians annotate a stratified sample drawn equally from the infection-window and vaccine-window notes. Dirkson et al. traced 36% of classifier errors to annotator inconsistency, so agreement is measured and adjudicated before any model is trained.
The second is the classifier stack. A clinical-domain transformer is trained with the same local-context restriction the IBD pilot found helpful. A character 3-gram linear model follows Dirkson et al. An instruction-prompted LLM receives the same label definitions. Each can be pretrained or calibrated on the public narrative corpora: StorySeeker, the GIST forum set, the SMM4H 2019 personal health mention task, and storyARG.
The third is the epidemiological estimate. The sequence-symmetry and self-controlled case-series analyses are rerun under four nested outcome definitions: ICD codes, discourse-blind text mentions, discourse-filtered mentions, and discourse-filtered mentions dated by onset. For sequence symmetry, the crude ratio is
$$\mathrm{SR} = \frac{N_{\text{after}}}{N_{\text{before}}},$$
where $N_{\text{after}}$ counts people whose first outcome is dated within 90 days after exposure and $N_{\text{before}}$ counts those dated within 90 days before. Onset anchoring changes only the date used for each person, and that is exactly how it can move people from $N_{\text{after}}$ to $N_{\text{before}}$.
Baselines
The naive reference point is Kwan et al.'s codes-only sequence-symmetry analysis, reproduced on the same site, windows, and exposure definitions. It reported a post-vaccination POTS odds ratio of 1.33, a post-infection odds ratio of 2.11, a myocarditis positive-control odds ratio of 2.57, and a 5.35-fold higher POTS risk after infection than after vaccination (Kwan et al.). Reproducing these figures within sampling error is a precondition. Every text-based estimate is judged against them. Beating this baseline means higher sensitivity against chart review, with ascertainment of positive predictive value no worse than the codes.
The strongest text competitor is a discourse-blind clinical extractor: a domain-pretrained transformer with negation handling and the two-sentence local-context restriction, as in the IBD pilot study, which reports 61–68% macro F1 on its own adverse-event tasks. The file reports no result on Lacuna for any extractor on vaccine- or infection-window clinical notes, and none for discourse-mode classification of adverse-event mentions in clinical text. This baseline and the proposed system share the same pretrained encoder, annotation budget, and tuning budget, and differ only in whether the discourse-mode and onset labels are used. Beating it means two things. First, higher positive predictive value in the vaccine arm at matched sensitivity. Second, a between-arm difference in positive predictive value that is significantly smaller than the extractor's.
Two ablations isolate the key ingredients. Removing the onset-anchoring step while keeping the discourse filter shows how much of any odds-ratio shift comes from counseling contamination and how much from timing displacement. Replacing the learned discourse classifier with the bare pronoun-and-tense rule from Jijkoun et al. shows whether the linguistic footprint alone carries the signal. For narrative-detection transfer, the reference numbers are RoBERTa's F1 of 0.86 for documents and 0.78 for spans on StorySeeker (Antoniak et al.), and 0.783 F1 for personal health mentions on SMM4H 2019 (Mahata et al.).
Why not just…?
Why not just keep ICD codes, as the published studies do? Codes cannot be contaminated by counseling text. That is their real advantage, and it is why they stay in the analysis as the anchor. But Kwan et al. and Patone et al. both note that codes undercount ambulatory and unconfirmed presentations. Codes are also dated at the encounter, so they carry the full timing displacement. The four-definition comparison shows directly how much sensitivity codes give up and whether their odds ratios are inflated by prevalent cases diagnosed at visits the exposure prompted.
Why not just use keyword matching with negation detection? If counseling and background mentions turn out to be rare in vaccine-window notes, this simpler approach would be enough, and the discourse filter would add little. That threat gets tested head on. Before any modeling, the proportion of outcome-term mentions in each arm that are instructional or informational is estimated from the gold standard. If that proportion is similar across arms and small, say under 10%, the claim fails and should be reported that way. The evidence in favor of the claim is indirect: keyword approaches confuse reasons with adverse events (Belousov et al.), and they confuse personal reports with commentary in health text (Wang et al., Mahata et al.).
Why not just prompt a general LLM to extract "new patient-reported symptoms and their onset"? This is a real threat. The file does not show that fine-tuned models beat GPT-4 on span detection: Antoniak et al. report fine-tuned RoBERTa at 0.78 span F1 and GPT-4 at about 0.79, although GPT-4 over-identified stories under chain-of-thought prompting. The contribution here is the outcome definition and the demonstration of differential bias, not a particular classifier. The prompted LLM therefore goes into the classifier comparison on equal terms. If it matches the fine-tuned models on the gold standard and produces the same odds-ratio shifts, it becomes the recommended implementation. Over-identification is checked separately in each arm, because an LLM that over-labels experience more often in counseling-heavy notes would bring the bias back.
Why not just handle detection bias the epidemiologist's way, with negative-control outcomes or longer lookback washouts? These methods can detect and partly offset timing displacement. They are included: a negative-control outcome is analyzed in every configuration. But a washout can only exclude people with an earlier code. It cannot recover the symptom history that a patient states in text when no code was ever recorded. The decisive comparison is whether onset anchoring moves the negative-control ratio closer to 1 than a matched-length washout does.
Proposed Experiments
The first experiment is measurement validity. On the adjudicated gold standard, sensitivity and positive predictive value are reported for each outcome definition, separately for each arm. The prediction is that discourse-blind extraction shows a gap in positive predictive value between the vaccine and infection arms, and that discourse filtering closes most of it.
The second experiment is the decisive epidemiological check. The sequence-symmetry and self-controlled case-series analyses are rerun under all four definitions. The claim survives only if three things hold. The post-vaccination POTS ratio falls more than the post-infection ratio when moving from discourse-blind to discourse-filtered-and-anchored outcomes. The myocarditis positive control stays clearly above 1. The negative-control ratio moves toward 1 under onset anchoring. If both arms shift equally, or the positive control collapses, the bias is not exposure-dependent, and the method amounts to better phenotyping rather than bias correction.
The third experiment separates the two mechanisms. The onset-anchoring ablation estimates what fraction of post-exposure first mentions come with a stated onset before exposure, broken down by arm.
The fourth experiment tests transfer. Classifiers are pretrained on the public narrative corpora and then evaluated on portal messages without in-domain labels, which tests whether the forum-derived footprint carries over to patient text inside the health record. Falk and Lapesa's finding that out-of-domain performance drops makes this an open question rather than an assumption.
References
- Apparent risks of postural orthostatic tachycardia syndrome diagnoses after COVID-19 vaccination and SARS-Cov-2 Infection
- Risks of myocarditis, pericarditis, and cardiac arrhythmias associated with COVID-19 vaccination or SARS-CoV-2 infection
- Risk of thrombocytopenia and thromboembolism after covid-19 vaccination and SARS-CoV-2 positive testing: self-controlled case series study
- Neurological complications after first dose of COVID-19 vaccines and SARS-CoV-2 infection
- Long-Term Survival in Men With High Grade Prostate Cancer: A Comparison Between Conservative Treatment, Radiation Therapy and Radical Prostatectomy—A Propensity Scoring Approach
- Algorithmic identification of treatment-emergent adverse events from clinical notes using large language models: a pilot study in inflammatory bowel disease
- GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries
- Mining User Experiences from Online Forums: An Exploration
- An Annotated Dataset of Discourse Modes in Hindi Stories
- Narrative Detection in Online Patient Communities
- Where Do People Tell Stories Online? Story Detection Across Online Communities
- Stories and Personal Experiences in the COVID-19 Discourse
- Multi-instance Domain Adaptation for Vaccine Adverse Event Detection
- MIDAS@SMM4H-2019: Identifying Adverse Drug Reactions and Personal Health Experience Mentions from Twitter