TL;DR
PRISM is an agentic large-language-model (LLM) pipeline designed to identify individuals in online health communities whose genetic test results have been reclassified in clinical databases. By scanning 3.9 million Reddit posts, the system surfaced 123 candidate cases where a variant's pathogenicity had changed since the user's original post, demonstrating a scalable way to bridge the "recontact gap" in clinical genomics.
What Problem Does This Paper Address?
Genetic testing is not a one-time event; our understanding of DNA variants evolves. A variant initially labeled as a "Variant of Uncertain Significance" (VUS) may later be reclassified as "pathogenic" (disease-causing) or "benign" as more data accumulates in databases like ClinVar.
However, a massive "recontact gap" exists. Clinicians often lack the resources or clear mandates to track down former patients when these classifications change. Consequently, many patients rely on outdated information that could impact their medical management. Many of these individuals turn to online communities (like Reddit) to discuss their results, creating a public but untapped signal of unmet medical needs.
Methods
The authors developed PRISM, a three-agent cascade powered by GPT-4o-mini, to process unstructured social media text at a population scale.
- Triage Agent: A high-throughput classifier that screens the entire corpus to identify "reinterpretation-relevant" posts (labeled B2).
- Branch-2 Agent: For B2 posts, this agent extracts genetic identifiers (gene, variant, and test date). The pipeline then performs a deterministic programmatic lookup in ClinVar to compare the classification at the time of the post versus the current classification.
- Outreach Agent: If a discrepancy is found, this agent drafts a peer-toned, disclaimer-bearing message suggesting the user follow up with a genetic counselor.
- Outreach Coach Agent: A meta-agent used during development to automatically refine the Outreach Agent's prompts based on reviewer feedback, ensuring messages are non-alarmist and contextually appropriate.
The system was evaluated using a "dry-run" approach—no messages were actually sent to users. Instead, drafts were subjected to rigorous clinician review.
Datasets
- PRISM Corpus: A newly assembled dataset of 3.9 million public posts from 32 health-related Reddit subreddits, spanning 2008 to 2026. These include specialist genetics forums (e.g.,
r/BRCA), consumer testing sites (e.g.,r/23andme), and general medical advice forums (e.g.,r/AskDocs). - Synthetic Control Set: 6,940 synthetic posts with known "ground truth" answer keys, built backward from real ClinVar variant histories to validate the pipeline's extraction and resolution accuracy.
Key Insights and Ideas
- Concentration vs. Volume: While reinterpretation-relevant discussion is rare in the general population (0.288%), it is highly concentrated in specialist communities. For example, the rate in
r/BRCAis 21.15%, compared to just 0.11% inr/AskDocs. - The Dissociation of Rate and Volume: As shown in Figure 2, specialist communities provide the "densest" signal, but general advice forums, due to their massive size, still contain a significant absolute number of relevant cases.
- Factual Accuracy $\neq$ Clinical Fitness: The LLM was highly accurate at stating the facts of reclassification (27/30 cases). However, clinicians only deemed 50% of the messages "fit to send." The failures were often due to subtle communication issues like "overreach" (e.g., implying a diagnosis) or poor contextualization.
- Cost-Effectiveness: The authors processed the entire 3.9M post corpus for approximately $698 in API fees, suggesting that population-scale screening is financially viable for large health systems or registries.
Key Results
- Detection: PRISM identified 11,253 reinterpretation-relevant posts and surfaced 123 candidate classification differences.
- High-Consequence Changes: Among the 123 candidates, 5 involved a shift from an uncertain submission to a "pathogenic" or "likely pathogenic" classification.
- Clinician Validation: In a held-out set of 30 real-post drafts, a clinician found that 90% (27/30) were factually grounded. However, the "fit to send" rate was lower at 50% (15/30), highlighting the need for a human-in-the-loop for actual outreach.
Limitations and Future Work
- Representativeness: The study is limited to Reddit, which skews toward younger, U.S.-based, and more educated users.
- Human-in-the-Loop Necessity: Because the "fit to send" rate was only 50%, the authors emphasize that PRISM is a "surfacing layer" intended to assist clinicians, not an autonomous outreach tool.
- Ethics and Consent: While the study used public data for a dry run, moving to live outreach raises complex questions regarding user consent and the "duty to recontact" in digital spaces.
- Future Directions: The authors suggest the PRISM template could be adapted for other unmet needs, such as identifying patients who might benefit from new pharmacogenomic guidelines.
Summary updated