TL;DR
KT-LLM is a multi-agent framework that bridges the gap between structured longitudinal patient data and the complex, text-based policy rules governing kidney transplantation (KT). By combining a Mamba-based sequence model for survival prediction with a Retrieval-Augmented Generation (RAG) system grounded in authoritative medical corpora, the framework provides auditable, policy-compliant clinical decision support.
What Problem Does This Paper Address?
In the world of kidney transplantation, clinical practice is pulled in two different directions. On one hand, clinicians must manage structured, longitudinal follow-up data (lab results, immunosuppression, etc.). On the other, they must adhere to a massive, ever-evolving set of text-defined rules: the Banff classification for pathology, OPTN policies for waitlist management, and SRTR reporting cadences.
Currently, these "strands" are disconnected. This leads to inconsistent reporting, poor policy compliance, and non-reproducible outcomes across different transplant centers. Most existing AI models focus purely on predictive accuracy but fail to provide the "audit trail" necessary to show why a decision was made or if it aligns with current national policy.
Methods
The authors introduce KT-LLM, an orchestration layer built on the MedLLaVA framework that coordinates three specialized, auditable agents:
- Agent-A (SRTR-MambaSurv): Uses a Mamba (Selective State Space Model) backbone to predict survival and competing risks. Unlike Transformers, which have quadratic complexity, Mamba offers linear-time inference, making it ideal for the long, sparse, and irregular sequences found in multi-year transplant follow-ups.
- Agent-B (OPTN-BlackClust): Employs stable deep embedded clustering (DEC) to identify clinically distinct population subtypes. It specifically focuses on Black recipients to surface outcome heterogeneity and support equitable treatment stratification.
- Agent-C (Policy-Ops): A "rule engine" that converts static text from policy manuals into executable code. It checks for submission deadlines and terminology alignment, returning "pass/warn/fail" outcomes with versioned evidence.
The system uses Retrieval-Augmented Generation (RAG) to ensure all outputs are grounded. It restricts the LLM’s knowledge base to official Banff, OPTN, and SRTR documents, preventing "hallucinations" by forcing the model to cite specific clauses for its conclusions.
Datasets
The study utilized de-identified registry extracts under data use agreements:
- SRTR Standard Analysis Files (SAF): Used for longitudinal follow-up and survival modeling.
- OPTN STAR Files: Used for the 2015–2019 cohort of Black recipients to perform unsupervised subtype discovery.
- Official Policy Corpora: The Banff Central Repository, OPTN/UNOS policy manuals, and SRTR methodological notes served as the authoritative text base for the RAG system.
Key Insights and Ideas
- "Rules Align Data, Governance Clocks Constrain Models": This is the paper's central philosophy. Instead of just training a model on data, the authors synchronize the model's "internal clock" with the semiannual reporting cycles (PSR) of national transplant registries.
- Mamba as a Clinical Backbone: The shift from Transformers to Mamba for medical sequences is significant. It handles the "recipients × years" scale of data efficiently while maintaining high representation quality for sparse observations.
- Auditable Checklists: KT-LLM doesn't just give a natural language answer; it produces a structured checklist (item name, definition, formula, threshold, and source ID). This transforms the LLM from a "black box" into a verifiable auditor.
- Equitable Subgroup Analysis: By identifying stable clusters among Black recipients, the model avoids overgeneralization and supports the recent clinical shift toward race-neutral eGFR (Estimated Glomerular Filtration Rate) calculations.
Key Results
- Predictive Performance: Agent-A achieved a C-index of 0.82 for Death and 0.80 for Graft Loss, surpassing strong deep survival baselines like Dynamic-DeepHit.
- Evidence Attribution: KT-LLM reached an 83.5% Evidence Hit Rate, significantly outperforming general-purpose LLMs (like GPT-4 with RAG) and specialized biomedical models (like Med-PaLM 2) in correctly citing governing policy clauses.
- Policy Compliance: The "Policy-Ops" agent improved the Rule F1 score (a measure of rule-validation accuracy) from 0.73 to 0.89.
- Calibration: The model showed superior predictive calibration (measured by Brier score), meaning its probability estimates closely matched real-world outcomes.
Limitations and Future Work
The authors acknowledge several hurdles:
- Data Granularity: The model's performance degrades when historical data is extremely sparse or when centers use inconsistent documentation styles.
- Human-in-the-loop: The system is designed for decision support, not automation. Definitive clinical diagnoses remain the responsibility of medical teams to avoid "automation overreach."
- Future Directions: The team plans to implement "rolling recalibration"—updating the model automatically as new SRTR data freezes are released—and integrating fairness constraints directly into the training objective to further reduce subgroup disparities.
Summary updated