Automatic medical coding means reading a hospital discharge note and assigning the set of diagnosis codes it supports, drawn from tens of thousands of possible codes. A common fix for rare codes is retrieval: look up similar past notes or code descriptions and borrow their labels. Retrieval fails in a quiet and harmful way, though. When the right precedent is missing from the datastore, the retriever does not say "nothing found." It returns the closest near-miss, and the model copies that near-miss's codes. This is hallucination in structured form: a confident, plausible code the note does not support, introduced by a gap in coverage.
We propose a coder that produces codes one at a time, where each new code is conditioned on the note and on the codes already emitted. At every step, a learned gate decides how much to trust labels borrowed from retrieval versus the model's own knowledge of which diagnoses tend to occur together. The gate is trained on deliberately punched holes in the datastore, so it learns what an absent precedent looks like. The central claim is that the codes emitted so far are a better signal of retrieval failure than retrieval similarity scores. A note already coded for type 2 diabetes and chronic kidney disease makes a retrieved "hypertensive heart disease" either strongly expected or suspicious, and no single similarity number carries that information.
Core Claim
Retrieval-augmented multi-label classifiers score retrieved labels independently. Autoregressive multi-label decoders model how labels depend on each other but never consult external memory. We argue that combining the two within each decoding step does two jobs at once. It shrinks the output space at each step to a small candidate set, which makes autoregression tractable over 26,000 codes. It also turns the label history into a detector of coverage failure. The measurable prediction is specific. As a controlled fraction of gold codes is made "uncovered" (all their precedents removed from the datastore), a history-conditioned gate should keep precision and suppress retrieval-induced false positives much better than two alternatives: gating on retrieval similarity, and a hard fallback switch. On fully covered data it should give up little.
Positioning
Three strands motivate the gap. SGM treats multi-label classification as sequence generation. Its LSTM decoder emits labels until an end token and blends label embeddings to soften error cascades. It beat binary relevance on RCV1-V2 (Micro-F1 0.878 vs 0.858). Its vocabulary is fixed, and it has no external memory. Attribute Recognition by Joint Recurrent Learning of Context and Correlation shows that learned co-occurrence can correct weak evidence in degraded inputs. We borrow that idea and apply it to degraded retrieval instead of degraded pixels. On the retrieval side, RAE-XMC builds a key-value memory of training instances and label descriptions and pools the neighbors' labels with a non-parametric softmax, scaling to 1.3M labels. It scores labels independently, cuts off by rank rather than stopping on its own, and has no mechanism for noticing that the memory lacks a precedent.
The closest existing system is the VCMI entry to ImageCLEFmedical Caption 2023. It compares an autoregressive concept detector against nearest-neighbor retrieval and ensembles them with retrieval as a hard fallback when classifiers lack confidence. Our proposal differs in three ways. The switch is soft and made at every decoding step. It is conditioned on label history rather than on classifier confidence. It is trained with supervision about coverage.
Two papers constrain the design. Enhancing medical coding efficiency through domain-specific fine-tuned LLMs reports that standard RAG over ICD-10 injects lexical distractors. It finds that parametric fine-tuning of compact Llama models eliminates non-existent codes but leaves "hierarchy-level errors," meaning related codes at the wrong depth. This rules out a pure retrieve-and-prepend design and tells us to measure category-level as well as exact-code errors. Citation-Enhanced Generation finds accuracy degrading beyond about four retrieved documents because attention disperses. This is why we restrict retrieval to a small candidate set at each step instead of widening context. RAGTruth shows that retrieval-conditioned generators mostly hallucinate by adding unsupported content rather than contradicting the source, and that such additions pile up late in outputs. We test whether the same late-sequence pattern appears in code sequences, which would be predicted if error cascades are a real concern.
Research Program
The model has a domain encoder, a retrieval step at each decoding step, and a small autoregressive decoder.
The encoder follows PLM-ICD: a biomedical pretrained transformer with segment pooling over long notes.
The datastore holds training notes, keyed by encoder embeddings and valued by their code sets, plus code descriptions.
The decoder is a small transformer over code embeddings. At step $t$, it forms a query from the note representation and the emitted history $l_{<t}$. It retrieves the top neighbors and forms a candidate set $C_t$, which is the union of three things: codes carried by those neighbors, the top-$m$ codes from a parametric shortlist head, and an end token. It then predicts the next code with
$$P(l \mid x, l_{<t}) = g_t \, P_{\text{ret}}(l \mid x, l_{<t}) + (1-g_t)\, P_{\text{par}}(l \mid x, l_{<t}), \quad l \in C_t .$$
Here $x$ is the note. $P_{\text{ret}}$ is a softmax over neighbor-weighted candidate labels in the style of RAE-XMC's pooling. $P_{\text{par}}$ is the decoder's own softmax restricted to $C_t$. The gate $g_t \in [0,1]$ is computed from the decoder state, the retrieval scores, and agreement statistics between the history and the neighbors' code sets. Already-emitted codes are masked, as in SGM.
The ingredient that is new is how the gate is trained. For each training note, with probability $p$ we choose a gold code $c$ and remove from the datastore every neighbor carrying $c$. This makes $c$ uncovered for that example. The gate receives a binary cross-entropy target at each step indicating whether the gold next code lies in the retrieved set, alongside the usual next-code likelihood. The decoder therefore has to produce uncovered codes from the parametric path and learn to down-weight neighbor labels that the history makes implausible.
Two decisions stay deliberately simple. For code order, we compare frequency order, which is SGM's heuristic, with the order in which codes are recorded in the MIMIC-IV records, typically principal diagnosis first. For hierarchy, we optionally allow $C_t$ to include parent and sibling codes of retrieved candidates, so that a wrong-depth neighbor can be corrected to the right depth.
We will analyze where along the emitted sequence false positives fall, and whether gate values fall when the history disagrees with the neighbors.
Baselines
The naive reference point is the flat label-attention classifier with independent per-code sigmoids and a tuned threshold, specifically PLM-ICD as reimplemented in the Mimic-IV-ICD benchmark. That benchmark reports best models reaching Macro AUC up to 97.07% and Macro F1 of 5.42% on the full ICD-10 set. We treat 5.42% Macro F1 as the number to beat on covered data. The file contains no Micro-F1 on MIMIC-IV-ICD10. PLM-ICD's 59.8% Micro-F1 is reported on MIMIC-3 and is not directly comparable.
The strongest retrieval competitor is RAE-XMC's retrieve-and-pool scorer, built on the identical datastore and encoder. No reported result was found on Lacuna for RAE-XMC on MIMIC-IV. Its 58.48% P@1 on LF-AmazonTitles-1.3M indicates it is a serious extreme-classification method.
The closest structural competitor is a VCMI-style ensemble. It combines our autoregressive decoder without retrieval and the RAE-XMC scorer, with a hard confidence switch tuned on validation.
A compact fine-tuned LLM coder following Hou's two-phase recipe on Llama-3.2-1B serves as the specialization baseline. We add a prefix tree that restricts outputs to valid codes. Its reported 69.20% exact match and 87.16% category match were measured on a different single-code protocol over 2,211 summaries, so we rerun it on our split rather than quote it.
The ablations that isolate the key ingredient are three:
- Retrieval-gated autoregression trained without the simulated gaps.
- A gate that sees only retrieval similarity scores.
- Retrieval-conditioned decoding without label history in the query.
All systems share the encoder initialization, datastore, data split, and a matched hyperparameter search budget. Compute is reported per system. Beating the field means two things. First, statistically significant gains in precision and Micro-F1 on the uncovered-code condition over both RAE-XMC and the hard-switch ensemble. Second, staying within one Micro-F1 point of the best baseline on fully covered data.
Why not just…?
Why not just gate on retrieval similarity, abstaining when the nearest neighbor is far? This is the cheapest detector and would make our history signal unnecessary if it worked. The file flags continuous similarity scores as a known weak indicator of absence. A close near-miss looks just like a true match. MOTOR documents dense retrieval returning cases with unrelated pathologies, and fixes this with fine-grained re-ranking rather than score thresholds. We do not assume the answer. The similarity-only gate is a direct ablation, and if it matches the history-conditioned gate under simulated gaps, the core claim fails.
Why not just ensemble a classifier with retrieval using a confidence switch, as VCMI did? This is the most serious threat, because it already reached 0.4998 F1 on ImageCLEFmedical 2023 concept detection. Its weakness is that the switch is made once per input, not per code. A single uncovered comorbidity then forces a choice between discarding all retrieved help and accepting the near-miss labels. We run it on both benchmarks under identical coverage manipulations. If it matches the per-step gate, we will report that the benefit comes from the fallback and not from label-history conditioning.
Why not just fine-tune a compact LLM with valid-code constrained decoding? Hou's results show parametric specialization removes non-existent codes, and a prefix tree guarantees validity. However, the same paper reports persistent hierarchy-level errors, and nothing in parametric memory adapts to codes whose precedents change after training. We measure exact versus category-level errors and performance on held-out codes. If the constrained LLM wins on coverage-gap precision at equal compute, the retrieval gate is not worth its complexity.
Why not just retrieve more neighbors to close coverage gaps? Widening $k$ is exactly what Citation-Enhanced Generation found degrades accuracy past about four documents. More neighbors also cannot supply a precedent that is truly absent. We sweep neighbor count for RAE-XMC and our model to confirm this.
Proposed Experiments
The main benchmark is MIMIC-IV-ICD10 from the Mimic-IV-ICD benchmark: 122,000 notes and 26,096 codes. We report Micro-F1, Macro F1, P@k, exact and three-character category match, and the fraction of predicted sequences that stop at the correct length.
The decisive experiment is a coverage sweep. At test time we make 0%, 10%, 25%, and 50% of each note's gold codes uncovered, and separately hold out an entire set of codes from the datastore. For each condition we measure retrieval-induced false positives: predicted codes that are not gold but were carried by retrieved neighbors, counted on notes with an uncovered gold code. We also measure recall of the uncovered codes. The hypothesis is confirmed if the history-conditioned gate's false-positive rate grows much more slowly with the uncovered fraction than RAE-XMC's and the similarity gate's, while full-coverage Micro-F1 stays within one point.
A secondary check on ImageCLEFmedical Caption 2023 concept detection, with 2,125 UMLS concepts, tests transfer to images against the VCMI ensemble's reported 0.4998 F1.
Two diagnostic analyses complete the study. The first tests whether gate values correlate with actual coverage. The second tests whether false positives cluster late in sequences, as RAGTruth reports for prose.
A negative outcome would also be informative. If label history does not improve on similarity gating, then the knowledge-base coverage bottleneck in structured prediction cannot be detected from the output side, and effort belongs in retrieval re-ranking of the MOTOR kind instead.
References
- SGM: Sequence Generation Model for Multi-label Classification
- Attribute Recognition by Joint Recurrent Learning of Context and Correlation
- Retrieval-augmented Encoders for Extreme Multi-label Text Classification
- Detecting Concepts and Generating Captions from Medical Images: Contributions of the VCMI Team to ImageCLEFmedical Caption 2023
- Enhancing medical coding efficiency through domain-specific fine-tuned large language models
- Citation-Enhanced Generation for LLM-based Chatbots
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
- PLM-ICD: Automatic ICD Coding with Pretrained Language Models
- Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classification
- MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answering