Hospitals pay trained coders to read each discharge summary and turn it into billing codes. The International Classification of Diseases (ICD) has tens of thousands of such codes, and most of them are rare. Automated coders do well on common codes and poorly on rare ones. They also rarely show, in terms a coder can check, which sentences justified each code. This proposal ties the two problems together. The model is allowed to assign a code only through short text spans it has first marked as evidence. Each marked span is then matched to the written description of a code. The explanation therefore is the computation, and the evidence detector does not care which code a span will end up supporting. What counts as evidence ("a finding the physician asserts is present and treated") looks similar across codes. So a detector trained on common codes should transfer to rare codes that have almost no training examples.
In two sentences: first find the spans in the note that a coder would cite, using a detector shared across all codes. Then score each span against code descriptions, so that every predicted code, rare or not, comes with the exact text it was based on. The bet is that this constraint helps most in the long tail. That is where supervised classifiers collapse and where no one has yet measured whether explanations match what human coders cite.
Core Claim
Current evidence-aware coders fall into three groups. The first explains after the fact: The Anatomy of Evidence scores gradient-and-attention attributions of PLM-ICD against coder-annotated spans. The second relies on an external mention finder: Entity Anchored ICD Coding depends on Amazon Comprehend Medical, and Explainable ICD Coding via Entity Linking assumes gold mentions. The third chains several large language model (LLM) calls: KREL. Few-shot coders, including GPsoap, KEPT and the GAN-based generalized zero-shot coders, report only classification scores. None of them reports evidence fidelity on rare codes.
We claim that a compact model can do three things at once if it is trained end to end, with the code decision forced to pass through a learned, code-agnostic span layer. It can match the rare-code accuracy of prompt-based few-shot coders. It can produce evidence that matches human coder spans better than post-hoc attribution does, and the advantage should grow as code frequency falls. And it can run at roughly classifier cost rather than multi-stage LLM cost.
The claim also offers a way to reconcile a tension in the file. Clinical Concept Extraction for Document-Level Coding found that ontology concepts add almost nothing. Most MIMIC codes are expressed by one or two phrases, a finding the file calls the "variation myth." The same paper found that knowing where the clinical phrases are, with only 35% of tokens kept, preserves full performance. We read this as saying that location, not normalization, is the useful intermediate signal. Location should matter most exactly where a model has not memorized a code's one recurrent phrase.
Mechanism
A long-context clinical encoder, initialized as in PLM-ICD with segment pooling, reads the note. A span head assigns each candidate span $s$ an evidence probability $e(s)$. This head knows nothing about any particular code. A matcher, which shares its encoder with the code-description side, scores how well span $s$ supports code $c$ as $m(s,c)\in[0,1]$. The document-level probability of code $c$ is a noisy-OR over spans:
$$p(c \mid d) = 1 - \prod_{s \in S(d)} \big(1 - e(s)\, m(s,c)\big),$$
where $S(d)$ is the set of candidate spans in document $d$. Because code probability depends on the text only through spans, the spans that drive $p(c\mid d)$ are the explanation. This is a concept bottleneck in the sense of Concept Bottleneck Models, with evidence spans playing the role of concepts. Ours differs from that work in two ways: the concepts are open-ended text spans rather than a fixed attribute list, and they need no dense per-concept labels.
Some codes are clinical deductions from labs, procedures and medications rather than verbatim mentions, a problem stated in GPsoap. For these we add a second term: a small attention layer over the top-$k$ evidence spans that scores span sets against the code description. The explanation then becomes a small constellation of spans rather than one phrase.
To scale to the 70,000-code ICD-10 space, we use bi-encoder retrieval of code candidates per span, followed by cross-encoder rescoring. The retrieval stage is a recall risk, the same one that caps KREL, so we measure retrieval recall separately.
Research Program
Evidence labels are scarce. MDACE has 302 admissions with coder spans. We train the span head on these using cross-validation over admissions. We add silver spans on MIMIC-III and MIMIC-IV notes, taken from lexical and embedding overlap between note text and the descriptions of assigned codes. This is the same weak-supervision idea DRGCoder used for saliency, applied here to span boundaries.
Training has three parts. The code loss is binary cross-entropy on $p(c\mid d)$ over all assigned codes. The span loss uses gold and silver spans. A contrastive term on the matcher separates sibling codes, borrowing the dynamic-margin idea from KEPT so that the matcher does not collapse near neighbors in the hierarchy.
The central analysis stratifies every metric by training frequency of the code. The strata are zero, one to five, six to fifty, and more than fifty occurrences, with frequency counted in the MIMIC-IV ICD-10 training split. Within each stratum we report classification scores and evidence scores together.
Baselines
The naive reference point is frequency-thresholded prediction plus a lexical matcher that fires a code whenever its description appears verbatim. This shows how much of the tail is plain string matching. It also directly tests the "variation myth" in the rare regime.
The strongest compact supervised system is PLM-ICD, which reports micro-F1 of 59.8% on MIMIC-III full. Following The Anatomy of Evidence, we pair it with AttInGrad attributions. That paper reports that supervised models place at least one correct evidence token in roughly 80% of MDACE test cases. This pairing is the natural "just explain afterwards" competitor. We retrain it on our exact splits with the same encoder initialization and the same number of tuning trials. To beat it, our model must match its micro-F1 on frequent codes within noise, and must exceed its exact-match and proximate-match evidence rates in the one-to-five and zero-shot strata.
For rare-code accuracy, the strongest reported numbers come from prompt-based few-shot coders. GPsoap reports macro-F1 of 30.2 on MIMIC-III-few. KEPT reports macro-F1 of 30.4 on MIMIC-III-rare50. GPsoap depends on proprietary VA pretraining, so we cannot rerun it and will quote its number. KEPT we rerun with its public recipe on our splits. Reaching roughly 30 macro-F1 on either split while also producing evidence counts as a win. For zero-shot codes, we compare against the entity-anchored ranker of Entity Anchored ICD Coding, reimplemented with an open clinical entity tagger in place of Comprehend Medical. That paper reports NDCG@12 of 61.03 on ICD-10-re-annotated MIMIC-III.
For full-label-space ICD-10 with evidence, the strongest system is KREL, which reports F1 of 0.51 on MDACE across 70,000+ codes. We will not claim to beat it outright. The target is to come within a stated margin of it at a small fraction of its inference cost, with both measured on the same MDACE folds.
Two ablations isolate the key ingredient. The first keeps the evidence loss but removes the bottleneck: codes attend to the whole document, and span supervision is only an auxiliary task. The second keeps the bottleneck but replaces the learned span head with fixed spans from a rule-based extractor, cTAKES, as in Clinical Concept Extraction for Document-Level Coding.
No reported result was found on Lacuna for token-level evidence fidelity on rare or zero-shot codes. Our stratified evidence tables would be the first such numbers.
Why not just…?
Why not just run AttInGrad on a strong classifier? Post-hoc attributions describe a model trained mostly on frequent codes. On rare codes the classifier itself is near chance: MSMN reaches 4.3 macro-F1 on MIMIC-III-few, per GPsoap. Attribution cannot show good evidence for a decision the model gets wrong. Still, the head-code evidence rate reported in The Anatomy of Evidence is already high. If our frequency-stratified comparison shows post-hoc attribution matching us in the tail, the bottleneck is unnecessary and we will report that.
Why not just use an off-the-shelf clinical entity tagger to propose spans, as Entity Anchored ICD Coding does? This is the most serious threat. Wiegreffe et al. showed that cTAKES token locations alone preserve performance. Against that, the same paper documents a specificity mismatch between rule-based extractors and billing codes. MDACE evidence also often includes qualifiers and assertion context that entity boundaries cut off, and the "obese" versus "obesity" near-misses show how strict matching punishes such boundary errors. The fixed-span ablation settles the question directly. If it matches the learned head on rare-code F1 and evidence exact-match, we drop the learned detector and present the contribution as the noisy-OR bottleneck over external spans.
Why not just prompt an LLM, or run KREL? In Explainable ICD Coding via Entity Linking, in-context prompting with ten examples fell well short of fine-tuning. KREL reports that plain prompting reaches only 0.077 recall on combination codes. KREL itself is accurate but uses several LLM stages per note. The comparison we care about is therefore accuracy per unit of inference cost. We report F1 alongside wall-clock time and GPU time per note on MDACE.
Proposed Experiments
The first decisive check is the tail curve. We plot evidence exact-match and proximate-match rates, and code F1, against training frequency on MDACE cross-validation folds. We do this for our model, PLM-ICD with AttInGrad, and both ablations. The hypothesis predicts that the curves meet for frequent codes and separate below about five training examples. If there is no separation, the core claim fails.
The second check is transfer of the code-agnostic detector. We train the span head only on MDACE admissions whose gold codes are frequent. We then measure evidence fidelity on held-out admissions for rare codes. If the detector really learns what evidence looks like rather than which code it belongs to, fidelity should drop only modestly.
The third check is classification on standard few-shot splits: MIMIC-III-few and MIMIC-III-rare50 with ICD-9 descriptions, using the MDACE-trained detector without any ICD-9 evidence labels. This tests whether the method reaches the roughly 30 macro-F1 range reported by GPsoap and KEPT.
The fourth check targets inferred codes. We take the subset of MDACE codes whose gold evidence has low lexical overlap with the code description. On this subset we compare the single-span noisy-OR against the span-set term. This tells us whether the bottleneck can represent deductions assembled from several findings, or only verbatim mentions.
Across all experiments we report retrieval recall for the ICD-10 candidate stage, so that failures of retrieval are not confused with failures of the bottleneck. We also report per-note inference cost against KREL.
References
- The Anatomy of Evidence: An Investigation Into Explainable ICD Coding
- Entity Anchored ICD Coding
- Explainable ICD Coding via Entity Linking
- KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs
- Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt
- Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding
- Generalized Zero-Shot Text Classification for ICD Coding
- Clinical Concept Extraction for Document-Level Coding
- PLM-ICD: Automatic ICD Coding with Pretrained Language Models
- Concept Bottleneck Models
- DRGCoder: Explainable Clinical Coding for the Early Prediction of Diagnostic-Related Groups