Modeling Diagnostic Label Correlation for Automatic ICD Coding

Shang-Chi Tsai, Chao-Wei Huang, Yun-Nung Chen

Abstract: Shang-Chi Tsai, Chao-Wei Huang, Yun-Nung Chen. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

2021.

Introduction

Assigning International Classification of Diseases (ICD) codes to clinical notes is a cornerstone of modern healthcare administration, yet it remains a manual, labor-intensive task. While AI researchers have treated this as a "multi-label classification" problem, most existing models make a fatal assumption: they treat every code as independent. In reality, medical codes are deeply intertwined, following hierarchies and exhibiting strong co-occurrence patterns. A patient is unlikely to be diagnosed with two mutually exclusive conditions simultaneously, yet standard models often predict "conflicting" codes because they evaluate each label in isolation. This paper addresses the label correlation gap—the inability of current models to understand how medical codes relate to one another.

Methodology

The researchers propose a two-stage reranking framework that can be wrapped around almost any existing medical coding model to capture these hidden dependencies.

Overview of the two-stage reranking framework, illustrating the transition from candidate generation to label set reranking.
  1. Candidate Generation: A "base predictor" (such as the popular CAML or LAAT models) processes clinical notes to generate the top-$k$ most probable sets of ICD codes. Because there are $2^N$ possible combinations, the system uses dynamic programming to efficiently identify the most likely candidates.
  2. Label Set Reranking: A second model "rescores" these candidates by looking at the labels themselves, independent of the original text. The authors tested two architectures for this: MADE (Masked Autoencoder for Distribution Estimation), an autoregressive model, and Mask-SA (Masked Self-Attention), a Transformer-based approach that uses a "cloze-style" task to predict masked labels based on surrounding context.

The final score is a weighted combination of the original base model's confidence and the reranker's correlation score, adjusted by a "length penalty" to ensure the model doesn't unfairly favor smaller sets of codes.

Results and Key Insights

The framework was evaluated on the gold-standard MIMIC-2 and MIMIC-3 clinical NLP benchmarks. The results demonstrated that accounting for label correlation provides a significant "sanity check" for automated systems.

Performance comparison showing Macro and Micro F1 improvements across different base models (CAML, MultiResCNN, LAAT) on the MIMIC-3 dataset.
  • Model Agnosticism: The framework is a "plug-and-play" module that improved the performance of three very different state-of-the-art base models, proving that label correlation is a universal weakness in current architectures.
  • Consistent Gains: The framework improved Macro F1 scores across all datasets. In the challenging "all-code" settings, relative improvements in Macro F1 ranged from 1% to 16%.
  • Helping the "Long Tail": One of the biggest hurdles in ICD coding is imbalanced data, where rare diseases are hard to predict. By learning how rare codes co-occur with common ones, the reranker significantly boosted the F1-score for infrequent labels.
  • Qualitative Success: In case studies, the reranker successfully added missing but highly correlated codes (e.g., adding "Invasive mechanical ventilation" when an "Endotracheal tube" was already predicted) and deleted logically inconsistent ones.

Conclusion

While the reranker is currently limited by the initial candidate pool generated by the base model, it represents a significant step forward in making AI-driven medical coding more logically consistent. Future work may involve training the base predictor and the reranker jointly (end-to-end) to allow the text-processing components to learn from label correlations directly.

Summary updated

BibTeX
@inproceedings{tsai2021modeling,
  author = {Shang-Chi Tsai and Chao-Wei Huang and Yun-Nung Chen},
  title = {{Modeling Diagnostic Label Correlation for Automatic ICD Coding}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=xJPMEn_lY3l},
}
Figure