Modeling Diagnostic Label Correlation for Automatic ICD Coding

Shang-Chi Tsai, Chao-Wei Huang, Yun-Nung Chen

TL;DR

This paper tackles the challenge of automatically assigning diagnostic codes (ICD codes) from clinical notes, a task complicated by the vast number of codes and their interdependencies. They propose a novel two-stage framework that first generates candidate ICD code sets and then "reranks" these sets to better capture label correlations, significantly improving prediction accuracy.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) contain valuable clinical notes, but extracting structured diagnostic information, like International Classification of Diseases (ICD) codes, is a difficult multi-label classification problem. The challenges are threefold: a massive number of potential codes, a hierarchical structure where codes are related, and a severe class imbalance (some codes are very rare). Existing methods often treat each code prediction independently, ignoring the fact that certain codes are more likely to co-occur or that some combinations are mutually exclusive. This can lead to incomplete or contradictory diagnoses. The authors aim to improve ICD coding accuracy by explicitly modeling these label dependencies.

Methods

The proposed solution is a two-stage framework:

  1. Candidate Generation: A "base predictor" (e.g., a neural network) is used to generate an initial set of probable ICD code candidates for a given clinical note. This stage leverages standard multi-label classification techniques.
  2. Candidate Reranking: This is the core novelty. Instead of directly predicting labels, this stage takes the candidate sets from the first stage and reranks them. The reranker learns to estimate the joint probability of a label set, effectively capturing correlations between codes. The authors introduce two specific reranker architectures:
  3. MADE (Masked Autoencoder): This density estimator models the joint probability of a label set autoregressively. It decomposes the probability of a set into a product of conditional probabilities, learning dependencies between labels.
  4. Masked Self-Attention (Mask-SA): Inspired by BERT, this reranker uses a Transformer architecture. It employs a cloze-style prediction task, masking one label at a time and predicting it based on the other labels in the set. This allows it to condition on all other labels, potentially capturing more complex dependencies than MADE.

The final prediction is the reranked label set with the highest score, calculated by combining the base predictor's score with the reranker's score.

Datasets

The experiments were conducted on two widely used benchmark datasets for ICD code prediction:

  • MIMIC-2: A dataset containing clinical notes.
  • MIMIC-3: A larger dataset with discharge summaries from an ICU.

The authors followed established settings for these datasets, including variations that consider only the top-50 most frequent codes or all available codes. No new benchmarks were introduced.

Key Insights and Ideas

  • Label Correlation is Crucial: The paper strongly emphasizes that treating ICD code predictions independently is a significant limitation. Explicitly modeling label correlations leads to substantial improvements.
  • Reranking as a Powerful Tool: The two-stage retrieve-and-rerank approach, inspired by speech recognition and parsing, proves effective for ICD coding. It allows for leveraging existing base predictors while adding a layer of sophisticated dependency modeling.
  • Model Agnosticism: The framework is designed to be model-agnostic, meaning it can work with various base predictors. This flexibility is a key advantage.
  • Learning from Data, Not Experts: Unlike some prior work that relied on pre-defined hierarchical structures or expert knowledge, this method learns label correlations directly from the data.
  • Infrequent Labels Benefit Most: The reranking mechanism shows particular promise in improving predictions for less frequent ICD codes, addressing a critical aspect of the class imbalance problem.

Key Results

  • The proposed reranking framework consistently improved the performance of all tested base predictors on both MIMIC-2 and MIMIC-3 datasets.
  • Relative improvements in macro F-score for all-code settings ranged from 1% to 16%, demonstrating significant gains on a challenging task.
  • The Mask-SA reranker consistently provided larger improvements in macro F-score, especially in the all-code settings.
  • The average ranking of the best-performing label set improved from the 24th to the 19th place after reranking, indicating that the correct diagnosis is brought closer to the top.
  • The performance on infrequent labels also increased with reranking.

Limitations and Future Work

  • Candidate Number Sensitivity: The paper notes that reranking performance plateaus or slightly decreases after a certain number of candidates (around 10). They hypothesize this is because candidates ranked lower by the base predictor have original scores that are too low to be significantly boosted by the reranker.
  • Hyperparameter Tuning: The effectiveness of the reranker depends on hyperparameters like $\alpha$ (weighting the reranker score) and $\beta$ (length penalty), which require careful tuning.
  • Computational Cost: While the framework is flexible, the reranking stage adds computational overhead.
  • Future Work: The authors suggest that their framework's flexibility allows for designing new rerankers. They also highlight the potential for benefiting from additional ICD label information and reducing the demand for paired training data, which could lead to greater scalability in medical NLP.

Summary updated

Abstract

Given the clinical notes written in electronic health records (EHRs), it is challenging to predict the diagnostic codes which is formulated as a multi-label classification task. The large set of labels, the hierarchical dependency, and the imbalanced data make this prediction task extremely hard. Most existing work built a binary prediction for each label independently, ignoring the dependencies between labels. To address this problem, we propose a two-stage framework to improve automatic ICD coding by capturing the label correlation. Specifically, we train a label set distribution estimator to rescore the probability of each label set candidate generated by a base predictor. This paper is the first attempt at learning the label set distribution as a reranking module for medical code prediction. In the experiments, our proposed framework is able to improve upon best-performing predictors on the benchmark MIMIC datasets. The source code of this project is available at https://github.com/MiuLab/ICD-Correlation.
BibTeX
@inproceedings{tsai2021modeling,
  author = {Shang-Chi Tsai and Chao-Wei Huang and Yun-Nung Chen},
  title = {{Modeling Diagnostic Label Correlation for Automatic ICD Coding}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=xJPMEn_lY3l},
}
Figure