Coding Electronic Health Records with Adversarial Reinforcement Path Generation

Shanshan Wang, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jian-Yun Nie, Jun Ma, Maarten de Rijke

Abstract

RPGNet treats Electronic Health Record (EHR) coding as a coarse-to-fine path generation problem through the ICD taxonomy tree. By employing adversarial reinforcement learning and a novel message-passing module, the model navigates the complex hierarchy of medical codes, significantly improving the detection of rare diseases compared to traditional flat classification models.

Introduction

Assigning International Classification of Diseases (ICD) codes to patient records is a critical task for hospitals, impacting billing, clinical research, and epidemiology. However, automated coding faces two major hurdles: high dimensionality with a long-tailed distribution (over 18,000 possible codes) and structural neglect. Most existing models treat coding as a "flat" multi-label classification problem, ignoring the inherent hierarchical organization of the ICD taxonomy where specific conditions are nested under broader categories.

Methodology: Reinforcement Path Generation Network (RPGNet)

The authors propose RPGNet, which models ICD coding as a sequential decision-making process. Instead of a single classification step, the model "walks" down the ICD tree from the root to specific leaf codes, effectively reducing the search space at each step.

Hierarchical path generation in the ICD taxonomy.

Path Generator (PG)

The Path Generator utilizes a Path Message Passing (PMP) module to integrate information across three relationship types: EHR-to-code, parent-to-child, and sibling-to-sibling. This allows the model to leverage context from related diagnoses. To mitigate error propagation—where an early "wrong turn" prevents reaching the correct leaf—a hybrid policy combines local search (children) with a global view of all codes at a given level. The EHR itself is processed using a multi-channel CNN to extract clinical features from unstructured text.

Path Discriminator (PD)

To avoid the difficulty of manual reward engineering in reinforcement learning, RPGNet uses an adversarial approach. A Path Discriminator is trained to distinguish between ground-truth paths and those generated by the PG. The probability that the Discriminator identifies a path as "authentic" serves as the reward signal for the Generator.

Architecture of the RPGNet showing the Path Generator and Path Discriminator components.

Experiments and Results

The model was evaluated on the MIMIC-III dataset, a gold-standard benchmark for clinical AI, using two settings: Top-50 labels and the Full-label set (9,219 codes).

RPGNet demonstrated superior performance over state-of-the-art baselines like CAML and MSATT-KG. In the Top-50 setting, it achieved a micro-recall of 56.81%, representing a 14.5% improvement over the best baseline (HA-GRU). Notably, the model excelled in identifying rare diseases, reflected in significantly higher Macro-F1 and Macro-AUC scores. In the challenging full-label setting, RPGNet maintained its lead with a 14.68% Jaccard similarity, outperforming the previous best of 13.69%.

Conclusion and Future Work

RPGNet demonstrates that leveraging the hierarchical structure of medical taxonomies through path generation significantly improves coding accuracy, especially for rare conditions. While currently limited to unstructured clinical notes, future research directions include the integration of multi-modal EHR data (e.g., lab results, medications) and grounding the path generation in external medical knowledge bases to further refine diagnostic logic.

Summary updated

Abstract

Electronic Health Record (EHR) coding is the task of assigning one or more International Classification of Diseases (ICD) codes to every EHR. Most previous work either ignores the hierarchical nature of the ICD codes or only focuses on parent-child relations. Moreover, existing EHR coding methods predict ICD codes from the leaf level with the greatest ICD number and the most fine-grained categories, which makes it difficult for models to make correct decisions. In order to address these problems, we model EHR coding as a path generation task. For this approach, we need to address two main challenges: (1) How to model relations between EHR and ICD codes, and relations between ICD codes? (2) How to evaluate the quality of generated ICD paths in order to obtain a signal that can be used to supervise the learning? We propose a coarse-to-fine ICD path generation framework, named Reinforcement Path Generation Network (RPGNet), that implements EHR coding with a Path Generator (PG) and a Path Discriminator (PD). We address challenge (1) by introducing a Path Message Passing (PMP) module in the PG to encode three types of relation: between EHRs and ICD codes, between parent-child ICD codes, and between sibling ICD codes. To address challgenge (2), we propose a PD component that estimates the reward for each ICD code in a generated path. RPGNet is trained with Reinforcement Learning (RL) in an adversarial manner. Experiments on the MIMIC-III benchmark dataset show that RPGNet significantly outperforms state-of-the-art methods in terms of micro-averaged F1 and micro-averaged AUC.
BibTeX
@inproceedings{wang2020coding,
  author = {Shanshan Wang and Pengjie Ren and Zhumin Chen and Zhaochun Ren and Jian-Yun Nie and Jun Ma and Maarten de Rijke},
  title = {{Coding Electronic Health Records with Adversarial Reinforcement Path Generation}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2020},
  doi = {10.1145/3397271.3401135},
}
Figure