Abstract
Medical code assignment from clinical text is a fundamental task in clinical information system management. As medical notes are typically lengthy and the medical coding system's code space is large, this task is a long-standing challenge. Recent work applies deep neural network models to encode the medical notes and assign medical codes to clinical documents. However, these methods are still ineffective as they do not fully encode and capture the lengthy and rich semantic information of medical notes nor explicitly exploit the interactions between the notes and codes. We propose a novel method, gated convolutional neural networks, and a note-code interaction (GatedCNN-NCI), for automatic medical code assignment to overcome these challenges. Our methods capture the rich semantic information of the lengthy clinical text for better representation by utilizing embedding injection and gated information propagation in the medical note encoding module. With a novel note-code interaction design and a graph message passing mechanism, we explicitly capture the underlying dependency between notes and codes, enabling effective code prediction. A weight sharing scheme is further designed to decrease the number of trainable parameters. Empirical experiments on real-world clinical datasets show that our proposed model outperforms state-of-the-art models in most cases, and our model size is on par with light-weighted baselines.
Introduction
Assigning standardized International Classification of Diseases (ICD) codes to clinical notes is a critical task for billing, clinical decision support, and research. However, clinical notes are often lengthy, unstructured, and filled with domain-specific jargon. Furthermore, the ICD code space is vast and sparse, with thousands of possible labels. Traditional automated methods often struggle with document length and label complexity. This paper introduces GatedCNN-NCI, a novel approach that combines gated convolutions with a bipartite graph interaction mechanism to effectively capture semantic information from long clinical documents and map them to their corresponding medical codes.
Methodology
The GatedCNN-NCI architecture consists of two main components: a Stacked Gated CNN for document encoding and a Note-Code Interaction (NCI) module for label assignment.
Stacked Gated CNNs
To handle the extreme length of clinical notes, the model utilizes dilated convolutions. Dilation increases the receptive field without significantly increasing the parameter count, allowing the model to capture long-range dependencies. A gating mechanism is applied to these convolutional layers to control the flow of information. Additionally, "embedding injection" is used to re-introduce original word embeddings at each layer, preventing the loss of specific diagnostic terms during deep feature extraction.
Note-Code Interaction (NCI)
Unlike models that treat codes as arbitrary indices, GatedCNN-NCI incorporates the textual descriptions of ICD codes (e.g., from the WHO). The model formulates code assignment as a bipartite graph problem. Textual features from the notes and code vectors interact via a message-passing mechanism, explicitly capturing the fine-grained dependencies between specific clinical terms and their corresponding medical codes.
Experiments and Results
The model was evaluated on the MIMIC-II and MIMIC-III datasets, which are gold-standard benchmarks for clinical text mining.
As shown in the results, GatedCNN-NCI consistently outperforms previous state-of-the-art models like CAML and MultiResCNN. On the MIMIC-III Top-50 subset, it achieved a Macro F1-score of 62.9%, representing a significant improvement over prior baselines. The model is particularly robust in Micro-averaged metrics, indicating high accuracy across the most frequent codes. Furthermore, GatedCNN-NCI is highly efficient; with only 7.6M parameters, it is significantly smaller than transformer-based models like ClinicalBERT (113.8M) while achieving superior performance on long documents.
Conclusion
GatedCNN-NCI addresses the dual challenges of document length and label sparsity in medical code assignment. By leveraging gated convolutions and explicit note-code interactions, the model achieves state-of-the-art performance with a relatively small parameter footprint. Future work will explore incorporating the hierarchical structure of ICD codes into the graph interaction module to further improve performance on rare codes.
Summary updated