Introduction
Automating the assignment of International Classification of Diseases (ICD-9-CM) codes to clinical documents is a critical step toward reducing healthcare administrative costs and improving research efficiency. However, a significant barrier to effective automation is the "long tail" distribution of medical codes. While common conditions have ample training data, rare but clinically significant diseases suffer from a lack of examples, leading to poor classifier performance.
Methodology
To address this data imbalance, we propose a method to enrich training datasets by strategically "borrowing" data from PubMed. The core innovation lies in bridging the linguistic gap between formal medical literature and practical clinical notes.
- Identification of Data-Poor Classes: We identified "Group 1" codes—those characterized by low training sample counts and below-average baseline performance.
- Synonym-Based Retrieval: Recognizing that official ICD-9-CM descriptions are often too formal (e.g., "Anorexia" vs. "loss of appetite"), we utilized synonyms from ICD9Data.com to query PubMed. This approach ensures retrieved articles better match the linguistic style of clinical reports.
- Model Training: Titles and abstracts from retrieved PubMed articles were added as supplementary positive training instances for Logistic Regression (LR) and Support Vector Machine (SVM) classifiers.
- Preprocessing: Medical concepts were extracted using MetaMap, and Negex was employed to filter out negated conditions.
Key Insights
A critical finding was the importance of Macro-averaging over Micro-averaging. Micro-average scores can be misleadingly high by favoring frequent classes, whereas Macro-averaging treats every code as equally important, revealing the true struggles of models on rare diseases. Furthermore, we observed a "sweet spot" for data enrichment; adding approximately 10 supplementary documents per code provided the best balance, while excessive data introduced noise that degraded performance.
Results
The integration of synonym-based PubMed data yielded substantial improvements, particularly for the underrepresented "Group 1" codes.
For SVM classifiers, the baseline Macro-F1 score of 48.03% increased to 67.43% using the synonym-based retrieval method. Similarly, LR performance improved from 51.52% to 62.85%. Statistical analysis via the Wilcoxon Signed Ranks test confirmed that the synonym-based method significantly outperformed both the baseline and retrieval based solely on official ICD descriptions.
Limitations and Future Work
Despite these gains, challenges remain. Broad codes lacking specific synonyms are difficult to retrieve effectively. Additionally, the system occasionally struggles with contextual nuances, such as distinguishing between an active diagnosis and a "history of" mention. Future research will focus on refining context-aware algorithms to better handle speculative language and historical medical mentions.
Summary updated