Improving Automatic KCD Coding: Introducing the KoDAK and an Optimized Tokenization Method for Korean Clinical Documents

Geunyeong Jeong, Juoh Sun, Seokwon Jeong, Hyunjin Shin, Harksoo KimACL 2023

TL;DR

This paper introduces KoDAK, the first large-scale dataset for automatic Korean Clinical Document (KCD) coding, and an optimized tokenization method tailored for Korean clinical text. Their approach significantly improves KCD coding accuracy compared to existing models, even with fewer parameters, addressing a critical gap in Korean medical NLP resources.

What Problem Does This Paper Address?

Accurately assigning standardized International Classification of Diseases (ICD) codes to electronic health records is crucial for healthcare analytics and management. However, this process is traditionally manual, costly, and prone to errors. While deep learning has shown promise for automated ICD coding, research has been hampered by several factors, particularly for non-English languages and specific national healthcare systems. These include:

  • Divergent ICD Systems: Countries often use modified versions of ICD (e.g., Korea's KCD), requiring specialized models.
  • Linguistic Differences: Each language has unique grammatical structures and ways of expressing medical terms, impacting model performance.
  • Resource Scarcity: Many countries, including Korea, lack large, publicly available clinical datasets and systematic medical knowledge resources, slowing down research progress.

This paper specifically targets the challenge of automatic KCD coding in Korea, where these resource and linguistic barriers are significant.

Methods

The authors propose a two-pronged approach:

  1. KoDAK Dataset Construction: They created the Korean Dataset for Automatic KCD coding (KoDAK) by collecting and preprocessing clinical documents with assigned KCD codes from Korean medical institutions. This involved obtaining ethical approval, ensuring patient confidentiality, and excluding sensitive data.
  2. Optimized Tokenization Method: Recognizing that Korean clinical documents often mix Korean and English medical terms, they developed a novel tokenization strategy. This method applies morpheme-aware subword tokenization for Korean, word-level tokenization for English medical terms and abbreviations (to preserve meaning), and character-level tokenization for numbers and special characters. This contrasts with standard tokenizers that might excessively split crucial English medical terms.

For the core coding model, they employed a transformer encoder architecture, initializing token embeddings with pre-trained Word2Vec embeddings (trained using skip-gram) to enhance representation richness. The model's final classification layer uses the [CLS] token representation, a common practice in BERT-like models.

Datasets

  • KoDAK (Korean Dataset for Automatic KCD coding): This is a newly constructed dataset for this study. It comprises 1,196,739 clinical documents collected from 23 departments over 17 years (2005-2021). Each record is labeled with a single, most appropriate KCD code. The dataset contains 8,862 unique KCD codes, representing 49% of all KCD codes, and exhibits a long-tail distribution.
  • MIMIC-III: Mentioned as a prominent English dataset for automatic ICD coding, used for comparison context.

Key Insights and Ideas

  • The "Mixed-Language" Challenge: Korean clinical notes frequently incorporate English medical terms and abbreviations. Standard tokenizers, even those for Korean, can break these terms into meaningless sub-units, hindering accurate coding. The proposed tokenization method directly addresses this by treating English terms differently.
  • Importance of Contextualized Embeddings: Using Word2Vec for initial token embeddings, rather than random initialization, significantly boosts model performance and accelerates convergence. This highlights the value of pre-trained representations, even if the main model isn't fully pre-trained.
  • Dataset Uniqueness Matters: The KoDAK dataset is crucial because it captures the specific linguistic nuances and coding practices of the Korean medical context, which are not represented in general-purpose or English-centric datasets.
  • Efficiency Gains: The proposed model achieves competitive results with significantly fewer parameters (60M vs. 105M for KM-BERT) and without requiring extensive pre-training, making it more practical for resource-constrained environments.

Key Results

The proposed method demonstrated superior performance compared to the Korean Medical BERT (KM-BERT) model:

  • Macro-F1 Improvement: Achieved a 0.14%p improvement in Macro-F1 score over KM-BERT.
  • Parameter Efficiency: The proposed model uses only 55% of the parameters of KM-BERT.
  • Ablation Study: Removing Word2Vec embeddings (w/o W) led to a performance decline, and using the KM-BERT tokenizer instead (w/o W & T) further degraded results, underscoring the effectiveness of both the proposed tokenization and Word2Vec embeddings.
  • Case Study: The paper presents examples where the proposed model correctly identifies KCD codes based on specific English medical terms (e.g., "accessory thumb," "retrograde amnesia"), while KM-BERT misclassifies them due to its less nuanced tokenization.

Limitations and Future Work

  • Unbalanced Label Distribution: The authors acknowledge the long-tail distribution in the KoDAK dataset, where a few codes are very frequent, and many are rare. This imbalance can pose challenges for model training.
  • Future Work: The authors plan to address the unbalanced label distribution in KoDAK and develop an enhanced pre-trained language model specifically designed for the Korean clinical field.

Summary updated

BibTeX
@inproceedings{jeong2023improving,
  author = {Geunyeong Jeong and Juoh Sun and Seokwon Jeong and Hyunjin Shin and Harksoo Kim},
  title = {{Improving Automatic KCD Coding: Introducing the KoDAK and an Optimized Tokenization Method for Korean Clinical Documents}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure