Multi-task learning for Chinese clinical named entity recognition with external knowledge

Ming Cheng, Shufeng Xiong, Fei Li, Pan Liang, Jianbo Gao

Extracting structured information from Electronic Medical Records (EMRs) is a cornerstone of modern clinical research. However, Chinese clinical Named Entity Recognition (NER) faces unique hurdles: linguistic complexity (abbreviations and grammatical errors), the "long tail" problem (difficulty identifying rare or unseen medical entities), and the high cost of expert annotation.

To address these challenges, researchers have developed DicMT (Dictionary-based Multi-Task), a deep learning framework that integrates domain-specific prior knowledge with multi-task neural networks to improve recognition accuracy, particularly for rare medical terms.

The DicMT Architecture

The DicMT model moves beyond purely data-driven approaches by combining context-aware embeddings with structural and external knowledge. The architecture is designed to process clinical text through a series of specialized layers, from feature encoding to multi-task prediction.

The DicMT architecture, illustrating the flow from input sentence through feature concatenation and BiLSTM layers to the multi-task output.

Feature Engineering: Radicals and Dictionaries

A key innovation of DicMT is its multi-faceted feature representation. The model concatenates three distinct types of embeddings:

  1. Fine-tuned BERT: A Chinese BERT model pre-trained on 1.2GB of unlabelled clinical records, providing deep, domain-specific contextual understanding.
  2. Radical-level Features: Chinese characters are pictographic. Many medical terms share sub-character components called "radicals." For instance, the radical for "meat" (月) often appears in characters related to organs. By encoding these, the model can infer the semantic category of rare or unseen characters.
  3. Dictionary Features: Instead of using dictionaries as simple lookups, DicMT integrates them as neural features. The model uses n-gram templates to match text fragments against external medical dictionaries, creating a feature vector that signals potential entity boundaries.
The dictionary feature extraction process using n-gram templates to identify potential entity boundaries and types.

Multi-Task Learning Strategy

The framework employs a Multi-Task Learning (MTL) approach to optimize performance. The network is trained on two tasks simultaneously:

  • Primary Task (NER): Predicting specific entity categories (e.g., symptoms, drugs, anatomy) using a BiLSTM-CRF (Conditional Random Field) architecture.
  • Auxiliary Task (NES): Named Entity Segmentation. This is a simpler binary classification task (determining if a token is part of an entity or not). By training on NES, the model learns to focus more effectively on entity boundaries, which in turn reduces the search space and complexity for the primary NER task.

Performance and Key Results

The model was evaluated on two public benchmarks (CCKS2017 and CCKS2018) and a private real-world dataset (FCCd) consisting of discharge records from the First Affiliated Hospital of Zhengzhou University.

DicMT consistently outperformed baseline models, achieving an average F-measure of 91.07% on the public benchmarks and 87.05% on the real-world FCCd dataset. Most notably, the model excelled at identifying "unseen" entities—those not present in the training data. For these rare terms, DicMT achieved a recall of approximately 54-56%, significantly outperforming standard BERT-based models.

Conclusion

DicMT demonstrates the power of "hybrid" AI in specialized domains. By combining the raw contextual power of Transformers with the structured wisdom of medical dictionaries and the semantic hints found in Chinese character radicals, the framework provides a robust solution for clinical information extraction. Future work will focus on addressing remaining challenges, such as identifying discontiguous, nested, or exceptionally long medical entities.

Summary updated

Abstract

Named entity recognition (NER) on Chinese electronic medical/healthcare records has attracted significantly attentions as it can be applied to building applications to understand these records. Most previous methods have been purely data-driven, requiring high-quality and large-scale labeled medical data. However, labeled data is expensive to obtain, and these data-driven methods are difficult to handle rare and unseen entities.
BibTeX
@article{cheng2021multi,
  author = {Ming Cheng and Shufeng Xiong and Fei Li and Pan Liang and Jianbo Gao},
  title = {{Multi-task learning for Chinese clinical named entity recognition with external knowledge}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2021},
  doi = {10.1186/s12911-021-01717-1},
}
Figure