Abstract
Normalizing disease names is a crucial task for biomedical and healthcare domains. Previous work explored various approaches, including rules, machine learning and deep learning, which focused on only one approach or one model. In this study, we systematically investigated the performances of various neural models and the effects of different features. Our investigation was performed on two benchmark datasets, namely the NCBI disease corpus and the BioCreative V Chemical Disease Relation (BC5CDR) corpus. The convolutional neural network (CNN) performed the best (F1 90.11%) in the NCBI disease corpus and the attention neural network (Attention) performed the best (F1 90.78%) in the BC5CDR corpus. Compared with the state-of-the-art system, DNorm, our models improved the F1s by 1.74% and 0.86% respectively. In terms of features, character information could improve the F1 by about 0.5-1.0% while sentence information worsened the F1 by about 3-4%. Moreover, we proposed a novel approach for pre-training models, which improved the F1 by up to 9%. The CNN and Attention models are comparable in the task of disease name normalization while the recurrent neural network performs much worse. In addition, character information and pre-training techniques are helpful for this task while sentence information hurts the performance.
Introduction
In biomedical text mining, "disease name normalization" (or entity linking) involves mapping mentions of diseases in unstructured text to standardized identifiers in a controlled vocabulary, such as MEDIC. This task is challenging due to the frequent use of synonyms, abbreviations, and morphological variations by clinicians and researchers. For instance, mentions like "dyskinesias" and "Parkinson's disease" must be accurately linked to their respective unique concept IDs.
While previous methods relied on hand-crafted rules or specific machine learning models, this research provides a systematic comparison of neural network architectures and introduces a novel pre-training strategy to address the scarcity of labeled training data.
Methods
The researchers evaluated three primary neural architectures:
- Convolutional Neural Networks (CNN): Effective at identifying local patterns and key terms within a mention.
- Recurrent Neural Networks (Bi-LSTM): Designed for sequence processing and long-term dependencies.
- Attention Networks: Designed to assign varying weights to words based on their importance to the mention.
These architectures were tested across three feature configurations:
- Word-based Neural Model (WNM): Utilizes FastText embeddings to capture semantic word meanings.
- Character-based Neural Model (CNM): Incorporates character-level information to handle typos and common medical affixes (e.g., "renal").
- Sentence-based Neural Model (SNM): Includes the surrounding sentence context of the disease mention.
Vocabulary Pre-training
A key innovation of this study is a semi-supervised pre-training approach. Instead of relying solely on small labeled datasets, the models were pre-trained using the entire MEDIC vocabulary (approximately 9,700 concepts and 67,000 names). By treating the dictionary names as "automatically labeled" data, the models learned the general structure and terminology of diseases before being fine-tuned on specific corpora.
Results and Discussion
The study utilized the NCBI Disease Corpus and the BC5CDR Corpus for evaluation. Several critical insights emerged:
- Architecture Performance: CNNs and Attention networks significantly outperformed Bi-LSTMs. The poor performance of RNNs is likely because disease mentions are typically short, making long-term dependency modeling unnecessary and potentially noisy.
- Character-Level Benefits: Adding character information (CNM) improved F1 scores by roughly 1%. This helps the model process Out-of-Vocabulary (OOV) words by analyzing sub-word components.
- Negative Impact of Context: Surprisingly, incorporating sentence-level context (SNM) decreased performance by 3-4%. In this specific task, surrounding words often act as noise rather than providing useful disambiguation.
- Impact of Pre-training: Pre-training on the MEDIC vocabulary provided the most significant boost, improving F1 scores by up to 9%. This demonstrates that reference dictionaries are highly valuable for training in specialized domains.
The CNN-based model achieved an F1 of 90.11% on the NCBI corpus, while the Attention-based model reached 90.78% on the BC5CDR corpus. Both results surpassed the previous state-of-the-art system, DNorm.
Conclusion
This investigation highlights that CNN and Attention models are well-suited for disease name normalization. While character-level features and vocabulary pre-training are highly beneficial, sentence-level context can be detrimental. The proposed framework is entity-agnostic and can be adapted to other biomedical entities such as chemicals or genes. Future work will explore larger corpora to further reduce the impact of OOV words.
Summary updated