Deep Contextualized Biomedical Abbreviation Expansion

Qiao Jin, Jinling Liu, Xinghua LuACL 2019

TL;DR

This paper introduces DECBAE, a novel deep learning model for automatically expanding ambiguous biomedical abbreviations. By leveraging contextualized word embeddings from BioELMo and specialized biLSTM classifiers, DECBAE achieves state-of-the-art performance, even surpassing human experts in some cases, and demonstrates robustness across various challenging settings.

What Problem Does This Paper Address?

Biomedical literature is rife with abbreviations, which can be highly ambiguous (e.g., "ER" can mean estrogen receptor, endoplasmic reticulum, or emergency room). This ambiguity poses a significant challenge for information retrieval, question answering, and other natural language processing (NLP) tasks in the biomedical domain. While some methods can expand abbreviations when their definitions are explicitly provided in the text, this paper focuses on the more difficult problem where disambiguation requires reasoning over the context. Existing approaches often struggle with the sheer volume and ambiguity of these abbreviations, necessitating more sophisticated methods for accurate expansion.

Methods

The core of the DECBAE model lies in its innovative approach to leveraging contextual information and specialized classifiers.

  • Dataset Construction: The authors employ a heuristic to automatically construct a large supervised dataset. They identify sentences where an abbreviation is defined and then use that definition as the label for all subsequent occurrences of that abbreviation within the same abstract. This process is refined by grouping similar definitions using MeSH terms and edit distance.
  • Contextualized Embeddings: DECBAE utilizes BioELMo, a domain-specific version of ELMo pre-trained on a massive corpus of PubMed abstracts. BioELMo provides rich, contextualized word embeddings that capture the meaning of words based on their surrounding text.
  • Abbreviation-Specific Classifiers: For each ambiguous abbreviation, a dedicated bidirectional LSTM (biLSTM) classifier is trained. These biLSTMs take the BioELMo embeddings of a sentence as input and learn to predict the correct definition for the specific abbreviation. This specialization allows the model to tailor its predictions to the nuances of each abbreviation.
  • Inference: During inference, the model first identifies potential ambiguous abbreviations in a sentence. Then, it uses BioELMo to generate embeddings and feeds these into the corresponding pre-trained abbreviation-specific biLSTM classifier to predict the most likely definition.

Datasets

  • Training and Evaluation Dataset: The authors automatically collected a dataset from PubMed abstracts, focusing on 950 ambiguous abbreviations. This dataset was further filtered by a biomedical expert to ensure the abbreviations were meaningful and truly ambiguous.
  • Subsets for Evaluation: The dataset was divided into several subsets to test the model's robustness:
  • Random Set: A random sample of 100 abbreviations.
  • Imbalanced Set: Abbreviations with a dominant definition occurring over 95% of the time.
  • Low-resources Set: Abbreviations with fewer than 1,000 training instances.
  • Clinical Set: 11 abbreviations identified from previous clinical abbreviation disambiguation work.
  • Human Expert Test: A specific subset for evaluating human performance on the abbreviation "DAT."

Key Insights and Ideas

  • Specialization is Key: Training separate classifiers for each abbreviation, rather than a single general model, significantly boosts performance. This allows the model to learn the unique contextual cues for each ambiguous term.
  • Contextual Embeddings are Powerful: BioELMo's ability to capture nuanced word meanings based on context is crucial for disambiguating abbreviations, outperforming simpler feature representations like bag-of-words.
  • Automatic Dataset Creation is Viable: The heuristic approach to generating a large, supervised dataset from PubMed abstracts, while potentially noisy, proves effective and significantly reduces the need for expensive manual annotation.
  • Robustness Across Settings: The model demonstrates impressive resilience to imbalanced data, low-resource scenarios, and even clinical text, suggesting broad applicability.

Key Results

DECBAE achieved state-of-the-art performance across various subsets:

  • Overall Performance: On the random subset (representing the whole dataset), DECBAE achieved an accuracy of 96.1% and a macro-F1 score of 91.7%.
  • Outperforming Baselines: DECBAE consistently outperformed other baselines, including Majority, Bag-of-Words (BoW-FFN), BioELMo alone, and biLSTM models, by significant margins across all metrics and subsets.
  • Human Performance: In a direct comparison on the "DAT" abbreviation, DECBAE's accuracy (95.1%) and macro-F1 (98.3%) surpassed the single human expert's performance (97.1% and 92.6%, respectively). The paper notes that human errors often involved confusing similar-sounding or related definitions.
  • Robustness: The model maintained high performance even on imbalanced datasets (98.9% accuracy) and low-resource settings (98.7% accuracy), highlighting its practical utility.

Limitations and Future Work

  • Dataset Specificity: The authors acknowledge that DECBAE was primarily tested on their automatically collected dataset. Evaluating its performance on other established biomedical abbreviation datasets, such as MSH WSD, is a clear direction for future work.
  • Inference Speed: The current model relies on two recurrent models (BioELMo and biLSTM), which can lead to slower inference. The paper suggests that exploring simpler architectures, like BioELMo with feed-forward networks, might offer a trade-off for faster inference without a drastic drop in performance.
  • Expert Involvement: While the dataset collection is largely automated, a minimal level of expert filtering was used. Further research could explore fully unsupervised methods or more sophisticated active learning strategies to minimize expert intervention.

Summary updated

BibTeX
@inproceedings{jin2019deep,
  author = {Qiao Jin and Jinling Liu and Xinghua Lu},
  title = {{Deep Contextualized Biomedical Abbreviation Expansion}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure