Efficient Biomedical Entity Linking: Clinical Text Standardization with Low-Resource Techniques

Akshit Achara, Sanand Sasidharan, Gagan NACL 2024

TL;DR

This paper tackles the challenge of standardizing clinical text by accurately linking mentions of medical concepts to standardized codes. They introduce a novel, low-resource approach that significantly reduces training data requirements and leverages semantic information for improved disambiguation, achieving state-of-the-art performance on a benchmark dataset.

What Problem Does This Paper Address?

Clinical text is notoriously diverse, with many different ways to refer to the same medical concept (e.g., "heart attack," "myocardial infarction," "MI"). This variability makes it difficult for computers to understand and process medical information consistently across different institutions and systems. Biomedical entity linking (BEL) aims to bridge this gap by mapping these textual mentions to standardized codes from medical ontologies like the Unified Medical Language System (UMLS). However, existing BEL methods often require massive amounts of labeled training data and struggle with ambiguity, leading to inaccurate linking. This paper addresses the need for efficient, low-resource BEL techniques that can handle this complexity effectively.

Methods

The authors propose a multi-stage approach:

  • Low-Resource Training: Instead of relying on extensive labeled data, they fine-tune a distilled transformer model (MiniLM) using pairs of canonical names of entity synonyms from UMLS. This significantly reduces the data and computational resources needed.
  • Candidate Generation: They create a "prototype vector space" of UMLS entities using the trained model. Input mentions are then encoded and used to perform a semantic search within this space to generate a list of potential candidate entities.
  • Disambiguation (Reranking): To resolve ambiguity where multiple candidates have similar scores, they introduce several reranking techniques:
  • Parametric Reranking: This method uses a combination of cosine similarity score, candidate entity frequency, and alias information, optimized through grid search.
  • UMLS Semantic Information: They leverage UMLS's semantic types and groups to further refine candidate rankings by calculating similarity between the mention's semantic information and candidate entities' semantic information.
  • Contextualized Queries: To address cases where surface form alone is insufficient, they explore adding context to the mention query. This includes neighboring words, attention-based context, and implicit context (mean-pooled embeddings of the entire article).

Datasets

The primary dataset used is Medmentions. This dataset consists of titles and abstracts from 4392 English biomedical articles, with textual spans annotated with mentions of UMLS 2017AA entities. It comes in two versions: st21pv (recommended for information retrieval) and full. The authors also mention using UMLS 2023AB for context, noting that some annotations from 2017AA might be deprecated in the newer version.

Key Insights and Ideas

  • Data Efficiency is Key: The paper demonstrates that significant improvements in entity linking can be achieved with surprisingly little training data, challenging the notion that massive datasets are always necessary.
  • Reranking is Crucial for Disambiguation: Candidate generation alone is insufficient. The proposed reranking strategies, especially those leveraging UMLS semantic information, significantly boost performance by correctly identifying the intended entity.
  • UMLS Semantic Information is Powerful: Incorporating semantic types and groups from UMLS provides valuable signals for disambiguation, leading to substantial performance gains.
  • Context Matters, But How? While adding context to queries can help, the effectiveness varies greatly. Attention-based context showed promise, but simple neighboring words or implicit context did not yield significant improvements and sometimes even degraded performance.
  • Evaluation Needs Nuance: The authors highlight that simple retrieval metrics might not tell the whole story. They propose an article-level analysis to reveal deeper insights into performance, including issues with annotation granularity and surface form bias.

Key Results

  • Candidate Generation: The MiniLM model, even without extensive fine-tuning, achieved competitive candidate generation performance (R@1 around 0.553 for st21pv and 0.567 for full version).
  • Reranking Performance: The proposed reranking techniques significantly improved R@1 performance.
  • Parametric reranking improved R@1 to 0.614 (st21pv) and 0.630 (full).
  • Semantic group reranking achieved R@1 of 0.649 (st21pv) and 0.670 (full).
  • Semantic type reranking achieved the highest R@1 of 0.661 (st21pv) and 0.697 (full).
  • Contextualized Queries: Attention-based context (MiniELAC) showed the best performance among contextual approaches, but overall, context addition sometimes led to performance drops.
  • Qualitative Findings: The analysis revealed that the model sometimes struggles with the granularity of gold-standard annotations, occasionally penalizing more specific or related predictions.

Limitations and Future Work

  • Abbreviation Preprocessing: While an abbreviation preprocessing module is used, it doesn't convert all abbreviations, leading to ambiguity and potential errors. Improving abbreviation expansion is a key area for future work.
  • Contextual Ambiguity: Missing context in surface forms can lead to broader entity predictions. The paper suggests exploring more sophisticated ways to incorporate context.
  • Granularity of Gold Standard: The authors note inconsistencies in the granularity of gold standard entities, which can impact evaluation.
  • Future Work:
    • Developing methods to determine the "closeness" of a prediction to a surface form beyond binary matching.
  • Incorporating expert-defined semantic similarities for ranking.
  • Improving abbreviation expansion.
  • Developing more robust contextual approaches for disambiguation.

Summary updated

BibTeX
@inproceedings{achara2024efficient,
  author = {Akshit Achara and Sanand Sasidharan and Gagan N},
  title = {{Efficient Biomedical Entity Linking: Clinical Text Standardization with Low-Resource Techniques}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure