Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

Hongyi Yuan, Zheng Yuan, Sheng Yu

Introduction

Biomedical Entity Linking (EL) is the critical task of mapping entity mentions in clinical or scientific text to standardized concepts in a Knowledge Base (KB), such as the Unified Medical Language System (UMLS). This task is uniquely challenging due to the extreme linguistic variability in medicine; a single condition like "Reactive arthritis" may appear as "Reiter syndrome," "Reiter's syndrome," or the abbreviation "ReA."

Traditional EL systems rely on similarity-based discriminative methods, which require storing massive vector representations for millions of KB concepts, leading to high memory overhead. While generative models like GENRE have emerged as a memory-efficient alternative by "typing out" concept names, they often struggle with the lack of domain-specific labeled data and the ambiguity of medical synonyms. To address these gaps, researchers have introduced GenBioEL, a generative framework that leverages KB-guided pre-training and a synonym-aware fine-tuning strategy.

Methodology

GenBioEL utilizes a BART-large backbone to treat entity linking as a sequence-to-sequence (seq2seq) task. The model's success is built on three core innovations:

1. Knowledge Base-Guided Pre-training

To overcome the scarcity of human-annotated biomedical data, the authors developed a synthetic pre-training phase using the UMLS. By transforming KB definitions and synonyms into natural language templates (e.g., "[Mention] is defined as [Definition]"), the model learns the semantic relationships between various names for the same concept and their clinical descriptions before seeing any labeled sentences.

2. Synonyms-Aware Fine-tuning

Standard fine-tuning often uses a single "canonical" name as the target, which can be difficult if the mention in the text is linguistically distant. GenBioEL employs a "synonym-aware" strategy: during training, it uses TF-IDF similarity to select the synonym from the KB that is most textually similar to the input mention. This reduces the optimization difficulty and helps the model learn more effectively from diverse mentions.

The GenBioEL workflow: from KB concepts to pre-training templates and synonym-aware fine-tuning.

3. Constrained Inference

To prevent the model from "hallucinating" non-existent medical terms, GenBioEL uses a multi-synonym constrained prefix tree during decoding. This ensures the model only generates strings that correspond to valid entries in the target KB. Additionally, "decoder prompts" (e.g., starting output with "[Mention] is...") are used to guide the model's internal language modeling capabilities.

Experimental Results

GenBioEL was evaluated on four major biomedical benchmarks: BC5CDR (chemicals/diseases), NCBI-disease, COMETA (colloquial Reddit terms), and AskAPatient (social media phrases).

Performance Benchmarks

The model achieved state-of-the-art (SOTA) results across nearly all datasets. Notably, it reached 93.3% Recall@1 on BC5CDR and showed significant improvements on challenging colloquial datasets like COMETA and AskAPatient.

Performance comparison of GenBioEL against previous state-of-the-art models on NCBI, COMETA, and AAP datasets.

Key Insights

  • Generalization to Unseen Concepts: GenBioEL significantly outperformed discriminative models on concepts not present in the training set. This suggests that the generative approach develops a deeper semantic understanding of medical terminology rather than just memorizing vector distances.
  • Efficiency: Unlike many top-performing models, GenBioEL achieves these results without a separate candidate selection (pre-filtering) step, greatly simplifying the deployment pipeline.
  • Synthetic Data Efficacy: The KB-guided pre-training proved more effective for the medical domain than general-domain pre-training on Wikipedia, highlighting the value of structured medical knowledge in training generative models.

Conclusion

GenBioEL demonstrates that generative AI can effectively bridge the gap in biomedical entity linking by leveraging the structured information already present in knowledge bases. By aligning pre-training with KB definitions and fine-tuning with textually similar synonyms, the model achieves superior accuracy and generalization while maintaining a streamlined, end-to-end architecture. Future work may explore integrating candidate sets to further boost performance or refining the model's handling of exceptionally long mentions.

Summary updated

Abstract

Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names. Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method. In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. We propose KB-guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names. We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multi-synonyms constrained prefix tree for inference. Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies.
BibTeX
@inproceedings{yuan2022generative,
  author = {Hongyi Yuan and Zheng Yuan and Sheng Yu},
  title = {{Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=WkdPResXXlI},
}
Figure