Abstract
Biomedical entity linking (BioEL) aims at mapping biomedical mentions to pre-defined entities. While extensive research efforts have been devoted to BioEL, applying large language models (LLMs) for BioEL has not been fully explored. Previous attempts have revealed difficulties when directly applying LLMs to the task of BioEL. Possible errors include generating non-entity sentences, invalid entities, or incorrect answers. To this end, we introduce LLM4BioEL, a concise yet effective framework that enables LLMs to adapt well to the BioEL task. LLM4BioEL employs restrictive decoding to ensure the generation of valid entities and utilizes entropy-based contrastive decoding to incorporate additional biomedical knowledge without requiring further tuning. Besides, we implement few-shot prompting to maximize the in-context learning capabilities of LLM. Extensive experiments demonstrate the effectiveness and applicability of LLM4BioEL across different BioEL tasks and with different LLM backbones, and the best-performing LLM4BioEL variant outperforms the traditional and LLM-based BioEL baselines.
Introduction
Biomedical Entity Linking (BioEL) is a critical task in bioinformatics, involving the mapping of textual mentions (e.g., "zirconia") to unique identifiers in a Knowledge Graph (KG) like UMLS. This task is complicated by significant morphological variations and inherent ambiguity in medical terminology.
While Large Language Models (LLMs) possess vast internal knowledge, they struggle with BioEL when used out-of-the-box. As shown in Figure 1, standard LLMs often hallucinate, provide conversational filler, or link to entities that do not exist in the target KG. These "guardrail" failures make them unreliable for clinical applications without specialized intervention.
Methodology: The LLM4BioEL Framework
To address these challenges, we propose LLM4BioEL, a framework that guides LLM generation at the token level without requiring expensive fine-tuning. The architecture, illustrated in Figure 2, relies on three core components:
- Restrictive Decoding: This mechanism enforces a hard constraint on the model's output. By masking tokens that would lead to strings outside the target KG, it ensures that every generated entity is valid.
- Entropy-based Contrastive Decoding: This is the framework's primary innovation. It monitors the LLM's internal uncertainty (entropy). When the model is "confused" (high entropy), the system increases the weight of an external semantic retriever (SapBERT). This allows for a dynamic balance between the LLM's reasoning and external biomedical knowledge.
- In-context Learning (ICL): The retriever identifies semantically similar training examples to include in the prompt, providing the model with task-specific context.
Experimental Results
We evaluated LLM4BioEL on three benchmark datasets: NCBI (diseases), BC5CDR (chemicals/diseases), and COMETA (informal Reddit terms).
The results demonstrate a transformative performance boost. For instance, on the NCBI dataset, Mistral-7b's accuracy (Hits@1) improved from 0.9% with standard greedy decoding to 92.9% using our framework. Furthermore, LLM4BioEL with a Llama-3-70b backbone achieved an average Hits@1 of 90.3%, significantly outperforming GPT-4, which averaged only 55.3%. Crucially, the restrictive decoding mechanism ensured 100% validity in generated entities (Hits@KG) across all models.
Conclusion and Future Work
LLM4BioEL effectively bridges the gap between general-purpose LLMs and the rigid requirements of biomedical entity linking. By intelligently combining internal model confidence with external knowledge retrieval, it achieves state-of-the-art results without fine-tuning.
Future work will focus on optimizing the retrieval bottleneck—ensuring the correct entity is always within the candidate set—and exploring hybrid retrieval methods to further enhance accuracy. One current limitation is the requirement for logit access, which restricts the framework's use with closed-source APIs.
Summary updated