Introduction
In the field of Natural Language Processing (NLP), identifying "Named Entities" such as diseases or chemicals is a fundamental task. However, biomedical language is rarely straightforward. Entities are frequently "nested" within one another—for instance, in the phrase "chronic heart failure," "heart" is an anatomical part, while the entire phrase represents a disease. Standard Named Entity Recognition (NER) systems typically treat text as "flat," selecting one label and ignoring others, which leads to significant information loss.
The BioNNE (Biomedical Nested Named Entity Recognition) task, presented at the BioASQ 2024 workshop, aimed to address this by challenging researchers to develop systems capable of recognizing these overlapping layers of information in both English and Russian medical texts.
The Challenge of Nested Entities
Traditional NER assumes that a string of text belongs to exactly one category. In scientific literature, this "nesting" is pervasive. For example, in the term "adenosine receptor," a flat system might label the entire span as a protein, missing the fact that "adenosine" is a distinct chemical entity.
The BioNNE task focused on the eight most common medical categories: FINDING, DISO (Disease), INJURY_POISONING, PHYS (Physiology), DEVICE, LABPROC (Laboratory Procedure), ANATOMY, and CHEM. By focusing on both English and Russian, the task also explored the potential for cross-lingual transfer—leveraging data from one language to improve performance in another.
Methodology and the Winning Approach
The shared task was divided into three tracks: a bilingual track (Track 1), an English-oriented track (Track 2), and a Russian-oriented track (Track 3). The top-performing team, fulstock, utilized the BINDER model. Unlike traditional sequence labeling, BINDER uses a bi-encoder architecture to map text spans and entity type descriptions into a shared vector space.
Key innovations from the winning team included:
- Backbone Model: The use of XLM-RoBERTa, a powerful multilingual transformer.
- Contextual Prompt Engineering: Instead of using simple entity names (e.g., "Disease"), they provided the model with "contextual prompts"—example sentences containing the target entity type. This approach yielded significantly better results than standard definitions.
Dataset and Evaluation
The task relied on a refined version of the NEREL-BIO dataset, consisting of PubMed abstracts. The training data was heavily skewed toward Russian (716 documents) compared to English (54 documents). To ensure a fair and robust evaluation, the organizers introduced a new, manually annotated test set of 154 parallel abstracts (the same text in both English and Russian).
Results and Analysis
The evaluation utilized the Macro F1-score to measure accuracy across all eight entity classes. The results highlighted several key insights:
- Cross-Lingual Transfer: Models trained on the combined RU+EN dataset outperformed those trained only on English. The larger Russian training set allowed models to learn general medical concepts that successfully transferred to the English tasks.
- Prompting Efficacy: The success of contextual prompts suggests that NNER models benefit more from seeing how an entity is used in situ than from abstract dictionary definitions.
The winning system significantly outperformed the organizers' baseline (which utilized the BERGAMOT model), particularly in the English track, where prompt engineering provided a nearly 5-point boost. However, data sparsity remains a hurdle; categories like DEVICE and INJURY_POISONING had fewer examples, leading to lower performance compared to DISO or ANATOMY.
Conclusion
The BioNNE task established a new benchmark for recognizing nested medical entities across multiple languages. The competition demonstrated that contrastive learning models combined with sophisticated prompt engineering can effectively navigate the hierarchical complexity of biomedical language. Future work will likely focus on addressing language imbalance and improving performance on rare entity types through synthetic data or few-shot learning.
Summary updated