xMEN: A Modular Toolkit for Cross-Lingual Medical Entity Normalization

Florian Borchert, Ignacio Llorca, Roland Roller, Bert Arnrich, Matthieu-P. Schapranow

Abstract

Medical Entity Normalization (MEN) is a critical task in clinical Natural Language Processing, yet it suffers from a significant English-centric bias. Most terminologies and annotated datasets are predominantly in English, leaving other languages with a substantial "resource gap." We present xMEN, an open-source Python toolkit designed to enable high-performance MEN across multiple languages. By integrating cross-lingual candidate generation with a novel rank regularization loss function and leveraging weakly supervised models trained on machine-translated data, xMEN achieves state-of-the-art results in low-resource scenarios.

Introduction

In clinical NLP, identifying a disease mention (Named Entity Recognition) is only the first step. The subsequent challenge is normalization: mapping a text string (e.g., "lupus") to a unique, canonical identifier in a knowledge base like the Unified Medical Language System (UMLS). While English normalization is well-supported, researchers working with French, Spanish, or German often face a lack of synonyms and training data. xMEN addresses this by providing a modular framework that utilizes English resources to enhance performance in other languages.

The xMEN Framework

The xMEN toolkit implements a modular pipeline consisting of four primary stages designed for flexibility and performance.

The xMEN pipeline architecture, illustrating the flow from data loading to candidate ranking using a modular approach.
  1. Data Loading: xMEN integrates with the BigBIO framework, ensuring compatibility with various biomedical datasets and simplifying the ingestion of NER-tagged text.
  2. Candidate Generation (CG): To identify potential matches, xMEN employs an ensemble of TF-IDF for surface-level similarity and SapBERT for semantic similarity. A key feature is cross-lingual CG, which uses English aliases to bridge gaps when target language synonyms are missing.
  3. Candidate Ranking (CR): A Cross-Encoder (CE) model re-ranks the generated candidates. To overcome the lack of non-English training data, the authors utilized weak supervision by projecting labels from the English MedMentions dataset into target languages using machine translation (NLLB-200).
  4. Rank Regularization: A core contribution of xMEN is a regularization term added to the loss function during re-ranker training. This penalizes the model if the new ranking deviates excessively from the initial candidate scores, preserving the "common sense" of the candidate generator and improving stability.

Experimental Setup and Datasets

The toolkit was evaluated on several multilingual benchmarks:

  • MANTRA GSC: Parallel subsets in English, Spanish, French, Dutch, and German.
  • QUAERO: A French corpus (MEDLINE abstracts and EMEA drug monographs).
  • BRONCO150: German oncological discharge summaries.
  • DISTEMIST: Spanish clinical case reports.

Results and Key Insights

xMEN established new state-of-the-art benchmarks across multiple datasets and languages. The combination of string matching (TF-IDF) and semantic embeddings (SapBERT) proved robust for specific medical terminology.

Performance comparison (F1@1) of xMEN components and baselines across multiple languages and datasets.

Key findings include:

  • Weak Supervision Efficacy: Models trained on machine-translated data consistently outperformed raw candidate generators, demonstrating that "noisy" translated data is a viable substitute for scarce human-annotated data.
  • Impact of Re-ranking: On the QUAERO MEDLINE subset (French), the full pipeline achieved an F1@1 of 0.788, compared to 0.662 for the candidate generator alone.
  • Domain Specificity: In the German BRONCO dataset, the re-ranker improved performance in the "Treatments" category from 0.199 to 0.530 F1.
  • Rank Regularization: Forcing the neural re-ranker to respect initial rankings prevented overfitting and led to more accurate predictions.

Conclusion and Future Work

xMEN provides a reproducible and extensible foundation for multilingual clinical AI. While highly effective, challenges remain in handling complex, multi-word entities and high lexical ambiguity. Future work will focus on expanding the language scope beyond European languages, improving pre-processing for long entity spans, and exploring advanced generative models for candidate generation. The toolkit is publicly available on GitHub to support the broader research community.

Summary updated

BibTeX
@inproceedings{borchert2024xmen,
  author = {Florian Borchert and Ignacio Llorca and Roland Roller and Bert Arnrich and Matthieu-P. Schapranow},
  title = {{xMEN: A Modular Toolkit for Cross-Lingual Medical Entity Normalization}},
  booktitle = {JAMIA},
  year = {2024},
  doi = {10.1093/jamiaopen/ooae147},
}
Figure