Abstract
While English biomedical Natural Language Processing (NLP) benefits from extensive annotated corpora and specialized tools, other languages like Spanish often lack similar resources. This paper demonstrates that a straightforward application of multilingual BERT (mBERT), pretrained on general-domain Wikipedia text, can achieve state-of-the-art results in Spanish biomedical named entity recognition (NER). On the PharmaCoNER task, the model achieved an 87.38% F-score, outperforming traditional feature-engineered baselines and showing strong generalization to unseen entities.
Introduction
The development of specialized NLP tools for the biomedical domain is often hindered by the scarcity of manually annotated data in languages other than English. The PharmaCoNER (Pharmacological Substances, Compounds and Proteins and Named Entity Recognition) task was established to address this gap for Spanish clinical texts. The primary challenge lies in the high degree of domain specificity and the difficulty of transferring performance from English-optimized tools to other languages. This research investigates whether a general-purpose multilingual model can be effectively adapted to a niche, non-English biomedical domain with minimal customization.
Methods
The researchers compared a traditional machine learning baseline with a modern transformer-based approach.
NERsuite Baseline
The baseline utilized NERsuite, a toolkit based on Conditional Random Fields (CRF). This approach relies on rich, manually defined features such as part-of-speech tags and lemmatization. To test language independence, the authors used a version optimized for English biomedical text to see how well its features would carry over to Spanish.
Multilingual BERT (mBERT)
The primary model was the cased version of mBERT, pretrained on Wikipedia articles across 104 languages. The architecture was modified by replacing standard output layers with a CRF layer to better handle sequence labeling dependencies. Fine-tuning was performed on the Spanish PharmaCoNER data using the Adam optimizer.
A critical component of the methodology involved handling BERT's subword tokenization (WordPiece). The authors implemented a strategy to map subword-level predictions back to whole-word labels to ensure compatibility with the evaluation metrics and original document offsets.
Datasets
The study employed the PharmaCoNER dataset, which consists of Spanish clinical case reports. The dataset tracks four main categories: PROTEINAS (Proteins/Genes), NORMALIZABLES (Chemicals mappable to external databases), NO_NORMALIZABLES (Non-mappable chemicals), and UNCLEAR (Miscellaneous related entities).
The training set comprises 500 documents with approximately 3,822 annotations, while the development set contains 250 documents. The data is characterized by a high volume of tokens relative to the number of annotations, emphasizing the sparsity of the target entities.
Results and Discussion
The mBERT model significantly outperformed the CRF-based baseline, achieving an 88% F-score on the development set and 87.38% on the official test set—a gain of approximately 6.5 percentage points over NERsuite.
Generalization and Pretraining
A key finding was the model's ability to generalize. mBERT achieved a 72% recall on entity spans that were entirely absent from the training data (which constituted 55% of the development set), compared to 96% for seen spans. This indicates that the model learns linguistic context rather than just memorizing a vocabulary list. Furthermore, an ablation study showed that without pretraining, the F-score dropped from 88% to 56%, highlighting the necessity of large-scale pretraining even when the source domain (Wikipedia) differs from the target domain (clinical text).
Category Performance
Performance varied by entity type. The model was most effective at identifying "Chemical(+)" (90.17% F-score) and "Proteins" (86.82% F-score). However, it struggled with the "Chemical(-)" category (19.05% F-score), likely due to the extreme data sparsity in that class, which contained only 24 training examples.
Conclusion
This research proves that multilingual BERT is a powerful tool for biomedical NER in non-English languages, even without domain-specific pretraining. While the model occasionally struggles with exact boundary offsets due to subword tokenization, its overall performance sets a new benchmark for Spanish clinical NLP. Future work may involve pretraining on Spanish clinical corpora or exploring zero-shot cross-lingual transfer from English biomedical datasets.
Summary updated