Introduction
Clinical documents contain critical information for healthcare research, yet their automated processing remains a significant challenge, particularly in non-English languages. In the Spanish medical domain, the MedProcNER 2023 shared task focused on the identification and standardization of "procedures"—ranging from surgeries and exams to therapies—mapping them to the SNOMED CT vocabulary. The Vicomtech NLP team addressed this by developing a robust pipeline for three sub-tasks: Recognition (NER), Normalization (mapping mentions to codes), and Indexing (document-level code extraction).
Methodology
The proposed system utilizes a multi-stage pipeline architecture to handle the complexities of medical Spanish, moving beyond simple keyword matching to sophisticated semantic analysis.
Named Entity Recognition (NER)
For Task 1, the team employed Transformer-based sequence labeling. To address the length constraints of standard Transformers in clinical reports, they utilized a sliding window technique. The best-performing model was roberta-bio-es, a RoBERTa variant pre-trained specifically on Spanish biomedical data, which proved superior to larger, general-purpose multilingual models.
Normalization and Semantic Search
Normalization (Task 2) represented the core technical contribution. The team implemented a two-stage semantic similarity approach:
- Bi-Encoders: Utilizing
SapBERT-XLMR-large, the system generated vector embeddings for both text mentions and SNOMED CT definitions, enabling semantic retrieval of candidates even when lexical matches were absent. - Cross-Encoders: A secondary model was trained to perform high-precision reranking by simultaneously processing a mention and a candidate code to predict a similarity score.
The pipeline was further enhanced through Dictionary Enrichment, expanding the official SNOMED CT dictionary with synonyms extracted from the training data and the Unified Medical Language System (UMLS).
Datasets and Resources
The models were trained and validated using the MedProcNER Corpus, which consists of manually annotated Spanish clinical reports. The normalization task relied on a massive hierarchy of clinical terms from SNOMED CT, enriched to approximately 245,000 entries through the inclusion of UMLS synonyms.
Results and Discussion
The Vicomtech team achieved top-tier performance across all tasks, notably securing the highest official F1-scores for both Normalization and Indexing.
- Task 1 (NER): F1-score of 77.96%.
- Task 2 (Normalization): F1-score of 57.07% (Rank 1).
- Task 3 (Indexing): F1-score of 62.42% (Rank 1).
Key insights from the competition include the superiority of domain-specific pre-training and the significant performance boost provided by semantic search over traditional keyword-based algorithms like BM25. The team also noted a precision-recall trade-off, where models like xlm-roberta-large offered higher precision, which is often preferable in clinical applications where accuracy is paramount.
Limitations and Future Work
Despite these successes, several challenges remain. Composite codes—where a single mention maps to multiple SNOMED CT concepts—account for 2.6% of the data and remain difficult to resolve. Additionally, identical definitions for distinct codes require more advanced word-sense disambiguation. Finally, because the pipeline is sequential, error propagation from the initial NER stage remains a limiting factor for downstream normalization accuracy.
Summary updated