VICOMTECH at MedProcNER 2023: Transformers-based Sequence-labelling and Cross-encoding for Entity Detection and Normalisation in Spanish Clinical Texts

Elena Zotova, Aitor García Pablos, Montse Cuadros, German RigauCLEF (Working Notes) 2023 · OpenReview

Introduction

Clinical documents contain critical information for healthcare research, yet their automated processing remains a significant challenge, particularly in non-English languages. In the Spanish medical domain, the MedProcNER 2023 shared task focused on the identification and standardization of "procedures"—ranging from surgeries and exams to therapies—mapping them to the SNOMED CT vocabulary. The Vicomtech NLP team addressed this by developing a robust pipeline for three sub-tasks: Recognition (NER), Normalization (mapping mentions to codes), and Indexing (document-level code extraction).

Methodology

The proposed system utilizes a multi-stage pipeline architecture to handle the complexities of medical Spanish, moving beyond simple keyword matching to sophisticated semantic analysis.

Named Entity Recognition (NER)

For Task 1, the team employed Transformer-based sequence labeling. To address the length constraints of standard Transformers in clinical reports, they utilized a sliding window technique. The best-performing model was roberta-bio-es, a RoBERTa variant pre-trained specifically on Spanish biomedical data, which proved superior to larger, general-purpose multilingual models.

Normalization (Task 2) represented the core technical contribution. The team implemented a two-stage semantic similarity approach:

  1. Bi-Encoders: Utilizing SapBERT-XLMR-large, the system generated vector embeddings for both text mentions and SNOMED CT definitions, enabling semantic retrieval of candidates even when lexical matches were absent.
  2. Cross-Encoders: A secondary model was trained to perform high-precision reranking by simultaneously processing a mention and a candidate code to predict a similarity score.
Architectural comparison between the Bi-Encoder for semantic search and the Cross-Encoder for high-precision reranking.

The pipeline was further enhanced through Dictionary Enrichment, expanding the official SNOMED CT dictionary with synonyms extracted from the training data and the Unified Medical Language System (UMLS).

Datasets and Resources

The models were trained and validated using the MedProcNER Corpus, which consists of manually annotated Spanish clinical reports. The normalization task relied on a massive hierarchy of clinical terms from SNOMED CT, enriched to approximately 245,000 entries through the inclusion of UMLS synonyms.

Distribution of documents, procedure mentions, and unique SNOMED CT codes in the MedProcNER corpus.

Results and Discussion

The Vicomtech team achieved top-tier performance across all tasks, notably securing the highest official F1-scores for both Normalization and Indexing.

  • Task 1 (NER): F1-score of 77.96%.
  • Task 2 (Normalization): F1-score of 57.07% (Rank 1).
  • Task 3 (Indexing): F1-score of 62.42% (Rank 1).

Key insights from the competition include the superiority of domain-specific pre-training and the significant performance boost provided by semantic search over traditional keyword-based algorithms like BM25. The team also noted a precision-recall trade-off, where models like xlm-roberta-large offered higher precision, which is often preferable in clinical applications where accuracy is paramount.

Limitations and Future Work

Despite these successes, several challenges remain. Composite codes—where a single mention maps to multiple SNOMED CT concepts—account for 2.6% of the data and remain difficult to resolve. Additionally, identical definitions for distinct codes require more advanced word-sense disambiguation. Finally, because the pipeline is sequential, error propagation from the initial NER stage remains a limiting factor for downstream normalization accuracy.

Summary updated

BibTeX
@inproceedings{zotova2023vicomtech,
  author = {Elena Zotova and Aitor García Pablos and Montse Cuadros and German Rigau},
  title = {{VICOMTECH at MedProcNER 2023: Transformers-based Sequence-labelling and Cross-encoding for Entity Detection and Normalisation in Spanish Clinical Texts}},
  booktitle = {CLEF (Working Notes)},
  year = {2023},
  url = {https://openreview.net/forum?id=W4cXvXXyiW},
}
Figure