Abstract
The DEFT 2019 (Défi Fouille de Textes) challenge presented significant hurdles in French medical natural language processing (NLP), specifically regarding document matching and information extraction from clinical cases. This paper describes the systems developed by Qwant Research to address these tasks. For document matching, a Query Likelihood Model was employed, treating clinical cases as queries against medical discussion documents. For information extraction, a hybrid approach was used, comparing a neural Bi-LSTM-CNN-CRF model with data augmentation against a rule-based linguistic system. Our findings emphasize that while deep learning models are powerful, domain-specific preprocessing—particularly the expansion of medical abbreviations—and synthetic data generation are paramount in low-resource clinical domains.
Introduction
The medical field generates vast amounts of unstructured text, yet extracting actionable value remains a complex challenge. Clinical documents are characterized by extreme domain specificity, heavy reliance on jargon, and a high frequency of ambiguous abbreviations. For instance, the acronym "IVG" in French can refer to either "insuffisance ventriculaire gauche" (left ventricular failure) or "interruption volontaire de grossesse" (voluntary termination of pregnancy), depending entirely on the clinical context.
Furthermore, French-language medical datasets are significantly smaller than their English counterparts, creating a "low-resource" environment that hinders the training of data-intensive deep learning models. This paper addresses these hurdles within the framework of the DEFT 2019 challenge, focusing on two primary objectives: linking brief clinical case descriptions to detailed medical discussions (Semantic Matching) and identifying key patient demographics and outcomes from raw text (Information Extraction).
Methodology
Document Matching (Task 2)
The team approached the matching task using a Query Likelihood Model. In this framework, each medical discussion was treated as a "document" and each clinical case as a "query." The goal was to determine which document was most likely to have generated the query.
Unigram, bigram, and trigram language models were constructed for each discussion. The system then calculated the perplexity of the clinical cases against these models; a lower perplexity score indicated a higher likelihood of a match. The researchers tested both exclusive (one-to-one) and non-exclusive matching strategies, ultimately finding that training models on the longer, linguistically richer discussions to predict the cases yielded superior results compared to the inverse approach.
Information Extraction (Task 3)
To extract patient age, gender, reason for admission (origin), and clinical outcome, the authors compared two distinct philosophies:
- The Neural Approach: A Bi-LSTM-CNN-CRF architecture was implemented for sequence labeling. To mitigate the small size of the training set, the team employed data augmentation, generating 2,000 synthetic clinical introductions using a context-free grammar. Word embeddings were initialized using FastText, trained on a large aggregate of French medical Wikipedia and the EMEA (European Medicines Agency) corpus.
- The Linguistic Approach: This method relied on dependency parsing and hand-crafted lexico-syntactic patterns. For example, the system identified the keyword "an" (year) to locate and extract age-related modifiers.
Data and Preprocessing
The primary dataset was the DEFT 2019 Corpus, consisting of French clinical cases from the CAS corpus (290 training cases, 214 test cases). This was supplemented by the EMEA corpus and French Wikipedia articles under the "Medicine" portal for pre-training.
The most critical preprocessing step identified was "désabrègement" (abbreviation expansion). The authors developed a custom lexicon of 227 common medical abbreviations. Expanding these terms before modeling led to a 19-point increase in precision for the matching task, highlighting the necessity of domain-specific normalization.
Results and Discussion
Task 2 Performance
The best-performing system for document matching achieved a precision of 84.11%. Interestingly, a simple 1-gram (bag-of-words) model outperformed more complex bigram or trigram models on the final test set. This suggests that the clinical cases were too brief for higher-order models to learn meaningful sequences without overfitting, whereas the unigram approach remained robust.
Task 3 Performance
The neural approach excelled at identifying structured demographic data, reaching F1-scores of 0.937 for age and 0.944 for gender. However, extracting the clinical outcome (issue) proved more difficult, yielding a top F1-score of 0.59. This was achieved by a hybrid system using linguistic rules for specific outcomes like "death" and neural classification for others.
Identifying the "Origin" (reason for admission) remained the most significant challenge. Unlike age and gender, which typically appear in the opening sentence, the reason for admission can be located anywhere in the text and lacks a standard recurring linguistic pattern.
Conclusion
The Qwant Research participation in DEFT 2019 demonstrates that in specialized, low-resource domains like French clinical medicine, the choice of preprocessing and data strategy is as vital as the model architecture. Synthetic data generation proved to be an effective bridge for deep learning models, while abbreviation expansion served as a fundamental requirement for semantic matching. Future work will focus on ensemble methods for matching and refining the extraction of clinical outcomes by targeting specific linguistic markers at the conclusion of medical reports.
Summary updated