Abstract
Automated identification of disabilities in medical documentation remains a challenging task due to the nuance required to distinguish them from general diseases and the complexity of negation. This paper presents a language-independent approach using a general-purpose sequence labeling system, ixa-pipe-nerc. By combining shallow local features with distributional features derived from large unlabelled medical corpora, the system achieves competitive results in both English and Spanish. Our findings emphasize the efficacy of semi-supervised learning and the critical importance of consistent data preprocessing in medical NLP.
Introduction
While the medical NLP community has developed robust tools for identifying diseases and symptoms, the annotation of disabilities has received significantly less attention. Distinguishing a disability from a general medical sign requires understanding context and negation—for instance, identifying that a patient is "without dementia." The DIANN 2018 shared task addressed this gap, challenging researchers to automate the detection of disabilities, negation triggers, and their scope in bilingual medical abstracts.
Methodology
The proposed system utilizes ixa-pipe-nerc, a component of the IXA pipes toolkit, treating the problem as a sequence labeling task solved via the Perceptron algorithm. The core of the methodology rests on three types of features:
- Local Shallow Features: These include word-level attributes such as capitalization, digit presence, n-grams, and immediate context.
- Distributional Features: To incorporate domain-specific knowledge without manual rule-crafting, the system uses semantic clusters (Brown, Clark, and Word2vec) induced from massive unlabelled medical datasets like PubMed (English) and BARR-E (Spanish). This allows the model to generalize across semantically related terms like "Alzheimer's" and "dementia."
- Heuristic Negation: A simplified post-processing rule was implemented where any negation trigger within a sentence is assumed to apply to all disabilities identified in that same sentence.
The researchers also compared the standard BIO tagging scheme with the more expressive BILOU (Beginning, Inside, Last, Outside, Unit) scheme to optimize entity boundary detection.
Experimental Setup
The system was evaluated on the DIANN 2018 dataset, which consists of annotated medical abstracts in English and Spanish. Initial results were significantly impacted by tokenization discrepancies between the system and the official test set. Consequently, the authors re-evaluated the system using a properly tokenized test set to provide a more accurate measure of the model's underlying capability.
Results and Discussion
The inclusion of distributional features significantly improved the system's recall, enabling it to identify disabilities not explicitly seen during training. The BILOU encoding scheme generally outperformed BIO across most metrics.
English Performance
On the tokenized English test set, the system achieved an F1-score of 74.4 for disability identification (DIS) and 70.7 for the joint task of identifying both disabilities and their negation status.
Spanish Performance
The Spanish models followed a similar trend, with the best configuration reaching an F1-score of 73.5 for DIS and 71.6 for the joint task.
While a domain-specific system like IxaMed showed higher recall in some categories, our general-purpose approach maintained high precision and demonstrated remarkable flexibility across languages. The "tokenization trap" encountered during the competition highlights that preprocessing consistency is as vital as model architecture in clinical NLP tasks.
Conclusion
This research demonstrates that a general-purpose sequence labeling tool, when augmented with semantic clusters from unlabelled data, can effectively tackle specialized medical annotation tasks. This language-independent approach offers a scalable alternative to hand-engineered systems. Future work will focus on refining the negation scope heuristic and utilizing more targeted unlabelled corpora to further boost recall.
Summary updated