A tagger/lemmatiser for Dutch medical language

Peter SpynsCOLING 1996

TL;DR

This paper introduces a novel tagger/lemmatiser for Dutch medical language, designed to process free-text patient reports. It combines a comprehensive lexicon with a morphological recognizer and an expert system for disambiguation, offering a robust solution for analyzing complex medical terminology.

What Problem Does This Paper Address?

Medical patient reports are a rich source of information, but their free-text nature makes them challenging for computers to process. While numerical data is easily managed, extracting meaningful insights from unstructured text requires sophisticated Natural Language Processing (NLP) tools. Existing NLP systems for medical text are scarce, especially for Dutch. This paper addresses the need for a specialized tool that can accurately tag and lemmatize Dutch medical vocabulary, enabling deeper analysis of patient reports and facilitating corpus studies on medical language.

Methods

The core of the system is a tagger/lemmatiser (T/L) that operates as a "generate-and-test" engine.

  • Lexical Front-end: A full-form dictionary is used for efficient retrieval. For unknown words, a morphological recognizer identifies potential linguistic characteristics, including canonical forms, using heuristics that combine morphological and non-morphological knowledge.
  • Disambiguation Module: An "expert-like system" disambiguates morphological analyses. It considers the entire sentence context (not just immediate neighbors) and uses a rule base implemented as Prolog clauses. This system aims to reduce potentially valid analyses to those effectively applicable in the given context.
  • Data Structures: Key data structures include a lexical database and feature bundles implemented as Directed Acyclic Graphs (DAGs). DAGs are used for representing linguistic restrictions and filtering information.

Datasets

  • Cardiology Corpus: A learning set of 1314 tokens from cardiology department reports was used to minimize errors from unknown vocabulary.
  • Neurosurgical Corpus: A larger test set of 3167 tokens from 35 neurosurgical reports was used to evaluate robustness in a new domain.

Key Insights and Ideas

  • Full-form dictionary for speed: Using a full-form dictionary significantly speeds up the tagging process compared to canonical form computation.
  • Intelligent word recognizer for unknown words: Since a complete dictionary is impractical, an intelligent recognizer is crucial for handling the vast and evolving medical vocabulary.
  • Contextual disambiguation is key: Simply identifying possible analyses isn't enough; an expert system that leverages broader sentence context is necessary to accurately disambiguate medical terms.
  • DAGs for flexible data representation: Directed Acyclic Graphs provide a powerful and flexible way to represent linguistic features and impose restrictions, making the system adaptable.
  • Trade-off between overkill and undershoot: Evaluating NLP systems involves balancing the risk of discarding correct analyses (overkill) with the risk of keeping incorrect ones (undershoot).

Key Results

The paper presents evaluation results using two tagsets: an extensive one (tagset1) for full sentence analysis and a reduced one (tagset2) for simpler tagging.

  • Tagset2 shows improved results: The reduced tagset (tagset2) consistently yielded higher accuracy scores across both the cardiology and neurosurgical datasets.
  • Cardiology vs. Neurosurgery: The T/L performed better on the cardiology dataset (82.42% with tagset1, 91.32% with tagset2) compared to the neurosurgical dataset (73.63% with tagset1, 83.04% with tagset2). This difference is attributed to a higher percentage of unknown vocabulary in the neurosurgical domain.
  • Comparison to other taggers: The authors consider their results with tagset2 to be competitive with other taggers, especially given the complexity of medical Dutch and the use of a more extensive tagset than many general-purpose taggers.

Limitations and Future Work

  • Limited handling of complex wordforms and idiomatic expressions: The current database primarily contains simple wordforms, with complex wordforms and idiomatic expressions not yet fully handled.
  • Ambiguity resolution: Some ambiguities could not be resolved without semantic or pragmatic knowledge, leading to "double analysis" cases.
  • Potential for improvement: The authors believe the T/L has significant potential for further improvement, particularly in handling more complex linguistic phenomena and expanding its knowledge base. Future work could focus on incorporating more sophisticated semantic and pragmatic reasoning to further reduce ambiguity.

Summary updated

BibTeX
@inproceedings{spyns1996tagger,
  author = {Peter Spyns},
  title = {{A tagger/lemmatiser for Dutch medical language}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1996},
}
Figure