A General-Purpose Annotation Model for Knowledge Discovery: Case Study in Spanish Clinical Text

Alejandro Piad-Morffis, Yoan Guitérrez, Suilan Estevez-Velarde, Rafael MuñozNAACL 2019

TL;DR

This paper introduces a novel, general-purpose annotation model designed to capture the semantic meaning of clinical text in Spanish. By simplifying semantic roles and focusing on core concepts and actions, the model aims to facilitate the creation of reusable knowledge discovery tools, particularly for domains with limited resources.

What Problem Does This Paper Address?

Knowledge discovery from text, especially in specialized domains like clinical medicine, often relies on manually annotated corpora. However, existing annotation models are typically task-specific, meaning they are tailored to a narrow set of objectives like named entity recognition or relation extraction. This leads to a fragmentation of effort and a lack of reusability. While some general-purpose semantic representations exist (e.g., AMR, PropBank), they can be overly complex, requiring fine-grained lexicons and extensive processing for knowledge extraction. This paper addresses the need for a more accessible, general-purpose annotation model that can capture essential semantic information from clinical text, making it easier to build machine learning models for knowledge discovery, especially in languages like Spanish where computational resources are less abundant than in English.

Methods

The authors propose a general-purpose annotation model that aims to represent the core semantic meaning of sentences, detaching it from strict grammatical structure. The model is built around a simplified Subject-Verb-Object (SVO) framework, defining two primary semantic roles: Concept (representing entities or ideas) and Action (representing processes or events). These are further contextualized by roles like subject and target.

Key innovations include:

  • Simplified Semantic Roles: Instead of numerous fine-grained roles, the model uses general roles like "agent" (subject) and "receiver" (target) for actions, making annotation more straightforward and amenable to machine learning.
  • Predicate Role: A "Predicate" role is introduced to build more complex concepts by combining or filtering simpler ones, allowing for representations like "people older than 60 years."
  • Taxonomic Relations: Explicitly defines relations like is-a, part-of, and same-as to capture hierarchical and equivalence relationships, which are crucial for knowledge bases.
  • Causation and Entailment: Includes causes and entails relations to represent logical connections between events, moving beyond simple actions.
  • Contextualization: Introduces in-time, in-place, and in-context relations to capture preconditions for assertions, adding nuance to the representation.
  • Boolean Attributes: Negated, uncertain, diminished, and emphasized attributes can be attached to concepts to capture implicit nuances without needing specific stopword annotations.

The model is designed to be language-agnostic, though the case study focuses on Spanish clinical text.

Datasets

The paper describes the application of the annotation model to a corpus of 1000 Spanish sentences from the clinical text domain. This corpus is being used for the eHealth-KD challenge. The authors mention that configuration files and annotated examples are available online, serving as a benchmark for future work. No new benchmark datasets are explicitly introduced, but the annotated corpus serves as a resource.

Key Insights and Ideas

  • Simplification for Scalability: The core insight is that by simplifying semantic roles and focusing on core semantic meaning rather than grammatical structure, it becomes feasible to build general-purpose annotation models that can be more easily scaled and applied to machine learning tasks, especially in resource-scarce languages.
  • Bridging General and Specific: The model attempts to strike a balance between general semantic representations (like AMR) and domain-specific annotations, aiming to capture enough detail for knowledge discovery without becoming prohibitively complex.
  • Explicit Taxonomic Relations: Explicitly annotating is-a and part-of relations is more efficient for knowledge base construction than relying on implicit representations through verbs like "to be."
  • Context is Crucial: The inclusion of contextual relations highlights that the meaning of an assertion often depends on its surrounding circumstances (time, place, etc.), which is vital for accurate knowledge representation.

Key Results

The paper presents the annotation model and its application to a Spanish clinical text corpus. While specific quantitative results (e.g., F1 scores for annotation agreement) are discussed in the context of the proposed evaluation methodology (Section 4.1), the paper primarily focuses on defining and illustrating the annotation model itself. The authors state that the model has been applied to annotate a corpus of 1000 Spanish sentences for the eHealth-KD challenge. They also provide configuration files and annotated examples for the BRAT tool, enabling others to use and evaluate the model.

Limitations and Future Work

Limitations:

  • Meaning Inference: The model does not explicitly specify the semantic meaning of each Concept and Action; this meaning must be inferred from the text of the annotated entities. The exact meaning of semantic roles also needs to be resolved in a later stage.
  • Ambiguity Resolution: While aiming for high annotator agreement, the model acknowledges that resolving ambiguities and ensuring consistency across annotators is an ongoing challenge.
  • Language Focus: Although designed to be language-agnostic, the primary case study is in Spanish clinical text.

Future Work:

  • Mapping to Knowledge Bases: The authors plan to focus on mapping the defined semantic roles and relations to common concepts and relations in existing knowledge bases and ontologies, such as DBpedia and UMLS.
  • Corpus Expansion: Continued annotation of clinical text is planned.
  • Cross-Lingual Extension: Extending the annotation model to other languages (e.g., English, French, Portuguese) and domains (e.g., news, scientific papers, encyclopedic articles) is a key future direction.
  • Evaluation: Further evaluation of the model's performance and its utility in machine learning tasks is anticipated.

Summary updated

BibTeX
@inproceedings{piadmorffis2019general,
  author = {Alejandro Piad-Morffis and Yoan Guitérrez and Suilan Estevez-Velarde and Rafael Muñoz},
  title = {{A General-Purpose Annotation Model for Knowledge Discovery: Case Study in Spanish Clinical Text}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure