Supervised Classification for Extracting Biomedical Events

Arzucan Özgür, Dragomir R. RadevNAACL 2009 · OpenReview

TL;DR

This paper presents a supervised machine learning approach to automatically extract complex biomedical events from text. By grouping event types based on participant complexity and using Support Vector Machines with carefully engineered linguistic features, they achieved competitive results on the BioNLP'09 Shared Task.

What Problem Does This Paper Address?

Extracting structured information about biological events from unstructured text is a crucial task for advancing biomedical research. Unlike simple relationships between entities (like protein-protein interactions), biomedical events can be intricate, involving multiple participants (proteins, other events) with specific roles (theme, cause) and a central "trigger" word that signifies the event. The BioNLP'09 Shared Task aimed to tackle this by identifying event triggers, their types, and their participants. This paper focuses on the challenge of accurately identifying these complex events, especially when they involve multiple participants or causal relationships.

Methods

The authors frame the problem as a supervised classification task. Their core innovation lies in grouping the nine event types into three general classes based on the number and types of participants:

  • Class 1: Single theme participant.
  • Class 2: One or more theme participants.
  • Class 3: Theme and/or cause participant.

They then train separate Support Vector Machine (SVM) models for each class. The classification is performed on candidate trigger/participant pairs.

Key features used to represent the context of these pairs include:

  • Lexical and Part-of-Speech (POS) features: The words themselves and their grammatical roles, recognizing that certain words are more likely to be event triggers.
  • Positional features: The relative position of the participant to the trigger ("before" or "after") and the distance (number of tokens) between them.
  • Dependency Relation Features: This is a significant feature. They extract the shortest path in the dependency parse tree between the trigger and participant. Crucially, they use only the dependency relation types (e.g., "nsubj", "acomp") on the path, not the words themselves, to mitigate data sparsity issues and improve generalization. They also explore an edit-distance based kernel on these dependency paths.

Datasets

The experiments were conducted using data from the GENIA corpus, specifically the training and development sets provided for the BioNLP'09 Shared Task on Event Extraction. The evaluation was performed on the shared task's test data.

Key Insights and Ideas

  • Hierarchical Classification: Grouping event types by participant complexity (single theme, multiple themes, theme/cause) simplifies the learning problem and allows for specialized models.
  • Dependency Path Features are Powerful: Using dependency parse information, particularly the sequence of relation types, effectively captures the semantic relationships between event triggers and participants.
  • Avoiding Data Sparsity: By using only dependency relation types instead of words on the paths, the model becomes more robust to variations in wording and less prone to overfitting.
  • Event Composition is Challenging: The performance on events with multiple participants (like binding and regulation) is lower, suggesting that accurately identifying individual participants is easier than correctly composing the full event structure.

Key Results

The paper reports two sets of results:

  1. Official Submission (with pre-processing error): 30.42% recall, 14.11% precision, and 19.28% F-measure.
  2. Corrected Results (after fixing pre-processing error): Significantly improved performance. Using a linear SVM with the described features, they achieved an F-measure of 39.79% on the shared task test data.
  3. Edit-Distance Kernel: Using an edit-distance based kernel on dependency relation paths yielded a similar performance, with an F-measure of 39.83%.

Performance varied by event type, with regulation events showing considerably lower scores, likely due to errors propagating from participant identification to event composition.

Limitations and Future Work

  • Error Propagation: Errors in identifying event participants negatively impact the overall event extraction performance, especially for complex events like regulations.
  • Event Composition: The authors suggest that improving the methods for composing events, particularly those with multiple participants, is a key area for future work.
  • Trigger Detection and Disambiguation: Enhancing the strategy for detecting and disambiguating triggers could further boost performance.
  • Data Sparsity: While mitigated by using relation types, data sparsity remains a general challenge in biomedical NLP.

The paper implicitly suggests that exploring more sophisticated feature engineering or alternative kernel methods could be beneficial.

Summary updated

Abstract

Arzucan Özgür, Dragomir Radev. Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task. 2009.
BibTeX
@inproceedings{ozgur2009supervised,
  author = {Arzucan Özgür and Dragomir R. Radev},
  title = {{Supervised Classification for Extracting Biomedical Events}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
  url = {https://openreview.net/forum?id=8ea3a26HkTr},
}
Figure