Semantic role labeling for protein transport predicates

Steven Bethard, Zhiyong Lu, James H. Martin, Lawrence HunterBMC Bioinformatics 2008 · DOI · OpenReview

TL;DR

This paper tackles the challenge of automatically identifying semantic roles (like AGENT, PATIENT, ORIGIN, DESTINATION) for protein transport events described in biomedical text. They developed a novel word-chunking approach, enhanced with biomedical-specific features, that significantly outperforms previous methods, even when using automatically identified protein names.

What Problem Does This Paper Address?

Semantic Role Labeling (SRL) is a powerful Natural Language Processing (NLP) technique that aims to extract structured meaning from text by identifying the "who did what to whom, where, and when." While SRL has seen success in general text domains like news articles, applying it to specialized domains like biomedicine presents unique challenges. Biomedical text has a different style, often uses nominal predicates (nouns describing actions) instead of verbs, and features specialized vocabulary and sentence structures. Crucially, existing SRL resources like FrameNet and PropBank lack coverage of many biomedical predicates, particularly those related to protein transport (e.g., "translocation," "export"). This gap hinders automated information extraction from biomedical literature, which is essential for advancing biological research.

Methods

The authors adapted a word-chunking paradigm for SRL, moving away from traditional syntactic constituent-based approaches. This method treats SRL as a word classification problem, labeling each word as being at the beginning (B), inside (I), or outside (O) of a semantic role. This approach is beneficial because:

  • Avoids computationally expensive syntactic parsing: Biomedical text can be complex, making accurate syntactic parsing difficult.
  • Handles role boundaries: Protein transport roles often don't align neatly with syntactic phrase boundaries, which word-chunking can accommodate better.

They trained Support Vector Machine (SVM) classifiers using three sets of features:

  1. Word-Chunking Features: Basic features like the word itself, its part-of-speech (POS) tag, and its context.
  2. Phrase-Chunking Features: Features derived from a phrase-chunking model, adapted to the word-chunking framework.
  3. Protein-Transport Features: Novel features specifically designed for the biomedical domain. These include:
  4. Orthographic features (capitalization, numeric class, punctuation) to better identify protein names.
  5. Biomedical POS tags (from MedPost) which are more accurate for biological text.
  6. Protein BIO chunk labels (identifying protein entities).
  7. Features related to conjunctions and coreference.
  8. Features capturing relationships between predicates and potential arguments, including distance and phrasal paths.

Datasets

The primary dataset used is a collection of 837 GeneRIFs (Gene Reference Into Function) describing protein transport events. These GeneRIFs were manually curated by domain experts and annotated with AGENT, PATIENT, ORIGIN, and DESTINATION roles for protein transport predicates. This dataset is novel in its focus on protein transport and its inclusion of both verbal and nominal predicates.

Key Insights and Ideas

  • Biomedical text is fundamentally different: The study highlights that standard SRL models trained on newswire text perform poorly on biomedical data due to differences in predicate types (more nominals) and sentence structure.
  • Word-chunking is a viable alternative: For domains where syntactic parsing is unreliable or roles don't align with syntactic structures, word-chunking offers a robust alternative.
  • Domain-specific features are crucial: Simply adapting existing SRL models isn't enough. Incorporating features tailored to the biomedical domain (like protein name identification and specialized POS tags) is key to achieving high performance.
  • Role identification is easier than boundary detection: The models achieved very high labeled accuracy (correctly identifying the role type once boundaries were found), suggesting that distinguishing between AGENT, PATIENT, ORIGIN, and DESTINATION is relatively straightforward given the domain constraints. The main challenge was accurately identifying the boundaries of these roles.
  • Automatic protein identification is improving: The performance drop when using automatically identified protein names (from ABNER) was relatively small, indicating that the models are becoming more robust to imperfect entity recognition, a critical step for real-world applications.

Key Results

The final "Protein-Transport" model, incorporating all feature sets, achieved:

  • 87.6% precision and 79.0% recall when using manually annotated protein boundaries.
  • 87.0% precision and 74.5% recall when using automatically identified protein boundaries.

These results represent a significant improvement over baseline models (Word-Chunking and Phrase-Chunking), demonstrating the effectiveness of the novel features. The F-measure reached 83.1%, a strong performance for this challenging task.

Limitations and Future Work

  • AGENT role sparsity: The AGENT role was significantly underrepresented in the corpus, making it difficult to train robust models for this role. This is attributed to the nature of protein transport descriptions and the annotation style.
  • Coreference resolution: The models struggled with roles tied to predicates through distant coreference chains, suggesting a need for better coreference resolution features.
  • Syntactic parsing limitations: Errors related to boundary mismatches and the need for full syntactic parses were observed, indicating that integrating features from more advanced syntactic parsers (trained on biomedical text) could be beneficial.
  • Domain specificity: While the Protein-Transport features improved performance, the authors suggest that even more specialized features might be needed for specific sub-domains or predicate types within biomedicine.

Future work could focus on incorporating coreference resolution systems, improving biomedical text processing components (like syntactic chunkers and protein identifiers), and exploring features extracted from full syntactic parses. The authors also envision exploring synergies with ontology-driven systems like OpenDMAP.

Summary updated

Abstract

Background Automatic semantic role labeling (SRL) is a natural language processing (NLP) technique that maps sentences to semantic representations. This technique has been widely studied in the recent years, but mostly with data in newswire domains. Here, we report on a SRL model for identifying the semantic roles of biomedical predicates describing protein transport in GeneRIFs – manually curated sentences focusing on gene functions. To avoid the computational cost of syntactic parsing, and because the boundaries of our protein transport roles often did not match up with syntactic phrase boundaries, we approached this problem with a word-chunking paradigm and trained support vector machine classifiers to classify words as being at the beginning, inside or outside of a protein transport role. Results We collected a set of 837 GeneRIFs describing movements of proteins between cellular components, whose predicates were annotated for the semantic roles AGENT, PATIENT, ORIGIN and DESTINATION. We trained these models with the features of previous word-chunking models, features adapted from phrase-chunking models, and features derived from an analysis of our data. Our models were able to label protein transport semantic roles with 87.6% precision and 79.0% recall when using manually annotated protein boundaries, and 87.0% precision and 74.5% recall when using automatically identified ones. Conclusion We successfully adapted the word-chunking classification paradigm to semantic role labeling, applying it to a new domain with predicates completely absent from any previous studies. By combining the traditional word and phrasal role labeling features with biomedical features like protein boundaries and MEDPOST part of speech tags, we were able to address the challenges posed by the new domain data and subsequently build robust models that achieved F-measures as high as 83.1. This system for extracting protein transport information from GeneRIFs performs well even with proteins identified automatically, and is therefore more robust than the rule-based methods previously used to extract protein transport roles.
BibTeX
@article{bethard2008semantic,
  author = {Steven Bethard and Zhiyong Lu and James H. Martin and Lawrence Hunter},
  title = {{Semantic role labeling for protein transport predicates}},
  journal = {BMC Bioinformatics},
  year = {2008},
  doi = {10.1186/1471-2105-9-277},
}
Figure