Using Automated Feature Optimisation to Create an Adaptable Relation Extraction System

Barry HaddowBioNLP 2008 · OpenReview

Abstract

Relation extraction (RE) in the biomedical domain has traditionally focused on protein-protein interactions (PPI). However, the diversity of biological relationships necessitates systems that can easily adapt to new relation types, such as tissue expression (TE). This paper proposes an adaptable RE system that utilizes a greedy search algorithm to automatically optimize a large pool of shallow syntactic features. This approach reduces the need for manual feature engineering while improving extraction accuracy across different biomedical corpora.

Introduction

In biomedical Natural Language Processing (BioNLP), the historical emphasis on PPI has left a gap in extracting other critical relationships, such as gene-tissue expression or the relationship between protein fragments and their parent proteins. Manually crafting feature sets for every new relationship type is labor-intensive and often inefficient, as excessive features can degrade model performance. This research addresses the challenge of building a relation extraction system that automatically adapts its feature set to new biomedical problems with minimal manual intervention.

Methodology

The system treats relation extraction as a supervised binary classification task. Given a pair of entities, the classifier determines if a valid relation exists. Rather than using deep dependency parsers, which can be brittle in the biomedical domain, the system relies on a library of shallow syntactic and contextual features, including n-grams, part-of-speech tags, and entity distances.

The core of the system is an automated feature optimization process using a wrapper method. A greedy search algorithm explores the feature space through iterative steps:

  1. Initialization: Start with a baseline feature set.
  2. Search: Apply operators to add or remove feature groups or adjust numerical thresholds (e.g., context window size).
  3. Evaluation: Train a Maximum Entropy (MaxEnt) classifier and evaluate performance.
  4. Iteration: If performance improves, the new set becomes the baseline for the next iteration.

Data and Evaluation

The system was evaluated using the ITI TXM corpora, which contain approximately 400 full-text biomedical research papers annotated by biologists. The corpora are divided into two main datasets:

  • PPI Corpus: Focuses on Protein-Protein Interactions, including mutants, fragments, and complexes.
  • TE Corpus: Focuses on Tissue Expression, linking genes or proteins to specific tissues.
Table 3: Relation types and entity types defined within the PPI and TE corpora.

The evaluation distinguishes between intra-sentential relations (within the same sentence) and inter-sentential relations (spanning multiple sentences).

Results and Discussion

The experimental results demonstrate that automated feature pruning and optimization outperform a "kitchen sink" approach. By logically grouping features, the greedy search algorithm effectively identifies specialized subsets that enhance classifier performance.

The Inter-sentential Challenge

A significant finding is the "Inter-sentential Wall." Extracting relations across sentence boundaries is considerably more difficult due to extreme class imbalance; only about 1% of inter-sentential entity pairs represent actual relations. Shallow features proved insufficient for capturing the complex logic, such as coreference, required to bridge these gaps.

Table 5: Performance metrics (BEP) for intra-sentential, inter-sentential, and combined relations across both corpora.

Key Performance Gains

  • Intra-sentential Improvements: Optimized feature sets consistently outperformed baselines. For PPI extraction, the MaxEnt classifier's Break-Even Point (BEP) saw significant gains, jumping from 39.7% (base features) to 49.1% (optimized features) in specific tests. Fragment relations also saw substantial improvements, with BEPs reaching approximately 69%.
  • Model Generalization: Features optimized for MaxEnt also improved performance when used with Support Vector Machines (SVMs). The SVM achieved a 49.9% BEP on PPI and 70.4% on TE fragments, indicating that the selected features are robust across different learning algorithms.

Conclusion

Automated feature optimization provides a viable path toward adaptable relation extraction systems. While the current greedy search approach can be limited by local optima, it successfully reduces the manual effort required for feature engineering. Future work will explore larger beam sizes for the search algorithm, integration of classifier hyperparameters into the optimization loop, and more granular sentence splitting to better handle complex inter-clausal and inter-sentential relations.

Summary updated

BibTeX
@inproceedings{haddow2008using,
  author = {Barry Haddow},
  title = {{Using Automated Feature Optimisation to Create an Adaptable Relation Extraction System}},
  booktitle = {BioNLP},
  year = {2008},
  url = {https://openreview.net/forum?id=poY1ZHWceFv},
}
Figure