Abstract: Erik Velldal, Lilja Øvrelid, Stephan Oepen. Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task.
2010.
Introduction
In scientific literature, distinguishing between established facts and speculative hypotheses is a fundamental requirement for automated information extraction. This process, known as hedge resolution, involves two primary challenges: identifying the specific words or phrases that signal speculation (cue detection) and determining the exact span of the sentence affected by that uncertainty (scope resolution). For instance, in the sentence "It is possible that the protein binds to DNA," the cue is "possible," and its scope is "that the protein binds to DNA."
This paper details a high-performing hybrid system developed for the CoNLL-2010 Shared Task, which focuses on biomedical text. The authors propose a pipeline that combines machine learning for cue identification with a rule-based approach for scope resolution, leveraging deep linguistic features derived from dependency parsing.
Hybrid System Architecture
The authors employed a two-level hybrid architecture designed to maximize the strengths of both statistical and symbolic methods.
Cue Identification via Maximum Entropy
For the first stage, a binary Maximum Entropy (MaxEnt) classifier was built to label individual tokens as "cue" or "non-cue." While simple n-gram baselines proved surprisingly effective, the model was significantly enhanced through parser stacking. This technique involved a data-driven parser (MaltParser) that incorporated predictions from a large-scale, hand-crafted Lexical Functional Grammar (LFG). This provided the classifier with "deep" linguistic features, such as subcategorization frames and grammatical voice, which are often missing from shallower models.
Rule-Based Scope Resolution
Rather than training a second machine learning model for scope, the authors developed a set of heuristic rules that operate on the dependency trees generated during the parsing stage. These rules define the boundaries of uncertainty based on the grammatical relationship between the cue and other sentence elements. For example, if a cue is a modal verb like may, the rules identify its subject and lexical verb dependents to define the scope.
Data and Preprocessing
The system was trained and evaluated on the BioScope corpus, which includes over 14,000 sentences from biomedical abstracts and full-text articles. A critical component of the pipeline was a custom finite-state tokenizer. The authors found that standard tools often struggled with the specialized nomenclature of biomedical text (e.g., chemical names like "Ca(2+)" or LaTeX-style dashes). By improving tokenization accuracy, they ensured that downstream PoS tagging and parsing were more reliable.
Results and Key Insights
The system was remarkably successful in the CoNLL-2010 competition, achieving the highest average rank across both sub-tasks. The official results are summarized below:
Several key insights emerged from the development and evaluation of the system:
- Lexical Dominance: For cue identification, simple lexical information (n-grams) is the most powerful predictor. While deep dependency features improved performance, the gains showed diminishing returns once a robust lexical model was established.
- The Utility of "Deep" Features: Features like grammatical voice (active vs. passive) and subcategorization were vital for the scope resolution rules. Knowing a verb is passive, for instance, allows the system to correctly include the subject within the scope of speculation.
- Domain Sensitivity: The models were highly sensitive to text type. Performance on biomedical abstracts did not generalize perfectly to clinical reports, highlighting the need for domain-specific training data.
Conclusion and Future Work
The hybrid approach of combining MaxEnt classification with dependency-based rules proved to be a robust solution for hedge resolution in scientific text. While the system achieved top-tier results, the authors noted a performance drop on held-out test data, suggesting potential overfitting to training constructions. Future work may involve integrating constituent-based rules (phrase structure) or employing a statistical ranker to choose between multiple scope candidates, potentially further refining the precision of the scope boundaries.
Summary updated