Abstract
The LaSIE-II system, developed by the University of Sheffield, represents a highly modular Information Extraction (IE) framework built upon the General Architecture for Text Engineering (GATE) platform. By integrating shallow natural language processing (NLP) techniques with deeper semantic analysis and discourse modeling, LaSIE-II successfully competed in all five tasks of the MUC-7 competition. This paper describes the system's architecture, its multi-stage processing pipeline, and its performance across various extraction challenges.
Introduction
In the late 1990s, Information Extraction research was divided between "shallow" systems—optimized for speed and pattern matching—and "deep" systems—aiming for comprehensive linguistic understanding. The University of Sheffield's LaSIE-II (Large Scale Information Extraction) system was designed to bridge this gap. The goal was to create a system capable of processing large-scale datasets, such as New York Times articles, while performing complex tasks like coreference resolution and scenario template filling.
The LaSIE-II Architecture
A cornerstone of the Sheffield approach was the use of the GATE (General Architecture for Text Engineering) platform. GATE provided a collaborative environment that allowed for the independent development, visualization, and debugging of individual modules.
The system is structured as a pipeline of nine "TIPSTER-compliant" modules, allowing researchers to swap or refine components without disrupting the entire system. This modularity proved essential for iterative development during the MUC-7 competition.
Processing Pipeline
The LaSIE-II pipeline operates through four primary stages:
- Lexical Preprocessing: This initial phase involves tokenization, gazetteer lookup (matching entities against predefined lists), part-of-speech tagging using the Brill Tagger, and morphological analysis.
- Parsing: A two-pass "buchart" parser, implemented in Prolog, first identifies Named Entities (NE) and then performs a phrasal parse. This results in a "quasi-logical form" (QLF) that represents the predicate-argument structure of each sentence.
- Discourse Interpretation: Acting as the system's core logic, this module translates the QLF into a semantic net (the Domain Model). It is responsible for coreference resolution—linking pronouns and noun phrases to established entities—and handling presuppositions to infer missing information based on a world model.
- Template Writing: The final stage traverses the discourse model to generate the specific data structures required for the competition's output formats.
Coreference and Error Analysis
One of the most significant challenges in MUC-7 was coreference resolution. LaSIE-II utilized the GATE Viewer to perform detailed error analysis on "Walkthrough Texts."
The ability to visualize the output of each module independently allowed the team to identify where "spurious" (incorrect) or "missing" links occurred. The paper notes that coreference failures often have a cascading effect, negatively impacting higher-level tasks like Template Relation (TR) and Scenario Template (ST) extraction.
Results
The system was evaluated across five distinct tasks in MUC-7. Performance was strongest in Named Entity recognition, while more complex tasks involving deep domain knowledge proved more difficult.
| Task | Recall | Precision | F-Measure |
|---|---|---|---|
| Named Entity (NE) | 87% | 94% | 90.41 |
| Template Element (TE) | 75% | 80% | 77.17 |
| Coreference (CO) | 56.1% | 68.8% | 61.8 |
| Template Relation (TR) | 41% | 82% | 54.70 |
| Scenario Template (ST) | 47% | 42% | 44.04 |
Key Insights
- Pragmatic Middle Ground: By using a conservative grammar that identifies linguistic "chunks," LaSIE-II maintained robustness. If a full parse failed, the system could still rely on these fragments to extract meaningful information.
- The Coreference Bottleneck: The results underscore that coreference is a critical dependency. Inaccurate pronoun resolution (e.g., failing to link "it" to a specific company) directly leads to failures in extracting complex event details.
- Knowledge Acquisition: The lower scores in Scenario Templates (44.04) highlight the difficulty of manually encoding the world knowledge required to understand complex events, such as satellite launches.
Conclusion and Future Work
LaSIE-II demonstrated the viability of a modular, GATE-based architecture for complex NLP tasks. Future work identified by the Sheffield team includes improving pronoun resolution through focus-based approaches and developing automated methods for acquiring domain-specific knowledge to reduce the reliance on hand-crafted rules.
Summary updated