Abstract
In this paper, we present a classification-based approach towards single-slot as well as multi-slot information extraction (IE). For single-slot IE, we worked on the domain of Seminar Announcements, where each document contains information on only one seminar. For multi-slot IE, we worked on the domain of Management Succession. For this domain, we restrict ourselves to extracting information sentence by sentence, in the same way as (Soderland 1999). Each sentence can contain information on several management succession events. By using a classification approach based on a maximum entropy framework, our system achieves higher accuracy than the best previously published results in both domains.
Introduction
Information Extraction (IE) involves automatically identifying specific data points—such as names, times, or job titles—within text to populate structured databases. Historically, IE has been divided into two categories: single-slot extraction (one record per document) and multi-slot extraction (multiple related records from free-flowing text).
In 2002, existing methodologies like Hidden Markov Models (HMMs) and rule-based pattern learners faced significant hurdles in integrating non-independent, overlapping features. For instance, a word being capitalized and its presence in a surname database are correlated features that traditional generative models struggle to handle simultaneously. This paper proposes a Maximum Entropy (ME) framework to address these challenges, treating IE as a series of discriminative classification decisions.
Methodology
The core of the proposed system is a Maximum Entropy classifier. Unlike generative models, ME models are discriminative, estimating the probability of a specific tag (outcome) given its surrounding context (history).
Single-Slot Extraction
For semi-structured data like seminar announcements, the system treats IE as a sequence labeling task. Words are classified into one of 17 categories (e.g., speaker-begin, location-continue). To maintain structural coherence, the Viterbi algorithm is employed to determine the most probable valid sequence of tags.
Multi-Slot Extraction
Extracting multiple events from free text requires a more sophisticated, four-stage pipeline:
- Text Filtering: An SVM classifier removes irrelevant documents.
- Candidate Selection: ME classifiers identify potential entities (e.g., Person-In, Person-Out, Position).
- Relation Classification: The system evaluates pairs of entities to determine if they belong to the same event.
- Template Building: Using graph theory, the system identifies "cliques" (complete subgraphs) to group related entities into final database records.
Experimental Results
The system was evaluated on two benchmarks: the CMU Seminar Announcements (485 documents) and the MUC-6 Management Succession dataset (news articles).
Seminar Announcements
The Maximum Entropy approach established new performance benchmarks. The "ME2" system achieved an 86.9% F-measure, surpassing established models such as (LP)², SNoW, and HMMs. Even without external name lists (the "ME1" variant), the system demonstrated superior performance, validating the robustness of the ME framework itself.
Management Succession
On the more complex task of extracting job changes from news articles, the system achieved a 59.2% F-measure. This represents a significant improvement over the previous state-of-the-art, Soderland’s WHISK, which reached 55.5%.
Discussion and Conclusion
The success of this approach stems from its ability to leverage a "kitchen sink" of features—unigrams, bigrams, capitalization, and "zone" information—without the need for complex feature independence assumptions. By reframing multi-slot extraction as a relationship mapping problem rather than a monolithic pattern-matching task, the system gains significant flexibility.
However, the system remains dependent on initial syntactic tagging (via the BADGER system) and assumes independence between relations during the final template building phase. Despite these limitations, the results demonstrate that classification-based approaches offer a powerful, adaptive alternative for information extraction across diverse text formats.
Summary updated