A Maximum Entropy Approach to Information Extraction from Semi-Structured and Free Text

Hai Leong Chieu, Hwee Tou NgAAAI 2002 · OpenReview

Abstract

In this paper, we present a classification-based approach towards single-slot as well as multi-slot information extraction (IE). For single-slot IE, we worked on the domain of Seminar Announcements, where each document contains information on only one seminar. For multi-slot IE, we worked on the domain of Management Succession. For this domain, we restrict ourselves to extracting information sentence by sentence, in the same way as (Soderland 1999). Each sentence can contain information on several management succession events. By using a classification approach based on a maximum entropy framework, our system achieves higher accuracy than the best previously published results in both domains.

Introduction

Information Extraction (IE) involves automatically identifying specific data points—such as names, times, or job titles—within text to populate structured databases. Historically, IE has been divided into two categories: single-slot extraction (one record per document) and multi-slot extraction (multiple related records from free-flowing text).

In 2002, existing methodologies like Hidden Markov Models (HMMs) and rule-based pattern learners faced significant hurdles in integrating non-independent, overlapping features. For instance, a word being capitalized and its presence in a surname database are correlated features that traditional generative models struggle to handle simultaneously. This paper proposes a Maximum Entropy (ME) framework to address these challenges, treating IE as a series of discriminative classification decisions.

Methodology

The core of the proposed system is a Maximum Entropy classifier. Unlike generative models, ME models are discriminative, estimating the probability of a specific tag (outcome) given its surrounding context (history).

Single-Slot Extraction

For semi-structured data like seminar announcements, the system treats IE as a sequence labeling task. Words are classified into one of 17 categories (e.g., speaker-begin, location-continue). To maintain structural coherence, the Viterbi algorithm is employed to determine the most probable valid sequence of tags.

Multi-Slot Extraction

Extracting multiple events from free text requires a more sophisticated, four-stage pipeline:

  1. Text Filtering: An SVM classifier removes irrelevant documents.
  2. Candidate Selection: ME classifiers identify potential entities (e.g., Person-In, Person-Out, Position).
  3. Relation Classification: The system evaluates pairs of entities to determine if they belong to the same event.
  4. Template Building: Using graph theory, the system identifies "cliques" (complete subgraphs) to group related entities into final database records.
Flowchart of the multi-slot information extraction pipeline.

Experimental Results

The system was evaluated on two benchmarks: the CMU Seminar Announcements (485 documents) and the MUC-6 Management Succession dataset (news articles).

Seminar Announcements

The Maximum Entropy approach established new performance benchmarks. The "ME2" system achieved an 86.9% F-measure, surpassing established models such as (LP)², SNoW, and HMMs. Even without external name lists (the "ME1" variant), the system demonstrated superior performance, validating the robustness of the ME framework itself.

Performance comparison on the Seminar Announcements dataset.

Management Succession

On the more complex task of extracting job changes from news articles, the system achieved a 59.2% F-measure. This represents a significant improvement over the previous state-of-the-art, Soderland’s WHISK, which reached 55.5%.

Discussion and Conclusion

The success of this approach stems from its ability to leverage a "kitchen sink" of features—unigrams, bigrams, capitalization, and "zone" information—without the need for complex feature independence assumptions. By reframing multi-slot extraction as a relationship mapping problem rather than a monolithic pattern-matching task, the system gains significant flexibility.

However, the system remains dependent on initial syntactic tagging (via the BADGER system) and assumes independence between relations during the final template building phase. Despite these limitations, the results demonstrate that classification-based approaches offer a powerful, adaptive alternative for information extraction across diverse text formats.

Summary updated

Abstract

In this paper, we present a classification-based approach towards single-slot as well as multi-slot information extraction (IE). For single-slot IE, we worked on the domain of Seminar Announcements, where each document contains information on only one seminar. For multi-slot IE, we worked on the domain of Management Succession. For this domain, we restrict ourselves to extracting information sentence by sentence, in the same way as (Soderland 1999). Each sentence can contain information on several management succession events. By using a classification approach based on a maximum entropy framework, our system achieves higher accuracy than the best previously published results in both domains.
BibTeX
@inproceedings{chieu2002maximum,
  author = {Hai Leong Chieu and Hwee Tou Ng},
  title = {{A Maximum Entropy Approach to Information Extraction from Semi-Structured and Free Text}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2002},
  url = {https://openreview.net/forum?id=H1ZUq0g_-r},
}
Figure