Bootstrapping for Named Entity Tagging Using Concept-based Seeds

Cheng Niu, Wei Li, Jihong Ding, Rohini K. SrihariNAACL 2003

TL;DR

This paper introduces a novel bootstrapping method for Named Entity (NE) tagging that leverages concept-based seeds and successive learners. By using simple common nouns or pronouns as initial seeds, the approach trains a high-performance NE tagger that approaches supervised performance for key entity types like PERSON and LOCATION, without requiring extensive hand-labeled data.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a crucial task in information extraction, aiming to identify and classify proper names into categories like PERSON, ORGANIZATION, and LOCATION. Traditional supervised learning methods for NER require large amounts of manually annotated data, which is expensive and time-consuming to create. While unsupervised methods exist, they often face challenges like knowledge bottlenecks and error propagation, especially in iterative learning schemes. This paper addresses the need for an effective unsupervised NE tagging method that can achieve high performance with minimal initial human input, specifically by overcoming the data scarcity problem.

Methods

The core of this paper is a two-stage bootstrapping approach:

  1. Concept-based Seed Learning (Decision List): The process begins with a small set of "concept-based seeds." These are common nouns or pronouns that semantically relate to the target NE type (e.g., "he," "she," "man," "woman" for PERSON). The system uses a parser to identify structural patterns between these seeds and their surrounding words in a large, pre-parsed corpus. A Decision List (DL) learner is then trained on these patterns to extract parsing-based NE rules. This first learner is designed for high precision but may have limited recall.
  2. HMM-based Successive Learner: The rules learned by the Decision List are applied to a large raw corpus to automatically generate a partially tagged corpus. This corpus, consisting of the identified NEs and their neighboring words, is then used to train a high-performance Hidden Markov Model (HMM). This HMM acts as the final NE tagger.

A key aspect of this method is that it avoids iterative learning between the two learners, thus mitigating error propagation issues common in co-training approaches. The use of parsing structures is essential because concept-based seeds share structural, rather than purely lexical, similarities with NEs.

Datasets

  • Training Corpus: A large raw corpus of approximately 100,000 news articles, totaling around 88,000,000 words. This corpus is parsed and saved into a repository for efficient retrieval.
  • Testing Corpus: A blind testing corpus of 300,000 words containing 20,000 PERSON, LOCATION, and ORGANIZATION instances.
  • User-Defined NE Benchmark: A separate test for PRODUCT NEs, using a specific set of concept-based seeds for this category.

Key Insights and Ideas

  • Concept-based Seeds as a Low-Resource Starting Point: The idea that simple, common words (like pronouns and nouns) can serve as effective seeds for bootstrapping NE tagging is a significant insight. This drastically reduces the initial human effort required compared to providing lists of proper names or hand-crafted rules.
  • Successive Learning vs. Iterative Co-training: The paper highlights the advantage of a non-iterative, successive learning approach. By training a high-precision DL first and then using its output to train a high-recall HMM, the system avoids the error accumulation that can plague iterative co-training methods.
  • Leveraging Parsers for Structural Similarity: The reliance on parsing structures is crucial. It allows the system to capture the underlying grammatical relationships that link concept-based seeds to NEs, even when the lexical forms are different.
  • Bias Correction in Rule Evaluation: The authors address the potential bias in rule accuracy evaluation (e.g., towards PERSON due to high-frequency seeds like "he" and "she") by employing a modification scheme that adjusts instance counts based on the minimum number of instances across all NE types.

Key Results

  • Performance Approaching Supervised: The system achieves performance comparable to supervised NE taggers for PERSON (87.7% F-measure) and LOCATION (82.3% F-measure) NE types.
  • ORGANIZATION Performance Gap: The performance for ORGANIZATION (52.7% F-measure) is significantly lower. The authors attribute this to the diverse sub-types of organizations that cannot be adequately represented by the limited set of concept-based seeds used in the experiment.
  • User-Defined NE Tagging: The approach demonstrates flexibility by successfully tagging user-defined NE types, achieving a 69.80% F-measure for PRODUCT NEs.
  • Degradation from Supervised: Table 1 quantifies the performance degradation from supervised methods, showing a 5.0% drop for PER, 6.0% for LOC, and a substantial 34.0% for ORG.

Limitations and Future Work

  • ORGANIZATION Performance: The primary limitation identified is the poor performance on ORGANIZATION entities. This is directly linked to the insufficient coverage of concept-based seeds for the wide variety of organization types.
  • Seed Selection: The effectiveness of the system is dependent on the quality and coverage of the initial concept-based seeds.
  • Future Work: The authors suggest that expanding the set of concept-based seeds, particularly for complex entity types like ORGANIZATION, would improve performance. Further research could also explore more sophisticated parsing techniques or alternative methods for bridging the gap between concept seeds and NEs.

Summary updated

BibTeX
@inproceedings{niu2003bootstrapping,
  author = {Cheng Niu and Wei Li and Jihong Ding and Rohini K. Srihari},
  title = {{Bootstrapping for Named Entity Tagging Using Concept-based Seeds}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
}
Figure