Abstract
This paper presents a minimally supervised, language-independent approach to Named Entity Recognition (NER) that requires only a small "seed" list of names and raw, unannotated text. By iteratively bootstrapping morphological patterns and contextual clues through hierarchical trie structures, the system achieves competitive performance across diverse languages without the need for specialized parsers or hand-coded rules.
Introduction
Named Entity Recognition—the task of identifying and categorizing names of people, places, and organizations—is a cornerstone of natural language processing. Traditionally, building an NER system required either massive amounts of manually labeled data or deep linguistic knowledge specific to a single language. This creates a bottleneck for low-resource languages or niche domains where such data and expertise are unavailable.
Cucerzan and Yarowsky address this by proposing a "maximally language-independent" system. Their model can be deployed for a new language with approximately 15 minutes of human effort to create a short list of example names, paired with a corpus of raw text.
Methodology
The core of the system is an Expectation-Maximization (EM)-style bootstrapping algorithm that relies on two relatively independent sources of evidence:
- Morphological Evidence (Word-Internal): The system analyzes prefixes and suffixes. For example, in Romanian, the suffix -escu is a strong indicator of a last name.
- Contextual Evidence (Word-External): The system examines the words to the left and right. Phrases like "Mr." or "mayor of" serve as strong indicators of a person or a title.
The Trie Structure
To store and process this information efficiently, the authors utilize character-based tries. They maintain four distinct tries: prefix, suffix, left-context, and right-context.
- Hierarchical Smoothing: Because many words or contexts appear only once, the system "backs off" up the trie. If it hasn't seen the full word "Mariana," it looks at the prefix "Marian-," then "Maria-," to estimate the probability of a name class.
- The "Questionable" Class: Instead of forcing a guess when data is sparse, the authors introduce a "questionable" category. This explicitly represents uncertainty, allowing the model to wait for more evidence before committing to a classification.
The Bootstrapping Loop
The algorithm begins with a small seed list (e.g., 100 names). It locates these in the raw text, learns the contexts in which they appear, uses those contexts to identify new names, and then analyzes the morphology of those new names to discover further patterns. This cycle continues until the model converges.
Experimental Setup and Datasets
The authors evaluated their system on five diverse languages to demonstrate its versatility: Romanian, English, Greek, Turkish, and Hindi. The training data consisted of small seed lists (150–300 words) and relatively small corpora of raw text (ranging from ~5,000 to ~18,000 words).
Key Insights
- Morphology as a Strong Starter: Morphological clues are often more informative than context when working with very small datasets. Character-level patterns are denser and more frequent than specific word-level contexts.
- One Sense Per Discourse: The model leverages the heuristic that if a word is classified as a "Place" in one part of a document, it is highly likely to remain a "Place" throughout that same document.
- Handling Non-Cased Languages: The system was tested on Hindi, which lacks capitalization. While performance was lower than in cased languages, the system still showed significant improvement over the baseline using only morphological and contextual patterns.
Results
On Romanian text, the system achieved an F-measure of ~70.5% for the complex task of distinguishing between first names, last names, and places. When simplified to a binary "Person vs. Place" distinction, accuracy rose to 91.76%. The authors also observed that performance increases roughly logarithmically with the size of the raw text and the seed list, suggesting that scaling with unannotated web data is a viable path to higher accuracy.
Limitations and Future Work
The system performs significantly better on languages with explicit case marking. In languages like Hindi, the "non-entity" prior probability is so strong that the model requires more data to overcome it. Additionally, the current model assumes words are already segmented. For languages like Chinese or Japanese, a language-independent segmentation phase would be a necessary precursor. Future work may involve "Active Learning," where the system identifies high-uncertainty examples for human labeling to maximize the impact of manual effort.
Summary updated