Automatic Semantic Tagging of Unknown Proper Names

Alessandro Cucchiarelli, Danilo Luzi, Paola VelardiACL 1998

TL;DR

This paper introduces a novel context-based method for automatically assigning semantic types (like Person, Organization, Location) to proper names that are not found in existing dictionaries (gazetteers). By analyzing syntactically and semantically similar contexts where known proper names appear, the system can infer the type of unknown names, significantly improving coverage beyond traditional gazetteer-based approaches.

What Problem Does This Paper Address?

Named Entity Recognition (NER), the task of identifying and classifying proper names in text, is crucial for information extraction. While current systems achieve high performance on newswire text, this is largely due to extensive, hand-crafted gazetteers and rules. However, these resources are often unavailable for other languages or specialized domains, and proper names are an "open class" (new ones are constantly created). This leaves a significant gap: a large percentage of detected proper names remain "unknown" and cannot be semantically tagged, limiting the usefulness of NER systems in real-world applications. The paper specifically addresses the challenge of semantically tagging these unknown proper names, particularly in Italian, where gazetteers are less comprehensive.

Methods

The proposed method tackles the problem of unknown proper name (PN) tagging by leveraging contextual evidence. It combines symbolic and statistical approaches, operating on a "learning corpus" (raw text from the target domain), a shallow syntactic parser, a "seed" gazetteer (a small initial list of known PNs and their types), and a synonym dictionary.

The core idea is to analyze "elementary syntactic links" (esls) extracted by the parser. An esl describes a relationship between words, like "subject-object" or "noun-preposition-noun." When an unknown PN (PN_U) is encountered, the system looks for esls where PN_U is an argument. It then identifies similar contexts where known PNs appear with the same syntactic relationship.

Two sets of similar contexts are constructed:

  • ESLA: Contexts where the known PN shares the same syntactic position and relationship type as PN_U.
  • ESLB: Similar to ESLA, but also considers synonyms of the words in PN_U's context.

The system then computes an "evidence" score for each potential semantic category (e.g., Person, Organization) based on the plausibility of these similar contexts and the ambiguity of the words involved. The category with the highest evidence score is assigned to the unknown PN. This approach is "untrained" in the sense that it doesn't require manual annotation of the learning corpus for PN tagging itself, but rather relies on existing linguistic resources and the corpus's inherent structure.

Datasets

The primary dataset used is a corpus of one million words extracted from economic news articles from the Italian newspaper "Il Sole 24 Ore." This corpus serves as the "learning corpus" for extracting contextual evidence. The authors also mention using a "seed" gazetteer and a dictionary of synonyms as resources. No new benchmarks were introduced; the evaluation is framed within the context of MUC (Message Understanding Conferences) standards.

Key Insights and Ideas

  • Context is King for Unknowns: The paper strongly argues that even for unknown proper names, the surrounding words and grammatical structures provide crucial clues for semantic classification. This moves beyond simple dictionary lookups.
  • Leveraging Existing Knowledge: Instead of requiring full manual annotation for every new domain, the method cleverly reuses existing knowledge (seed gazetteer, corpus structure) to infer types for new entities.
  • Syntactic Links as Bridges: Elementary syntactic links (esls) act as a powerful mechanism to connect unknown PNs to known ones through shared contextual patterns.
  • Information Gain as a Metric: The use of Information Gain alongside precision highlights the importance of evaluating not just accuracy but also the complexity and informativeness of the classification task, especially for categories with low prior probability.
  • Products as a Challenging Case: The experiments reveal that classifying "Product" entities is particularly difficult due to their rarity and the lack of specific contextual rules, underscoring the limitations of current approaches.

Key Results

  • The algorithm successfully assigns entity types to unknown PNs by analyzing similar contexts.
  • While not explicitly quantified for the overall system's performance on unknown PNs, the paper demonstrates the method's effectiveness through a detailed example ("Quick_Take_200").
  • In an experiment where 35 product names were removed from the gazetteer, the system was able to re-classify 31 of them, showing robustness.
  • Precision for classifying PNs as "Product" was 22.86% with an Information Gain of 0.65, indicating a difficult but informative classification task.
  • For other categories like "ORGANIZ," "LOCATION," and "PERSON," precision was significantly higher (e.g., 100% for ORGANIZ) with lower Information Gain, suggesting these are easier to classify due to more abundant contextual evidence.

Limitations and Future Work

  • Gazetteer and Rule Base Dependency: The system's performance is still affected by the size and quality of the initial "seed" gazetteer and contextual rules.
  • Inflected Languages: Challenges with POS taggers in inflected languages like Italian can impact the accuracy of syntactic analysis.
  • Limited Linguistic Resources: The scarcity of PN gazetteers for languages other than English remains a significant hurdle.
  • Synonymy vs. Semantic Category: Future work could explore enriching the context similarity by considering words that belong to the same semantic category, not just strict synonyms.
  • Generalizing Complex Nominals: The authors plan to investigate class-independent heuristics for capturing complex nominals, which are often challenging to classify.
  • Reproducing in English: Due to the availability of resources, they intend to reproduce their experiment in English to further validate their approach.

Summary updated

BibTeX
@inproceedings{cucchiarelli1998automatic,
  author = {Alessandro Cucchiarelli and Danilo Luzi and Paola Velardi},
  title = {{Automatic Semantic Tagging of Unknown Proper Names}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1998},
}
Figure