A New Minimally-Supervised Framework for Domain Word Sense Disambiguation

Stefano Faralli, Roberto NavigliEMNLP 2012

TL;DR

This paper introduces a novel framework for domain-specific Word Sense Disambiguation (WSD) that bootstraps its own domain-specific glossaries from the web, bypassing the limitations of general-purpose resources like WordNet. This approach achieves high performance in WSD across numerous domains with minimal human supervision.

What Problem Does This Paper Address?

Understanding text often requires knowing the specific domain it belongs to. For example, the word "restoration" can have very different meanings in the context of biology, art, or computer systems. Word Sense Disambiguation (WSD) is the task of automatically identifying the correct meaning of a word in a given text. Traditional WSD systems often rely on general-purpose dictionaries like WordNet. However, WordNet lacks specialized terms and fine-grained senses crucial for many domains, making it difficult to achieve high performance in domain-specific WSD. Existing domain-specific approaches often require significant manual effort or large amounts of labeled data, which are scarce for many specialized fields. This paper tackles the challenge of building effective domain WSD systems without relying on pre-existing, comprehensive domain knowledge bases.

Methods

The core of the proposed framework lies in a two-phase process:

  1. Bootstrapping Domain Glossaries: This phase aims to automatically acquire domain-specific glossaries from the web with minimal human intervention.
  2. Initial Seed Selection: The process begins with a small, manually chosen set of "seed" hypernymy relations (e.g., "firewall" is a type of "security system"). This is the only human input required.
  3. Iterative Expansion: The system then iteratively queries search engines using these seeds to find candidate web pages. It extracts term-gloss pairs from these pages using pattern matching and generalization techniques.
  4. Gloss Ranking and Filtering: Extracted glosses are ranked based on their relevance to the domain (using domain terms found in the gloss) and filtered to remove irrelevant ones.
  5. New Seed Selection: The highest-ranked glosses are used to identify new seed terms and hypernyms for the next iteration, progressively expanding the glossary.
  6. Coverage Enhancement: To further enrich the glossaries, the system incorporates relevant term-gloss pairs from WordNet after filtering them for domain relevance.
  7. Domain WSD: Once domain glossaries are acquired, they are used as sense inventories for WSD.
  8. Graph Construction: For each domain, a graph is built where nodes represent term-gloss pairs. Edges connect terms if one appears in the gloss of the other, capturing semantic relationships.
  9. Personalized PageRank (PPR): A graph-based WSD algorithm, Personalized PageRank, is applied. Given an input text, the algorithm identifies domain-specific content words and uses them to bias the PageRank computation on the domain graph. This helps in identifying the most relevant gloss for a target word.
  10. Domain-Boosted PPR: A second algorithm further enhances PPR by incorporating domain distribution information. It calculates a score for each potential gloss based on both its relevance within the domain graph and the overall relevance of that domain to the input text.

Datasets

  • Custom 30-Domain Dataset: The authors created a dataset from 30 domains featured in Wikipedia. They extracted 6-paragraph text snippets for each domain from the Gigaword corpus, resulting in 180 paragraphs.
  • SPORTS and FINANCE Datasets: These are gold-standard datasets from Koeling et al. (2005), used for evaluating WSD performance.
  • ENVIRONMENT Dataset: This dataset is from the Semeval-2010 domain WSD task.
  • WordNet Domains (WND): Used for comparing the coverage and precision of their acquired sense inventory.

Key Insights and Ideas

  • Bootstrapping Glossaries is Feasible and Effective: The paper demonstrates that it's possible to automatically build rich, domain-specific glossaries from the web with minimal human seeds, overcoming the limitations of static resources like WordNet.
  • Domain-Specific Glossaries are Crucial for High-Performance WSD: The acquired glossaries, which contain specialized terms and senses, significantly improve WSD performance compared to using general-purpose inventories.
  • Two Levels of Granularity: The framework naturally supports both coarse-grained (domain-level) and fine-grained (gloss-level) WSD, offering flexibility.
  • Personalized PageRank Adapts Well to Domain Graphs: The graph-based approach, particularly with domain-specific graphs, proves effective for WSD.
  • Domain Distribution Matters: Incorporating the overall domain relevance of a text to a specific domain further boosts WSD performance.

Key Results

  • High Performance: The proposed framework, using Domain PPR, achieved an F1 score of around 80% in the coarse-grained setting and 69% in the fine-grained setting on their 30-domain dataset.
  • Significant Improvement over Baselines: Domain PPR outperformed Vanilla PPR and a "Predominant Domain" baseline.
  • Coverage: The acquired multi-domain glossary contained over 176,000 terms and 223,000 glosses, with a substantial portion of terms not found in WordNet.
  • Precision: A manual validation estimated the precision of their sense inventory at 87.0%.
  • Robustness on Gold Standards: The framework showed strong coarse-grained recall (70-80%) on challenging gold-standard datasets where WordNet-based systems struggled.

Limitations and Future Work

  • Manual Seed Selection: While minimal, the initial selection of seed hypernymy relations is a manual step.
  • Noisy Semantic Relations: The constructed domain graphs contain "noisy" semantic relations due to the nature of the bootstrapping process.
  • Future Work: The authors suggest exploring state-of-the-art supervised WSD systems if large, domain-specific sense-labeled corpora become available. They also note that the framework can be applied to any language with sufficient online resources. The availability of the multi-domain glossary and seeds is provided for further research.

Summary updated

BibTeX
@inproceedings{faralli2012new,
  author = {Stefano Faralli and Roberto Navigli},
  title = {{A New Minimally-Supervised Framework for Domain Word Sense Disambiguation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2012},
}
Figure