Kyoto: An Integrated System for Specific Domain WSD

Aitor Soroa, Eneko Agirre, Oier Lopez de Lacalle, Wauter Bosma, Piek Vossen, Monica Monachini, Jessie Lo, Shu-Kai HsiehSemEval @ ACL 2010 · OpenReview

Abstract: Aitor Soroa, Eneko Agirre, Oier Lopez de Lacalle, Wauter Bosma, Piek Vossen, Monica Monachini, Jessie Lo, Shu-Kai Hsieh. Proceedings of the 5th International Workshop on Semantic Evaluation.

2010.

Introduction

Word Sense Disambiguation (WSD)—the task of determining which "sense" of a word is used in a given context (e.g., "bank" as a financial institution vs. a river edge)—is notoriously difficult in specialized domains. Standard WSD systems are often trained on general text like news articles and struggle when words take on specific technical meanings. Supervised systems require massive amounts of hand-labeled data for every new domain, which is expensive and slow.

The Kyoto system addresses this by proposing an unsupervised, knowledge-based approach that can be quickly ported to any language or domain. It combines automated domain-term mining ("Tybots") with graph-based algorithms ("UKB") to solve WSD for specific technical fields. By extracting a domain-specific thesaurus and using it to bias a Personalized PageRank algorithm, the system successfully adapts to specialized vocabularies across multiple languages.

Methodology

The Kyoto system integrates two primary components:

  1. Tybots (Term Yielding Robots): These are text-mining tools that process domain-specific corpora to extract relevant terms. They organize these terms into a hierarchy and connect them to existing ontologies, effectively building a "domain-related thesaurus."
  2. UKB (Graph-based WSD): This tool represents a Language Knowledge Base (LKB) as a graph where nodes are concepts and edges are semantic relations. It uses Personalized PageRank (PPR) to find the most relevant concepts.
  3. In a standard run (kyoto-1), the algorithm is "seeded" with words from the immediate sentence.
  4. In domain-adapted runs (kyoto-2 and kyoto-3), the algorithm is instead seeded with the list of related domain terms discovered by the Tybots. This forces the system to prefer senses that are prominent within that specific field (e.g., favoring the "environment" sense of a word in a biology corpus).

The researchers also experimented with Bilingual Graphs, mapping Dutch, Italian, and Chinese WordNets to the much richer English WordNet 3.0 to leverage its denser web of relations.

Experimental Setup

The system was evaluated using the SemEval-2010 task #17 ("All-words Word Sense Disambiguation on a Specific Domain"). The evaluation involved four languages: English (WordNet 3.0), Dutch (Cornetto v1.3), Italian (Italwordnet), and Chinese (Chinese WordNet v1.6). Background documents provided by the organizers were used by the Tybots to learn domain-specific terminology.

Table 2: Statistics of the monolingual and bilingual knowledge bases used in the Kyoto system, showing the scale of entries and relations.

Key Results and Discussion

The evaluation results demonstrate that domain priming works: using a domain-specific thesaurus to bias the PageRank algorithm consistently outperformed using just the local sentence context. This proves that "global" domain knowledge is often more useful for WSD than "local" context in specialized fields.

Table 3: Performance comparison (Precision and Recall) of Kyoto system runs against baselines across four languages.
  • Dutch and Italian: The system's best runs successfully beat the "First Sense" baseline—a notoriously difficult baseline to top in WSD tasks.
  • English: The system performed close to, but slightly below, the First Sense baseline (Precision/Recall of ~0.48 vs 0.50).
  • Chinese: Performance was poor (Precision ~0.32), falling below even the random baseline. The authors discovered this was because the Chinese WordNet graph was "flat" and composed of many small, disconnected components, preventing PageRank from effectively spreading importance.
  • The "English Advantage": By linking other languages to the English WordNet (Bilingual graphs), performance generally improved, suggesting that the structural richness of the underlying graph matters more than language-specific nuances.

Conclusion

The Kyoto project's integrated architecture validates the utility of domain-specific knowledge in WSD. However, the system remains highly dependent on the quality of the underlying Knowledge Base. Future work involves re-examining the Chinese WordNet to understand how to better connect its components and fine-tuning the Tybot term extraction to improve overall system maturity.

Summary updated

BibTeX
@inproceedings{soroa2010kyoto,
  author = {Aitor Soroa and Eneko Agirre and Oier Lopez de Lacalle and Wauter Bosma and Piek Vossen and Monica Monachini and Jessie Lo and Shu-Kai Hsieh},
  title = {{Kyoto: An Integrated System for Specific Domain WSD}},
  booktitle = {SemEval @ ACL},
  year = {2010},
  url = {https://openreview.net/forum?id=JxN64jODqG},
}
Figure