Merging Word Senses

Sumit Bhagwani, Shrutiranjan Satapathy, Harish KarnickEMNLP 2013

TL;DR

This paper tackles the problem of WordNet's overly fine-grained senses, which can hinder performance in many NLP applications. They propose a novel framework that learns synset similarity using a personalized SimRank algorithm, allowing for the creation of coarser sense inventories that significantly improve noun sense disambiguation.

What Problem Does This Paper Address?

WordNet, a widely used lexical database, organizes words into sets of synonyms called synsets, each representing a distinct meaning or "sense." While this fine-grained approach is valuable for some tasks like language learning, it can be detrimental for others, such as document categorization or information retrieval. These applications often benefit from a coarser understanding of word meaning. Existing methods for merging WordNet senses often struggle with determining the right level of granularity or produce inconsistent results. This paper aims to address this by providing a flexible framework to generate coarse sense inventories at arbitrary granularities, improving performance on downstream tasks.

Methods

The core of the proposed method lies in learning a robust synset similarity metric. They adapt the SimRank algorithm, a graph-based similarity measure that leverages the intuition that "two objects are similar if they are related to similar objects."

Key methodological contributions include:

  • Personalized SimRank: To overcome limitations of standard SimRank, they introduce a personalized version that allows incorporating known similarities (seeded from supervised learning) into the similarity computation. This biases the similarity metric towards human-annotated judgments.
  • Semi-Supervised Synset Similarity Learning: They train Support Vector Machines (SVMs) on human-labeled data (from OntoNotes) to predict the similarity between synset pairs that share at least one word. The posterior probabilities from these SVMs are then used to "seed" the personalized SimRank algorithm.
  • Coarsening WordNet via Connected Components: After learning the synset similarity, they construct an undirected graph where synsets are nodes and edges represent similarity above a certain threshold. Connected components in this graph are then used to define coarse senses. All synsets within a connected component are grouped into a single coarse sense. The threshold for edge creation directly controls the granularity of the resulting coarse senses.

Datasets

  • OntoNotes Release 3.0: Used for extracting synset pairs and their "merged" or "not merged" labels. This dataset is crucial for training the SVMs that seed the SimRank algorithm.
  • Omega ontology (via OntoNotes): Provides manually labeled mappings from Omega ontology senses to WordNet senses, used for obtaining high-quality, inter-annotator agreement judgments.
  • Senseval-3 (English all-words task): Used for evaluating the performance of the proposed coarse-grained sense inventories on the noun sense disambiguation task. They specifically focus on the 890 noun instances.

Key Insights and Ideas

  • The power of personalized similarity: By incorporating human-annotated similarities as a starting point, SimRank can be guided towards more meaningful and task-relevant similarity measures.
  • Bridging supervised and unsupervised learning: The approach effectively combines supervised learning (for initial seeding) with an unsupervised graph-based method (SimRank and connected components) for broad application.
  • Granularity control is key: The ability to control the coarseness of senses by simply adjusting a threshold on the similarity graph is a significant practical advantage.
  • Domain similarity is a strong cue: Features derived from domain labels (e.g., from eXtended WordNet Domains Project) proved to be highly effective for predicting synset similarity, suggesting that shared topicality is a good indicator of semantic relatedness.
  • WordNet's structure is valuable, but needs adaptation: While WordNet's inherent structure (hypernymy, etc.) is important, the paper demonstrates that learning a more nuanced similarity metric and then clustering synsets is more effective than relying solely on predefined relations.

Key Results

  • The proposed framework, when applied to noun synsets, significantly improves noun sense disambiguation performance on the Senseval-3 dataset.
  • The Connected Components Clustering (CCC) approach achieved a maximum improvement of 5.78% on supervised systems and 7.18% on an unsupervised system compared to random clustering at the same granularity.
  • Feature ablation studies highlighted the importance of WordNet similarity measures and domain cosine similarity features for the SVM classifier.
  • The paper demonstrates that by varying the threshold for connected components, the granularity of the coarse senses can be effectively controlled, leading to varying levels of improvement.

Limitations and Future Work

  • Convergence of SimRank: The paper notes that for higher values of the SimRank decay parameter (C), convergence can be slower, potentially impacting performance.
  • Focus on Nouns: The evaluation was primarily conducted on noun synsets, and extending the framework to verbs and other parts of speech is a natural next step.
  • Grading Relations: The authors suggest that grading the weights of WordNet relations (e.g., Hypernymy, Hyponymy) based on their relative importance could lead to further improvements.
  • Enriching the Graph: Incorporating semantic relatedness from sources like the WordNet Gloss Corpus could create a denser and richer graph, potentially improving similarity learning.
  • Exploring Other Clustering Algorithms: While Connected Components Clustering is simple and effective, the authors suggest that other clustering algorithms could be explored with their learned similarity metric.

Summary updated

BibTeX
@inproceedings{bhagwani2013merging,
  author = {Sumit Bhagwani and Shrutiranjan Satapathy and Harish Karnick},
  title = {{Merging Word Senses}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
}
Figure