NASARI: a Novel Approach to a Semantically-Aware Representation of Items

José Camacho-Collados, Mohammad Taher Pilehvar, Roberto NavigliNAACL 2015

TL;DR

This paper introduces NASARI, a novel method for creating rich semantic representations of concepts by combining knowledge from both WordNet (a structured lexicon) and Wikipedia (a vast encyclopedia). NASARI achieves state-of-the-art results on word similarity and sense clustering tasks by leveraging lexical specificity for weighting and a unique synset-based dimensionality reduction.

What Problem Does This Paper Address?

Accurately representing the meaning of words and concepts is crucial for many Natural Language Processing (NLP) tasks. Traditional approaches often rely on either structured lexicographic resources like WordNet or unstructured encyclopedic resources like Wikipedia. However, relying on a single resource limits the richness of the representation and makes it resource-specific. Furthermore, purely distributional methods (based on word co-occurrence) struggle to capture polysemy (multiple meanings of a word) without extensive sense-annotated data. This paper aims to bridge this gap by creating a more robust and versatile semantic representation that benefits from the complementary strengths of both WordNet and Wikipedia.

Methods

NASARI's approach involves two main phases:

  1. Collecting Contextual Information: For a given concept (represented as a pair of a Wikipedia page and a WordNet synset), NASARI gathers a set of relevant Wikipedia pages. This set includes the target page itself and pages that link to it, as well as pages linked from the synset's neighbors in WordNet. This creates a rich contextual "bag" of information.
  2. Vector Construction:
    • Word-based Representation: The collected contextual information is processed into a vector where dimensions represent individual words. A key innovation here is the use of lexical specificity (a statistical measure based on hypergeometric distribution) to compute the weights of these words. This measure is more robust than traditional tf-idf, especially for identifying terms peculiar to a specific context.
  3. Synset-based Representation: To address sparsity and improve generalization, NASARI employs a novel semantically-aware dimensionality reduction technique. It groups words from the word-based vector into dimensions represented by WordNet synsets (specifically, common hypernyms). This process implicitly disambiguates word senses and smooths the representation. The weights for these synset dimensions are also calculated using lexical specificity.

The final representation for a concept is a combination of its word-based and synset-based vectors. For comparing these vectors, the paper uses Weighted Overlap (WO), a method that considers the rank of dimensions, which they found to be more effective than cosine similarity for their specificity-based vectors.

Datasets

The paper evaluates NASARI on the following standard datasets:

  • Word Similarity:
    • RG-65
    • MC-30
    • WordSim-353 (specifically the similarity subset, WS-Sim)
  • Sense Clustering:
    • A 500-pair dataset created by Dandala et al. (2013).
  • The SemEval dataset, also from Dandala et al. (2013).

The paper does not introduce new datasets but uses existing benchmarks to demonstrate the effectiveness of their approach.

Key Insights and Ideas

  • Complementary Knowledge is Key: Combining WordNet and Wikipedia provides a significant advantage over using either resource alone. This is demonstrated by NASARI outperforming systems that rely solely on WordNet or Wikipedia.
  • Lexical Specificity for Weighting: Using lexical specificity instead of traditional tf-idf for weighting vector dimensions leads to more robust and discriminative representations, especially for identifying context-specific terms.
  • Synset-based Reduction for Richness: The dimensionality reduction to synsets not only creates a more compact representation but also implicitly handles word sense disambiguation and smooths out noise, leading to richer semantic meaning.
  • Weighted Overlap for Comparison: For specificity-based vectors, Weighted Overlap proves to be a more suitable similarity measure than cosine similarity, particularly for tasks like sense clustering where vector dimensions might be sparse.

Key Results

  • State-of-the-Art Performance: NASARI achieves state-of-the-art results on the RG-65 and MC-30 word similarity datasets, and competitive results on WS-Sim.
  • Sense Clustering Superiority: In an unsupervised setting, NASARI outperforms supervised systems (like those by Dandala et al., 2013) on both sense clustering datasets, achieving accuracies of 84.6% and 88.4% on the 500-pair and SemEval datasets, respectively.
  • Efficiency Gains: The use of lexical specificity results in significantly smaller vectors (up to nine times smaller than tf-idf based vectors), leading to better space utilization and faster processing.
  • Combination Advantage: Combining the word-based and synset-based representations consistently improves performance across tasks compared to using either representation alone.

Limitations and Future Work

  • Limitations: The paper notes that for the WS-Sim dataset, their system produces more appropriate similarity judgments than the dataset's gold scores, which are known to conflate similarity and relatedness and assign unusual scores to certain word pairs (e.g., antonyms). The synset-based representation's cosine similarity performance on sense clustering was lower, attributed to the small vector size and the fixed threshold.
  • Future Work: The authors plan to integrate NASARI into BabelNet (a multilingual semantic network) to enable multilingual concept comparison. They also intend to apply NASARI to multilingual Word Sense Disambiguation.

Summary updated

BibTeX
@inproceedings{camachocollados2015nasari,
  author = {José Camacho-Collados and Mohammad Taher Pilehvar and Roberto Navigli},
  title = {{NASARI: a Novel Approach to a Semantically-Aware Representation of Items}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}
Figure