Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations

Goran Glavas, Simone Paolo Ponzetto

Abstract

Detecting asymmetric lexico-semantic relations, such as hypernymy (is-a) and meronymy (part-of), is fundamental for natural language understanding. While distributional word embeddings excel at capturing general similarity, they often struggle with the inherent directionality of these specific relations. We propose the Dual Tensor model, a neural architecture that specializes general word embeddings into role-specific representations using two distinct tensors. By leveraging multiple tensor slices to handle polysemy and a bilinear product for scoring, the model achieves state-of-the-art performance on hypernymy and meronymy detection tasks. Notably, this purely distributional approach outperforms more complex hybrid models that rely on syntactic dependency paths, demonstrating high efficiency and cross-lingual applicability.

Introduction

In computational semantics, identifying how words relate to one another is a primary goal. Standard embeddings like Word2Vec or GloVe are effective for finding "related" terms but fail to distinguish the direction of asymmetric relations. For instance, while a "dog" is an "animal" (hypernymy), the reverse is not true.

Existing solutions often treat word similarity symmetrically or rely on "path-based" features—analyzing grammatical structures in large corpora. However, path-based methods are computationally expensive and language-dependent. The Dual Tensor model addresses these issues by providing a language-agnostic, distributional framework that explicitly models the "subject" and "object" roles in a relation.

The Dual Tensor Architecture

The core innovation of the Dual Tensor model is its ability to transform "unspecialized" input embeddings into role-specific vectors.

The Dual Tensor model architecture, illustrating the transformation of input embeddings through dual specialization tensors and bilinear product scoring.

The architecture consists of several key stages:

  1. Dual Specialization Tensors: The model employs two tensors, $W_L$ and $W_R$, to transform input embeddings based on their position in a pair. For a pair $(e_1, e_2)$, $e_1$ is processed by the "left" tensor and $e_2$ by the "right" tensor.
  2. Multi-Slice Polysemy Handling: Each tensor contains $k$ slices. This allows the model to capture different semantic "senses" or facets of a word, effectively managing polysemy without explicit sense disambiguation.
  3. Bilinear Product Scoring: Interaction between the specialized vectors is measured via a bilinear product layer. The resulting scores from each slice are aggregated to produce a final relational score.
  4. Optimization: The model is trained using a margin-based loss function to maximize the score for valid relational pairs while minimizing it for negative examples.

Experimental Setup and Datasets

The model was evaluated across several benchmarks to test its robustness:

  • HypeNet: A large-scale hypernymy dataset.
  • WN-Hy and WN-Me: New datasets derived from WordNet for hypernymy and meronymy in English, Spanish, and French.
  • Ranking Benchmarks: BLESS, EVALuation, Weeds, and Benotto.

The authors utilized "lexical splits"—where the training and test sets contain entirely disjoint vocabularies—to ensure the model learns the underlying relation rather than simply memorizing specific word pairs.

Key Results and Insights

The Dual Tensor model demonstrates superior performance across both hypernymy and meronymy tasks.

Performance comparison on the HypeNet dataset, showing the Dual Tensor model (DUAL-T) outperforming hybrid and baseline models on lexical and random splits.

State-of-the-Art Performance

On the challenging HypeNet lexical split, the Dual Tensor model (DUAL-T) achieved an F1-score of 74.3, significantly surpassing the HypeNet hybrid model (70.0) and standard SVM baselines (63.7). It also excelled in meronymy detection, reaching an F1 of 67.9 on lexical splits, a task typically considered more difficult than hypernymy.

Role-Specific Embeddings and Simplicity

The results validate the hypothesis that a word's meaning is role-dependent in asymmetric relations. By explicitly modeling these transformations, the model avoids the need for complex syntactic dependency paths. This simplicity allows for consistent performance across different languages (English, Spanish, and French) without requiring language-specific parsers.

Polysemy and Stability

The multi-slice approach proved effective for handling polysemous words. Performance remained stable even as the number of word senses increased, suggesting that the tensor slices successfully partition the semantic space into relevant "senses" for the relation at hand.

Conclusion and Future Work

The Dual Tensor model provides a powerful yet simple distributional framework for detecting asymmetric lexico-semantic relations. By specializing embeddings for specific roles, it achieves high accuracy and cross-lingual stability. Future research may explore multi-relational models capable of discriminating between various relation types simultaneously, as well as the integration of contextualized embeddings to further enhance semantic representation.

Summary updated

Abstract

Goran Glavaš, Simone Paolo Ponzetto. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017.
BibTeX
@inproceedings{glavas2017dual,
  author = {Goran Glavas and Simone Paolo Ponzetto},
  title = {{Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1185},
}
Figure