Abstract
Detecting asymmetric lexico-semantic relations, such as hypernymy (is-a) and meronymy (part-of), is fundamental for natural language understanding. While distributional word embeddings excel at capturing general similarity, they often struggle with the inherent directionality of these specific relations. We propose the Dual Tensor model, a neural architecture that specializes general word embeddings into role-specific representations using two distinct tensors. By leveraging multiple tensor slices to handle polysemy and a bilinear product for scoring, the model achieves state-of-the-art performance on hypernymy and meronymy detection tasks. Notably, this purely distributional approach outperforms more complex hybrid models that rely on syntactic dependency paths, demonstrating high efficiency and cross-lingual applicability.
Introduction
In computational semantics, identifying how words relate to one another is a primary goal. Standard embeddings like Word2Vec or GloVe are effective for finding "related" terms but fail to distinguish the direction of asymmetric relations. For instance, while a "dog" is an "animal" (hypernymy), the reverse is not true.
Existing solutions often treat word similarity symmetrically or rely on "path-based" features—analyzing grammatical structures in large corpora. However, path-based methods are computationally expensive and language-dependent. The Dual Tensor model addresses these issues by providing a language-agnostic, distributional framework that explicitly models the "subject" and "object" roles in a relation.
The Dual Tensor Architecture
The core innovation of the Dual Tensor model is its ability to transform "unspecialized" input embeddings into role-specific vectors.
The architecture consists of several key stages:
- Dual Specialization Tensors: The model employs two tensors, $W_L$ and $W_R$, to transform input embeddings based on their position in a pair. For a pair $(e_1, e_2)$, $e_1$ is processed by the "left" tensor and $e_2$ by the "right" tensor.
- Multi-Slice Polysemy Handling: Each tensor contains $k$ slices. This allows the model to capture different semantic "senses" or facets of a word, effectively managing polysemy without explicit sense disambiguation.
- Bilinear Product Scoring: Interaction between the specialized vectors is measured via a bilinear product layer. The resulting scores from each slice are aggregated to produce a final relational score.
- Optimization: The model is trained using a margin-based loss function to maximize the score for valid relational pairs while minimizing it for negative examples.
Experimental Setup and Datasets
The model was evaluated across several benchmarks to test its robustness:
- HypeNet: A large-scale hypernymy dataset.
- WN-Hy and WN-Me: New datasets derived from WordNet for hypernymy and meronymy in English, Spanish, and French.
- Ranking Benchmarks: BLESS, EVALuation, Weeds, and Benotto.
The authors utilized "lexical splits"—where the training and test sets contain entirely disjoint vocabularies—to ensure the model learns the underlying relation rather than simply memorizing specific word pairs.
Key Results and Insights
The Dual Tensor model demonstrates superior performance across both hypernymy and meronymy tasks.
State-of-the-Art Performance
On the challenging HypeNet lexical split, the Dual Tensor model (DUAL-T) achieved an F1-score of 74.3, significantly surpassing the HypeNet hybrid model (70.0) and standard SVM baselines (63.7). It also excelled in meronymy detection, reaching an F1 of 67.9 on lexical splits, a task typically considered more difficult than hypernymy.
Role-Specific Embeddings and Simplicity
The results validate the hypothesis that a word's meaning is role-dependent in asymmetric relations. By explicitly modeling these transformations, the model avoids the need for complex syntactic dependency paths. This simplicity allows for consistent performance across different languages (English, Spanish, and French) without requiring language-specific parsers.
Polysemy and Stability
The multi-slice approach proved effective for handling polysemous words. Performance remained stable even as the number of word senses increased, suggesting that the tensor slices successfully partition the semantic space into relevant "senses" for the relation at hand.
Conclusion and Future Work
The Dual Tensor model provides a powerful yet simple distributional framework for detecting asymmetric lexico-semantic relations. By specializing embeddings for specific roles, it achieves high accuracy and cross-lingual stability. Future research may explore multi-relational models capable of discriminating between various relation types simultaneously, as well as the integration of contextualized embeddings to further enhance semantic representation.
Summary updated