Introduction
The principle of compositionality suggests that the meaning of a complex linguistic expression is a function of the meanings of its constituent parts and the rules used to combine them. While this holds for literal phrases like "green tree," it fails for idiomatic expressions such as "couch potato" or "rat race," where the overall meaning is non-compositional. Accurately predicting the degree of compositionality is vital for downstream Natural Language Processing (NLP) tasks, including machine translation and word sense disambiguation.
Traditional distributional models, such as Word2Vec or GloVe, typically assess compositionality by comparing the vector representation of a phrase to the vectors of its individual components. However, these models often lack the nuanced semantic information required to distinguish between literal and figurative uses. This paper proposes a novel hybrid approach that augments standard distributional semantics with hierarchical taxonomic information (hypernymy) represented in hyperbolic space using Poincaré embeddings.
Methodology
The proposed model, referred to as MODEL-DP, utilizes a hybrid scoring metric that integrates two distinct semantic signals:
- Distributional Component: This follows the standard approach of calculating the cosine similarity between the vector of the entire phrase and the sum of the vectors of its constituent words.
- Hypernymy Component: The authors hypothesize that literal phrases share common hypernyms with their "head" words. For instance, "art school" and "school" both fall under the hypernym "educational institution." In contrast, idioms like "hot dog" share few, if any, specific hypernyms with "dog."
To implement the hypernymy component, the authors extracted 27.6 million hyponym-hypernym pairs from a large corpus using Hearst patterns. These pairs were used to train Poincaré embeddings. Unlike Euclidean space, hyperbolic space (specifically the Poincaré ball model) is mathematically suited for representing hierarchical, tree-like structures with minimal distortion because its volume grows exponentially toward the boundary. The final compositionality score is a weighted average of the distributional and Poincaré similarity scores, controlled by a parameter $\alpha$.
Evaluation
The models were evaluated on three benchmark datasets:
- Reddy (RD): 90 compounds with human ratings from 0 to 5.
- Reddy++ (RD++): An extension of the Reddy dataset containing 180 adjective-noun pairs.
- Farahmand (FD): A larger dataset of 1,042 compounds with expert binary judgments.
Performance was measured using Spearman’s rank correlation ($\rho$) between the model's predicted scores and human gold-standard judgments.
Results and Discussion
Initial experiments with base models revealed that while distributional models provide a solid foundation, Poincaré embeddings alone are insufficient for compositionality prediction.
As shown in Figure 1, W2V-CBOW emerged as the strongest distributional baseline. However, the integration of hierarchical information in MODEL-DP provided a significant performance boost. The authors found that a weighting of $\alpha = 0.4$ (placing more weight on the distributional component but significantly incorporating the hierarchical signal) yielded the best results.
Figure 5 illustrates the consistent superiority of MODEL-DP over the W2V-CBOW baseline across both reduced and full dataset versions. The hybrid approach achieved a 2-4% absolute improvement in Spearman’s correlation. This suggests that "context" (from distributional data) and "hierarchy" (from Poincaré embeddings) provide complementary semantic information.
Conclusion and Future Work
This research demonstrates that incorporating taxonomic hierarchies via hyperbolic embeddings significantly enhances the prediction of noun phrase compositionality. By leveraging the natural fit of Poincaré balls for hierarchical data, the model better identifies literal phrases through hypernym consistency.
Despite these gains, challenges remain. The model's performance is limited by the sparsity of hypernyms in the training corpus—if Hearst patterns fail to identify a phrase's hypernyms, the model reverts to standard distributional methods. Additionally, "semi-idiomatic" phrases like "hard disk," which share hypernyms with their components despite having specialized meanings, remain difficult to classify. Future work will explore extending this approach to other languages and grammatical structures beyond noun phrases.
Summary updated