Abstract
Measuring semantic relatedness between two words is a significant problem in many areas such as natural language processing. Existing approaches to the semantic relatedness problem mainly adopt the co-occurrence principle and regard two words as highly related if they appear in the same sentence frequently. However, such solutions suffer from low coverage and low precision because i) the two highly related words may not appear close to each other in the sentences, e.g., the synonyms; and ii) the co-occurrence of words may happen by chance rather than implying the closeness in their semantics. In this paper, we explore the latent semantics (i.e., concepts) of the words to identify highly related word pairs. We propose a hierarchical association network to specify the complex relationships among the words and the concepts, and quantify each relationship with appropriate measurements. Extensive experiments are conducted on real datasets and the results show that our proposed method improves correlation precision compared with the state-of-the-art approaches.
Introduction
In Natural Language Processing (NLP), determining semantic relatedness—how closely words like tea and cup are connected—is a fundamental challenge. Traditional methods typically rely on the co-occurrence principle: if two words frequently appear in the same sentence, they are deemed related. However, this approach faces significant limitations. It often fails to capture synonyms that rarely appear together (low coverage) and can be misled by words that co-occur by chance without a deep semantic link (low precision). Furthermore, many existing models struggle with parts of speech beyond nouns, such as adjectives and verbs. The Hierarchical Association Network (HAN) addresses these issues by leveraging the structured knowledge of Wikipedia to map words to latent concepts and categories.
Methodology
The core of the proposed system is the Hierarchical Association Network (HAN), a weighted directed graph organized into two primary layers: the Word Layer and the Concept Layer.
The model calculates relatedness by analyzing three distinct types of relationships:
- Word-to-Word ($F_w$): This captures standard proximity using established tools like word2vec, GloVe, or sentence-level co-occurrence.
- Word-to-Concept ($F_{wc}$): This maps raw words to Wikipedia page titles (concepts). To resolve ambiguity—such as distinguishing between "Apple" the fruit and "Apple Inc." the company—the model utilizes a "Link Popularity" (LP) metric that evaluates the context of hyperlinks within Wikipedia.
- Concept-to-Concept ($F_c$): This layer represents the model's primary innovation. It combines three scores: co-occurrence ($M_1$), category-based overlap ($M_2$) using Jaccard coefficients, and link-structure distance ($M_3$) based on Normalized Wikipedia Distance.
By "lifting" words into the concept layer, HAN can identify relationships between words that never appear together in text but share a common conceptual foundation.
Experiments and Results
The researchers evaluated HAN using a 2015 Wikipedia dump (~5 million articles) and several high-profile benchmarks, including WordSimilarity-353 (WS353), Miller & Charles (MC), and Rubenstein & Goodenough (RG). They also introduced ConceptRel-250, a new dataset of 250 concept pairs rated by human volunteers to specifically test concept-level relatedness.
HAN consistently outperformed state-of-the-art baselines:
- WS353: Achieved a Spearman correlation of 0.586, significantly higher than the 0.476 achieved by basic association networks.
- MC and RG: In these synonym-focused datasets, HAN reached Pearson correlations of 0.886 and 0.884, respectively, surpassing specialized models like SaSA.
- Feature Integration: The results demonstrated that combining all three concept-level features ($M_1, M_2, M_3$) provided a much more accurate fit to human ratings than any single feature alone.
Conclusion
The Hierarchical Association Network demonstrates that integrating structural knowledge from Wikipedia—its hyperlinks and category trees—allows for a more nuanced understanding of semantic relatedness than raw text analysis alone. While current results are promising, future work will focus on expanding the model's granularity to handle phrases, full sentences, and larger documents, as well as refining the capture of deep contextual information within encyclopedia entries.
Summary updated