Abstract
Traditional Word Sense Induction (WSI) methods often struggle with data sparsity due to the independence assumption inherent in the Vector Space Model (VSM). This paper introduces a novel framework that leverages semantic relevance between words to enrich context representations. By constructing a word graph and applying Personalized PageRank (PPR), the method propagates importance from observed words to semantically related neighbors. Evaluation on the CLP2010 Chinese WSI dataset shows that this approach, WSI_SR, significantly outperforms standard Bag-of-Words baselines and provides more realistic estimations of the number of word senses.
Introduction
Word Sense Induction (WSI) is the unsupervised task of discovering the various meanings of a word without relying on pre-defined dictionaries. Most existing methods represent context using the Vector Space Model (VSM), where each dimension corresponds to a word. However, VSM assumes word independence, failing to recognize the semantic link between related terms like "river" and "lake." This leads to sparse vectors where contexts describing the same sense may show zero similarity if they do not share exact vocabulary. This paper addresses this "semantic gap" by smoothing context vectors through graph-based semantic modeling.
Methods
The proposed framework bridges the semantic gap through a multi-step process:
- Word Graph Construction: A word-by-context matrix is processed using Singular Value Decomposition (SVD) to reduce noise. The inner product between word vectors is then used to build a graph where nodes are words and edges represent semantic relevance.
- Personalized PageRank (PPR): To enrich a specific context, the words present in that text are treated as "seed" nodes. The PPR algorithm propagates weight from these seeds to their neighbors in the word graph. For instance, if "river" appears in a context, PPR assigns weight to "lake" and "stream," even if they are absent from the text.
- Clustering and Sense Detection: The resulting enriched vectors are clustered using the K-means algorithm. The optimal number of clusters ($k$), representing the number of senses, is automatically determined using the PK2 criterion.
Experimental Setup
The method was evaluated on the CLP2010 (CIPS-SIGHAN Joint Conference on Chinese Language Processing) Chinese WSI dataset. The dataset includes 100 target words (22 single-character and 78 multi-character) across 5,000 instances, with an average of 2.5 senses per word.
Results
The proposed method (WSI_SR) was compared against several baselines: a standard Bag-of-Words model (WSI_BOW), a graph-clustering approach (WSI_CW), and the "one cluster per word" baseline (1c1w).
As shown in Table 1, WSI_SR achieved a paired F-score of 58.5%, outperforming the WSI_BOW baseline by 7.26% absolute. While the trivial 1c1w baseline remains difficult to beat in raw F-score due to the Zipfian distribution of word senses in natural language, WSI_SR proves far more effective at discovering minority senses. Furthermore, WSI_SR predicted an average of 2.1 senses per word, which is much closer to the ground truth (2.5) than the WSI_CW baseline, which tended to over-partition words into an average of 6.7 senses.
Discussion and Conclusion
The core strength of this approach lies in its ability to use a graph to represent latent relationships, effectively "smoothing" context vectors to make them robust against vocabulary variation. Qualitatively, the PPR-based model successfully assigns significant weight to related terms (e.g., assigning 0.08 to "bridge" when only "river" is present), allowing the system to recognize semantic similarities that VSM would miss.
However, limitations remain. The model performs better on multi-character Chinese words than on highly polysemous single-character words. Future work may involve incorporating syntactic information (such as part-of-speech tags) or topic-level information to further refine sense induction, as well as exploring ways to improve the initial word graph construction on which the PPR propagation depends.
Summary updated