Abstract
Currently, most word sense disambiguation (WSD) systems function as individual word sense experts, rarely accounting for word sense transitions between linearly consecutive or syntactically dependent words. These transitions are crucial as they embody semantic fluency and help mitigate data sparsity. This paper utilizes the HowNet knowledge base to decompose word senses into basic semantic units called sememes. By transforming word sense transitions into multiple sememe transitions—which are easier to capture due to the limited number of sememes—WSD is redefined as a labeling task. We propose Multi-layered Conditional Random Fields (MLCRF) to model these transitions. Experimental results demonstrate that MLCRF outperforms baseline systems and maximum entropy models, with syntactic and hypernym features providing significant performance enhancements.
Introduction
Word Sense Disambiguation (WSD) remains a fundamental challenge in Natural Language Processing. Traditional approaches often treat words in isolation, ignoring the underlying "semantic flow" between adjacent words. While modeling transitions between full word senses would be ideal, the sheer number of possible senses leads to a massive data sparsity problem. For instance, if two adjacent words each have ten possible senses, there are 100 potential transitions to learn; across an entire vocabulary, this becomes computationally and statistically intractable.
This paper addresses the sparsity problem by decomposing complex word senses into sememes—the smallest, non-decomposable semantic units defined in the HowNet knowledge base. With only approximately 1,600 sememes in total, the transition space becomes manageable and generalizable.
Methodology: Multi-layered Conditional Random Fields
The core of our approach is the Multi-layered Conditional Random Field (MLCRF). Because a single word sense is often composed of multiple sememes (e.g., "research institute" involves sememes for InstitutePlace, research, and knowledge), we model WSD as a multi-layered labeling problem.
We implemented two distinct architectures for the MLCRF:
- Sequential MLCRF: Models transitions between sememes based on the linear order of words in a sentence.
- Tree MLCRF: Models transitions based on dependency syntax, capturing relationships between child and parent nodes (e.g., a verb and its object).
To decode these models efficiently, we developed a multi-layered Viterbi algorithm. This approach allows higher-layer labels to utilize lower-layer labels as features, ensuring that the primary category of a word informs its secondary attributes without an exponential increase in computational complexity.
Experimental Setup
We evaluated our models using data from the Chinese Linguistic Data Consortium (ChineseLDC), consisting of a training set of over 7,000 words and a testing set of over 1,600 words. Phrase structure trees from the corpus were converted into gold-standard dependency trees for the Tree CRF model. HowNet provided the necessary sememe definitions and hypernym hierarchies used as features.
Results and Discussion
The MLCRF approach was compared against a "Most Frequent Sense" baseline and a Maximum Entropy model. The results indicate that deconstructing senses into sememes allows the model to learn general semantic patterns (e.g., a "Place" sememe frequently follows an "Event" sememe) that are independent of specific word forms.
As shown in the results table, the Tree-based MLCRF achieved the highest sense accuracy at 85.9%, significantly outperforming the Sequential MLCRF (79.9%) and the baseline (79.0%). This suggests that semantic logic aligns more closely with syntactic dependency structures than with simple linear word order. Furthermore, the inclusion of hypernym features (marked as "+h" in the table) provided a substantial boost, particularly for the Tree model, by further reducing the impact of data sparsity.
Conclusion
By redefining WSD as a sememe labeling task, we successfully mitigated the data sparsity issues inherent in traditional sense-transition modeling. The Tree-based MLCRF demonstrates that syntactic structure is a powerful signal for semantic disambiguation. Future work will focus on integrating syntactic parsing and WSD into a joint optimization framework to prevent error propagation from automated parsers and exploring the applicability of sememe decomposition to languages with different morphological characteristics.
Summary updated