Word Sense Disambiguation through Sememe Labeling

Xiangyu Duan, Jun Zhao, Bo XuIJCAI 2007 · OpenReview

Abstract

Currently, most word sense disambiguation (WSD) systems function as individual word sense experts, rarely accounting for word sense transitions between linearly consecutive or syntactically dependent words. These transitions are crucial as they embody semantic fluency and help mitigate data sparsity. This paper utilizes the HowNet knowledge base to decompose word senses into basic semantic units called sememes. By transforming word sense transitions into multiple sememe transitions—which are easier to capture due to the limited number of sememes—WSD is redefined as a labeling task. We propose Multi-layered Conditional Random Fields (MLCRF) to model these transitions. Experimental results demonstrate that MLCRF outperforms baseline systems and maximum entropy models, with syntactic and hypernym features providing significant performance enhancements.

Introduction

Word Sense Disambiguation (WSD) remains a fundamental challenge in Natural Language Processing. Traditional approaches often treat words in isolation, ignoring the underlying "semantic flow" between adjacent words. While modeling transitions between full word senses would be ideal, the sheer number of possible senses leads to a massive data sparsity problem. For instance, if two adjacent words each have ten possible senses, there are 100 potential transitions to learn; across an entire vocabulary, this becomes computationally and statistically intractable.

This paper addresses the sparsity problem by decomposing complex word senses into sememes—the smallest, non-decomposable semantic units defined in the HowNet knowledge base. With only approximately 1,600 sememes in total, the transition space becomes manageable and generalizable.

Methodology: Multi-layered Conditional Random Fields

The core of our approach is the Multi-layered Conditional Random Field (MLCRF). Because a single word sense is often composed of multiple sememes (e.g., "research institute" involves sememes for InstitutePlace, research, and knowledge), we model WSD as a multi-layered labeling problem.

We implemented two distinct architectures for the MLCRF:

  1. Sequential MLCRF: Models transitions between sememes based on the linear order of words in a sentence.
  2. Tree MLCRF: Models transitions based on dependency syntax, capturing relationships between child and parent nodes (e.g., a verb and its object).
Graphical representation of (a) Sequential MLCRF and (b) Tree MLCRF architectures.

To decode these models efficiently, we developed a multi-layered Viterbi algorithm. This approach allows higher-layer labels to utilize lower-layer labels as features, ensuring that the primary category of a word informs its secondary attributes without an exponential increase in computational complexity.

Experimental Setup

We evaluated our models using data from the Chinese Linguistic Data Consortium (ChineseLDC), consisting of a training set of over 7,000 words and a testing set of over 1,600 words. Phrase structure trees from the corpus were converted into gold-standard dependency trees for the Tree CRF model. HowNet provided the necessary sememe definitions and hypernym hierarchies used as features.

Results and Discussion

The MLCRF approach was compared against a "Most Frequent Sense" baseline and a Maximum Entropy model. The results indicate that deconstructing senses into sememes allows the model to learn general semantic patterns (e.g., a "Place" sememe frequently follows an "Event" sememe) that are independent of specific word forms.

Performance comparison of Sequential and Tree MLCRF models across different layers and sense accuracy.

As shown in the results table, the Tree-based MLCRF achieved the highest sense accuracy at 85.9%, significantly outperforming the Sequential MLCRF (79.9%) and the baseline (79.0%). This suggests that semantic logic aligns more closely with syntactic dependency structures than with simple linear word order. Furthermore, the inclusion of hypernym features (marked as "+h" in the table) provided a substantial boost, particularly for the Tree model, by further reducing the impact of data sparsity.

Conclusion

By redefining WSD as a sememe labeling task, we successfully mitigated the data sparsity issues inherent in traditional sense-transition modeling. The Tree-based MLCRF demonstrates that syntactic structure is a powerful signal for semantic disambiguation. Future work will focus on integrating syntactic parsing and WSD into a joint optimization framework to prevent error propagation from automated parsers and exploring the applicability of sememe decomposition to languages with different morphological characteristics.

Summary updated

Abstract

Currently most word sense disambiguation (WSD) systems are relatively individual word sense experts. Scarcely do these systems take word sense transitions between senses of linearly consecutive words or syntactically dependent words into consideration. Word sense transitions are very important. They embody the fluency of semantic expression and avoid sparse data problem effectively. In this paper, How Net knowledge base is used to decompose every word sense into several sememes. Then one transition between two words' senses becomes multiple transitions between sememes. Sememe transitions are much easier to be captured than word sense transitions due to much less sememes. When sememes are labeled, WSD is done. In this paper, multi-layered conditional random fields (MLCRF) is proposed to model sememe transitions. The experiments show that MLCRF performs better than a base-line system and a maximum entropy model. Syntactic and hypernym features can enhance the performance significantly.
BibTeX
@inproceedings{duan2007word,
  author = {Xiangyu Duan and Jun Zhao and Bo Xu},
  title = {{Word Sense Disambiguation through Sememe Labeling}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2007},
  url = {https://openreview.net/forum?id=S1ZfHXGOZB},
}
Figure