Abstract
Abstract Linguistic Knowledge Graphs (e.g., WordNet and HowNet) describe linguistic knowledge in formal and structural language, which can be easily incorporated in modern natural language processing systems. In this chapter, we focus on the research about HowNet. We first briefly introduce the background and basic concepts of HowNet and sememe. Next, we introduce the motivations of sememe representation learning and existing approaches. At the end of this chapter, we review important applications of sememe representation.
Introduction: The Challenge of Semantic Atoms
In standard Natural Language Processing (NLP), words are often treated as atomic symbols. While word embeddings like Word2vec represent words as vectors, they frequently struggle with two major issues: polysemy (one word having multiple meanings) and the sparsity of rare words.
While resources like WordNet group words into "synsets," they don't explain why those words are related at a fundamental level. Sememes—the smallest indivisible units of semantic meaning—offer a solution. By looking "inside" the word at its constituent sememes (e.g., "boy" = human + male + child), models can gain a more precise and structured understanding of language. The challenge lies in computationally representing this expert-annotated knowledge and applying it to modern neural network architectures.
Background: HowNet and Sememes
The primary resource for sememe-based research is HowNet, a specialized linguistic knowledge base. Unlike WordNet, HowNet decomposes word senses into a structured set of sememes.
As shown in Figure 1, a word like "apple" is mapped to multiple senses (computer, phone, fruit, tree), each defined by a specific configuration of sememes. This provides a "closed set" of meanings; while the number of words is infinite, the number of sememes is limited (around 2,000 in HowNet), providing a powerful bottleneck for regularization.
Methods for Sememe Representation Learning
Several novel architectures have been designed to integrate sememe knowledge into representation learning:
- Simple Sememe Aggregation (SSA): A baseline approach that represents a word by averaging the embeddings of all its constituent sememes.
- Sememe Attention over Context (SAC): An attention-based model that uses a target word to "attend" to the most relevant senses/sememes of its context words, performing "soft" word sense disambiguation.
- Sememe Attention over Target (SAT): The inverse of SAC; it uses the surrounding context to select the most appropriate sememes for the target word itself.
- Sememe-Driven Language Model (SDLM): A hierarchical decoder that predicts the next word in three steps: predicting likely sememes, then the sense, and finally the specific word.
- Character-enhanced Sememe Prediction (CSP): For languages like Chinese, this model uses internal character information (morphemes) to predict sememes for new or rare words, combining internal signals with external context.
Key Insights and Results
- Contextual Disambiguation: Context acts as a filter that activates specific sememes. This allows a single word vector to behave differently depending on its neighbors.
- Improved Embeddings: Models incorporating sememes (SST, SAC, SAT) consistently outperform standard Skip-gram models in word similarity tasks and word sense disambiguation.
- Human Alignment: Sememe-based formulas for phrase compositionality achieved a 0.75 Pearson correlation with human judgments, proving that sememes accurately capture how humans combine meanings.
- Language Modeling Efficiency: The SDLM approach reduced perplexity, showing that predicting "meaning" before "words" is a more efficient path for neural networks.
- Rare Word Handling: For morphologically rich languages, internal word structure is a better predictor of sememes than context, especially for rare words.
Limitations and Future Directions
Current models often treat sememes as a "bag of labels," ignoring the complex tree-like taxonomy provided in HowNet. Additionally, while sememes help with rare words, more research is needed for low-resource languages that lack a HowNet-style KB. A major future direction is the automatic construction of sememe knowledge bases for other languages using cross-lingual transfer techniques.
Summary updated