Abstract
Linguistic and commonsense knowledge bases describe knowledge in formal and structural languages. Such knowledge can be easily leveraged in modern natural language processing systems. In this chapter, we introduce one typical kind of linguistic knowledge (sememe knowledge) and a sememe knowledge base named HowNet. In linguistics, sememes are defined as the minimum indivisible units of meaning. We first briefly introduce the basic concepts of sememe and HowNet. Next, we introduce how to model the sememe knowledge using neural networks. Taking a step further, we introduce sememe-guided knowledge applications, including incorporating sememe knowledge into compositionality modeling, language modeling, and recurrent neural networks. Finally, we discuss sememe knowledge acquisition for automatically constructing sememe knowledge bases and representative real-world applications of HowNet.
Introduction: Beyond Atomic Word Units
In traditional natural language processing (NLP), words are often treated as atomic units. While modern word embeddings like word2vec capture semantic relationships through distribution, they frequently struggle with polysemy—where one word has multiple distinct meanings—and low-frequency terms that lack sufficient context.
Existing knowledge bases (KBs) like WordNet address this by organizing words into "synsets" based on differential relationships (e.g., distinguishing meanings by their place in a hierarchy). However, this paper argues for a constructive approach. By defining a closed set of atomic semantic units called sememes, we can represent any word's meaning as a specific combination of these atoms. This "semantic DNA" allows models to understand the internal composition of meaning rather than just external relationships.
The Sememe Framework and HowNet
The primary resource for this research is HowNet, a specialized linguistic knowledge base containing over 100,000 Chinese and English words. In HowNet, each word is mapped to one or more senses, and each sense is further decomposed into a set of sememes.
As illustrated in the figure below, the word "apple" (苹果) is not a single point in semantic space. It branches into distinct senses: a computer brand, a mobile phone, a fruit, and a tree. Each of these senses is defined by a structured tree of sememes. For instance, the "phone" sense is defined by sememes like tool, SpecificBrand, and communicate. This granularity provides a strong inductive bias for neural models, as sememes form a small, closed set (~2,000 units) compared to the infinite nature of vocabulary.
Methods for Sememe-Encoded Learning
The authors propose several novel frameworks to integrate this discrete knowledge into continuous neural architectures:
- SE-WRL (Sememe-Encoded Word Representation Learning): This family of models includes SSA (Simple Sememe Aggregation), SAC (Sememe Attention over Context), and SAT (Sememe Attention over Target). These models use context to dynamically attend to the most relevant sememes of a polysemous word, significantly improving Word Sense Disambiguation (WSD).
- SDLM (Sememe-Driven Language Model): This model predicts the next word by first estimating a distribution over sememes. Using a Product of Experts (PoE) framework, each sememe acts as an "expert" to narrow down the possible word candidates.
- Sememe-Incorporated RNNs: The research explores baking sememes directly into LSTMs and GRUs via specialized output gates or dedicated sememe-RNN cells to manage the flow of linguistic knowledge.
- Automatic Acquisition: To overcome the bottleneck of manual annotation, the authors introduced Character-enhanced Sememe Prediction (CSP), which leverages the internal radical and character information of Chinese words to predict their sememes automatically.
Key Results and Applications
The integration of sememes has yielded impressive results across various NLP benchmarks:
- Sequence Modeling: Sememe-incorporated RNNs outperformed vanilla LSTMs/GRUs in sentiment analysis, natural language inference, and paraphrase detection.
- Reverse Dictionary: The WantWords system, which uses sememes as an internal semantic channel, achieved state-of-the-art results in finding words based on natural language definitions.
- Human Alignment: The automated computation of Semantic Compositionality Degrees (SCD) achieved a 0.74 Spearman correlation with human annotations, suggesting that sememe decomposition closely mirrors human semantic perception.
Limitations and Future Directions
Despite these successes, challenges remain. Most current methods treat sememes as a "bag of labels," often ignoring the rich hierarchical tree structure present in HowNet. Future research aims to better exploit these structural relationships. Additionally, while sememes are excellent for lexical semantics, there is a gap between this linguistic knowledge and the "encyclopedic" world knowledge found in massive pre-trained models like GPT-4. Bridging these two forms of knowledge and expanding sememe resources to more low-resource languages remain high-priority goals.
Summary updated