Abstract
Short Text Matching (STM) is a fundamental task in Natural Language Processing, yet it faces significant hurdles in Chinese due to word polysemy and segmentation sensitivity. We present the Linguistic knowledge Enhanced graph Transformer (LET) framework, which mitigates these issues by incorporating external semantic knowledge from HowNet and structural information from word lattice graphs. By representing sentences as graphs of potential word segmentations and disambiguating meanings through sememe-based attention, LET achieves state-of-the-art performance on major benchmarks, demonstrating particular effectiveness in scenarios with limited contextual information.
Introduction
Chinese Short Text Matching (STM) is critical for applications such as question answering, information retrieval, and dialogue systems. Unlike English, Chinese text lacks explicit word boundaries, making it highly sensitive to Chinese Word Segmentation (CWS) errors. A single mis-segmentation can fundamentally alter the semantic representation of a sentence. Furthermore, the inherent polysemy of Chinese words is exacerbated in short texts, where the lack of surrounding context makes disambiguation difficult for standard models.
Existing approaches typically rely on either character-level representations or a single "best-guess" word segmentation. However, these methods often fail to capture the full semantic richness or the multi-granularity nature of the language. The LET framework addresses these challenges by leveraging all potential word segmentations in a lattice structure and enriching them with "atomic" semantic units, known as sememes, from the HowNet knowledge base.
Methodology: The LET Framework
The LET architecture is designed to be a flexible enhancement for existing models, including pre-trained transformers like BERT. It consists of three primary components:
1. Word Lattice Graph Construction
Rather than committing to a single segmentation, LET constructs a word lattice graph that includes all possible words found in a lexicon. This approach preserves multi-granularity information, allowing the model to consider both individual characters and various overlapping word combinations simultaneously. This structural flexibility makes the model robust against cascading errors from imperfect segmenters.
2. HowNet and Sememe Integration
To resolve polysemy, LET integrates knowledge from HowNet, a comprehensive semantic knowledge base. HowNet decomposes word meanings into sememes—the smallest units of semantic meaning. For instance, the word "Apple" is associated with different sets of sememes depending on whether it refers to a fruit or a technology brand. By incorporating these sememes, the model gains access to explicit semantic features that are often implicit or missing in short contexts.
3. Semantic-aware Graph Transformer (SaGT)
The core of the model is the Semantic-aware Graph Transformer (SaGT). This component employs a graph-based attention mechanism to iteratively update word and sense representations. By considering the surrounding context within the lattice, SaGT dynamically selects the most relevant "sense" for each word and fuses this disambiguated semantic information back into the word embeddings.
Experimental Evaluation
The LET framework was evaluated on two prominent Chinese STM benchmarks:
- LCQMC: A large-scale corpus for open-domain question matching.
- BQ: A domain-specific dataset focused on banking customer service queries.
Key Results
As shown in the experimental results, LET and its BERT-enhanced version, LET-BERT, consistently outperformed strong baselines. On the BQ dataset, LET-BERT achieved an accuracy of 85.30%, surpassing standard Chinese BERT and other specialized models like ERNIE. On LCQMC, the improvement was even more pronounced, with LET-BERT reaching 88.38% accuracy compared to BERT's 85.73%.
Ablation studies confirmed that both the word lattice structure and the HowNet semantic information are vital to the model's success. Removing either component resulted in a significant drop in performance across both datasets.
Discussion and Insights
The researchers observed that the benefits of LET are most significant for very short sentences. For texts under 16 characters, the relative error reduction was nearly 10%. This suggests that external linguistic knowledge effectively compensates for the lack of internal contextual clues in short text. Furthermore, the use of sememes allows the model to identify semantic similarities between words that may not share common characters but share underlying concepts (e.g., "fry" and "cook" both sharing a "cooking" sememe).
Conclusion
The LET framework demonstrates that integrating structural lattice information and fine-grained semantic knowledge can significantly enhance Chinese short text matching. While the approach introduces some computational overhead and depends on the coverage of the underlying knowledge base, its ability to disambiguate polysemous words and remain robust to segmentation errors makes it a powerful tool for real-world NLP applications. Future work will explore the scalability of this graph-based approach to longer documents and more diverse knowledge graphs.
Summary updated