Overview
Researchers from Shanghai Jiao Tong University have introduced LET (Linguistic knowledge Enhanced graph Transformer), a model that tackles the dual challenges of word ambiguity and segmentation errors in Chinese short text matching (STM). By combining word lattice graphs with semantic knowledge from HowNet, LET significantly outperforms standard BERT-based models, especially on short sentences where context is scarce.
The Challenge of Chinese Short Text Matching
Short Text Matching is fundamental to NLP applications like chatbots and search engines. In Chinese, it faces two major hurdles:
- Word Segmentation Sensitivity: Chinese lacks spaces between words. If a segmenter incorrectly splits a word—for example, splitting "ChuiNiu" (bragging) into "Chui" (blowing) and "Niu" (cattle)—the original meaning is lost.
- Polysemy & Lack of Context: Many Chinese words have multiple meanings. In short texts, there is often insufficient context to resolve this ambiguity without external knowledge.
Methodology: The LET Framework
The LET model addresses these issues through a graph-based approach that considers multiple segmentation possibilities and external semantic definitions simultaneously.
Word Lattice Graph and HowNet Integration
Instead of committing to a single segmentation, LET utilizes a word lattice graph containing all potential words identified by various tools. To handle polysemy, it integrates HowNet, a knowledge base that defines words using "sememes"—atomic units of semantic meaning (e.g., "Apple" linked to "Fruit" or "Brand").
Semantic-aware Graph Transformer (SaGT)
The core engine is the SaGT, which uses Multi-Dimensional Graph Attention (MD-GAT) to iteratively update representations:
- Sense Updating: Updates a word's "sense" based on surrounding lattice context.
- Word Updating: Refines word representations using the most relevant senses.
LET can be integrated with pre-trained models like BERT, resulting in the hybrid LET-BERT.
Experimental Results
The model was evaluated on the LCQMC (open-domain) and BQ (banking domain) benchmarks. LET-BERT consistently outperformed vanilla BERT and other knowledge-enhanced variants.
| Dataset | Model | Accuracy (ACC) | F1 Score |
|---|---|---|---|
| LCQMC | BERT | 85.73% | 86.86% |
| LET-BERT | 88.38% | 88.85% | |
| BQ | BERT | 84.50% | 84.00% |
| LET-BERT | 85.30% | 84.98% |
Key Insights
- Short Text Advantage: External knowledge is most impactful for short sentences (<16 characters), where LET achieved a 9.63% Relative Error Reduction (RER) compared to models without sense information.
- Robustness: By using a lattice of all possible words, the model remains effective even if a specific segmenter makes an error, as the correct word is likely still present in the graph for the attention mechanism to identify.
Conclusion
LET provides a robust mechanism for incorporating linguistic knowledge into Chinese STM. By leveraging sememes and word lattices, it effectively mitigates the risks of segmentation errors and polysemy, setting a new standard for short text matching performance.
Summary updated