LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching

Boer Lyu, Lu Chen, Su Zhu, Kai Yu

Abstract

Short Text Matching (STM) is a fundamental task in Natural Language Processing, yet it faces significant hurdles in Chinese due to word polysemy and segmentation sensitivity. We present the Linguistic knowledge Enhanced graph Transformer (LET) framework, which mitigates these issues by incorporating external semantic knowledge from HowNet and structural information from word lattice graphs. By representing sentences as graphs of potential word segmentations and disambiguating meanings through sememe-based attention, LET achieves state-of-the-art performance on major benchmarks, demonstrating particular effectiveness in scenarios with limited contextual information.

Introduction

Chinese Short Text Matching (STM) is critical for applications such as question answering, information retrieval, and dialogue systems. Unlike English, Chinese text lacks explicit word boundaries, making it highly sensitive to Chinese Word Segmentation (CWS) errors. A single mis-segmentation can fundamentally alter the semantic representation of a sentence. Furthermore, the inherent polysemy of Chinese words is exacerbated in short texts, where the lack of surrounding context makes disambiguation difficult for standard models.

Existing approaches typically rely on either character-level representations or a single "best-guess" word segmentation. However, these methods often fail to capture the full semantic richness or the multi-granularity nature of the language. The LET framework addresses these challenges by leveraging all potential word segmentations in a lattice structure and enriching them with "atomic" semantic units, known as sememes, from the HowNet knowledge base.

Methodology: The LET Framework

The LET architecture is designed to be a flexible enhancement for existing models, including pre-trained transformers like BERT. It consists of three primary components:

1. Word Lattice Graph Construction

Rather than committing to a single segmentation, LET constructs a word lattice graph that includes all possible words found in a lexicon. This approach preserves multi-granularity information, allowing the model to consider both individual characters and various overlapping word combinations simultaneously. This structural flexibility makes the model robust against cascading errors from imperfect segmenters.

2. HowNet and Sememe Integration

To resolve polysemy, LET integrates knowledge from HowNet, a comprehensive semantic knowledge base. HowNet decomposes word meanings into sememes—the smallest units of semantic meaning. For instance, the word "Apple" is associated with different sets of sememes depending on whether it refers to a fruit or a technology brand. By incorporating these sememes, the model gains access to explicit semantic features that are often implicit or missing in short contexts.

3. Semantic-aware Graph Transformer (SaGT)

The core of the model is the Semantic-aware Graph Transformer (SaGT). This component employs a graph-based attention mechanism to iteratively update word and sense representations. By considering the surrounding context within the lattice, SaGT dynamically selects the most relevant "sense" for each word and fuses this disambiguated semantic information back into the word embeddings.

The architecture of the LET-BERT model, showing the flow from BERT embeddings through the Semantic-aware Graph Transformer (SaGT) to the final matching prediction.

Experimental Evaluation

The LET framework was evaluated on two prominent Chinese STM benchmarks:

  • LCQMC: A large-scale corpus for open-domain question matching.
  • BQ: A domain-specific dataset focused on banking customer service queries.

Key Results

As shown in the experimental results, LET and its BERT-enhanced version, LET-BERT, consistently outperformed strong baselines. On the BQ dataset, LET-BERT achieved an accuracy of 85.30%, surpassing standard Chinese BERT and other specialized models like ERNIE. On LCQMC, the improvement was even more pronounced, with LET-BERT reaching 88.38% accuracy compared to BERT's 85.73%.

Performance comparison of LET and LET-BERT against various baselines on the BQ and LCQMC datasets.

Ablation studies confirmed that both the word lattice structure and the HowNet semantic information are vital to the model's success. Removing either component resulted in a significant drop in performance across both datasets.

Discussion and Insights

The researchers observed that the benefits of LET are most significant for very short sentences. For texts under 16 characters, the relative error reduction was nearly 10%. This suggests that external linguistic knowledge effectively compensates for the lack of internal contextual clues in short text. Furthermore, the use of sememes allows the model to identify semantic similarities between words that may not share common characters but share underlying concepts (e.g., "fry" and "cook" both sharing a "cooking" sememe).

Conclusion

The LET framework demonstrates that integrating structural lattice information and fine-grained semantic knowledge can significantly enhance Chinese short text matching. While the approach introduces some computational overhead and depends on the coverage of the underlying knowledge base, its ability to disambiguate polysemous words and remain robust to segmentation errors makes it a powerful tool for real-world NLP applications. Future work will explore the scalability of this graph-based approach to longer documents and more diverse knowledge graphs.

Summary updated

Abstract

Chinese short text matching is a fundamental task in natural language processing. Existing approaches usually take Chinese characters or words as input tokens. They have two limitations: 1) Some Chinese words are polysemous, and semantic information is not fully utilized. 2) Some models suffer potential issues caused by word segmentation. Here we introduce HowNet as an external knowledge base and propose a Linguistic knowledge Enhanced graph Transformer (LET) to deal with word ambiguity. Additionally, we adopt the word lattice graph as input to maintain multi-granularity information. Our model is also complementary to pre-trained language models. Experimental results on two Chinese datasets show that our models outperform various typical text matching approaches. Ablation study also indicates that both semantic information and multi-granularity information are important for text matching modeling.
BibTeX
@inproceedings{lyu2021let,
  author = {Boer Lyu and Lu Chen and Su Zhu and Kai Yu},
  title = {{LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i15.17592},
}
Figure