Abstract
Chinese Spelling Correction (CSC) is an important natural language processing task. Existing methods for CSC mostly utilize BERT models, which select a character from a candidate list to correct errors in the sentence. World knowledge refers to structured information and relationships spanning a wide range of domains and subjects, while definition knowledge pertains to textual explanations or descriptions of specific words or concepts. Both forms of knowledge have the potential to enhance a model's ability to comprehend contextual nuances. As BERT lacks sufficient guidance from world knowledge for error correction and existing models overlook the rich definition knowledge in Chinese dictionaries, the performance of spelling correction models is somewhat compromised. To address these issues, within the world knowledge network, this study injects world knowledge from knowledge graphs into the model to assist in correcting spelling errors caused by a lack of world knowledge. Additionally, the definition knowledge network in this model improves the error correction capability by utilizing the definitions from the Chinese dictionary through a comparative learning approach. Experimental results on the SIGHAN benchmark dataset validate the effectiveness of our approach.
Introduction: The Knowledge Gap in CSC
Chinese Spelling Correction (CSC) is notoriously difficult because many characters share identical pronunciations (homophones) or similar visual structures (homographs). While modern models like BERT are excellent at capturing general linguistic context, they often stumble in two specific scenarios: a lack of world knowledge and an inability to distinguish subtle semantic nuances.
For example, a model might not recognize that "The famous landmark of Egypt is the golden tower" is incorrect because it lacks the factual knowledge that the Pyramids are the actual landmark. Similarly, characters like 主人 (owner) and 主任 (director) sound similar and appear in similar contexts, such as an "office." Without access to formal dictionary definitions, a model struggles to choose the semantically correct term based on the specific role described.
Proposed Framework
To bridge these gaps, this research introduces a three-component architecture designed to teach the model not just how characters look and sound, but what they actually mean in the real world.
1. World Knowledge Network
This module extracts entity-relation triples from a knowledge graph (CN-DBpedia) to build a "knowledge tree." To prevent "knowledge noise"—where irrelevant facts overwhelm the model—the authors implement a Visible Matrix. This matrix acts as a mask, ensuring that branch tokens (external facts) are only visible to their corresponding main tokens in the sentence, preserving the original syntax.
2. Correction Network
The "engine" of the model is a Mask-Transformer Encoder. Unlike standard BERT, it fuses three types of information: character embeddings, glyph embeddings (visual shape), and pinyin embeddings (phonetics). A specialized masking strategy is used during training, where 60% of masked tokens are replaced with phonetically similar characters to better simulate real-world human errors.
3. Definition Knowledge Network
To master semantic nuances, the model employs Contrastive Learning. It pulls the representation of a sentence closer to its correct dictionary definition (positive sample) while pushing it away from definitions of similar-sounding but incorrect characters (negative samples). This provides a semantic "anchor" for high-level reasoning.
Experimental Setup and Datasets
The model was evaluated on the standard SIGHAN 13, 14, and 15 benchmark datasets. The training data combined official SIGHAN sets with a large-scale pseudo-dataset (Wang271K) designed to simulate common errors. CN-DBpedia provided the world knowledge triples, and a structured Chinese dictionary provided the semantic explanations for the contrastive learning module.
Key Insights and Results
The proposed model achieved state-of-the-art results across all three SIGHAN datasets. On SIGHAN 15, the model reached an F1 score of 79.5% at the correction level, significantly outperforming the base BERT model (73.4%) and specialized models like SpellGCN (75.9%).
Ablation studies highlighted the importance of each component:
- Knowledge Depth: The authors found that a knowledge tree depth of 1 is optimal. Increasing depth introduces redundant information that can degrade performance by altering the sentence's original meaning.
- Component Impact: Removing the world knowledge module dropped the F1 score by 0.4%, while removing the definition knowledge module caused a larger drop of 0.7%. This suggests that dictionary definitions are particularly critical for high-precision correction.
- Fusion Effectiveness: Combining glyph and pinyin embeddings proved more effective than using either alone, confirming that Chinese errors are often a mix of visual and phonetic confusion.
Conclusion
By integrating world knowledge and dictionary definitions, this framework significantly enhances the model's ability to comprehend context and correct complex spelling errors. Future work will explore applying these knowledge-enhancement techniques to Chinese Grammar Error Correction (CGEC), addressing more complex structural issues like word order and missing components.
Summary updated