Contextual Similarity is More Valuable than Character Similarity: Curriculum Learning for Chinese Spell Checking

Ding Zhang, Yinghui Li, Qingyu Zhou, Shirong Ma, Yangning Li, Yunbo Cao, Hai-Tao Zheng

Abstract

Chinese Spell Checking (CSC) typically addresses errors involving phonologically or visually similar characters. While traditional methods leverage confusion sets, this research argues that contextual information is a superior signal for error correction. We introduce a model-agnostic Curriculum Learning (CL) framework that prioritizes training samples based on "contextual similarity"—the cosine similarity between representations of erroneous and corrected sentences. By transitioning from easy, contextually-rich samples to harder, ambiguous ones, our framework consistently improves the performance of various CSC models. Experimental results on SIGHAN benchmarks demonstrate that our CL-enhanced MacBERT achieves a state-of-the-art F1 score of 78.4% on SIGHAN15, outperforming previous methods and underscoring the value of context over character appearance.

Introduction

Chinese Spell Checking (CSC) is a critical task in natural language processing, primarily dealing with "confusing characters" that share similar sounds or appearances. Most existing approaches utilize pre-defined confusion sets to guide models. However, these methods often overlook the power of context. For instance, in the phrase "wearing a [wrong character] hat," the surrounding word "hat" strongly implies the correct character "wear" (戴), regardless of the typo's visual or phonetic properties.

Furthermore, standard training regimes often present easy and difficult samples randomly. This paper proposes that models, much like humans, benefit from a structured curriculum that builds foundational knowledge on clear-cut cases before tackling complex ambiguities.

Methodology: The Curriculum Learning Framework

The proposed framework consists of two main components: Difficulty Evaluation and Curriculum Arrangement. This approach is model-agnostic and can be integrated with various encoders such as BERT, MacBERT, or Soft-Masked BERT.

The Curriculum Learning framework, illustrating difficulty evaluation via contextual similarity and the subsequent arrangement of training subsets from easy to difficult.

Difficulty Evaluation

Instead of relying on human-annotated difficulty, the framework uses the model's own internal representations. It calculates the cosine similarity between the contextual embeddings of the source (erroneous) sentence and the target (corrected) sentence. High similarity indicates an "easy" sample where the context strongly supports the correct character, while low similarity denotes a "hard" sample with ambiguous or misleading context.

Curriculum Arrangement

The training data is partitioned into $k$ non-overlapping subsets based on their difficulty scores. During the "annealing" process, the model is initially exposed only to the easiest subsets. As training iterations progress, increasingly difficult subsets are introduced. Finally, the model is fine-tuned on the entire dataset to maintain the original data distribution.

Experimental Results

The framework was evaluated on the standard SIGHAN 13, 14, and 15 benchmarks, using a large-scale pseudo-dataset for initial training volume.

Performance comparison of various CSC methods across SIGHAN datasets. The CL-enhanced models consistently outperform their vanilla counterparts and previous state-of-the-art methods.

As shown in the results table, the CL framework provides a consistent performance boost across all base models. Notably, CL (MacBERT) achieved a correction-level F1 score of 78.4% on SIGHAN15, representing a +3.4% absolute improvement over the standard MacBERT baseline. This performance surpasses other competitive models like REALISE and SpellGCN.

Ablation studies confirmed that the contextual similarity metric (Difficulty Evaluation) is the primary driver of these gains, contributing more to the final performance than the curriculum arrangement alone.

Discussion and Conclusion

The success of this framework highlights that contextual similarity is a more valuable metric for training CSC models than traditional character-level similarity. By mimicking the human learning process, the curriculum approach allows models to better capture the nuances of Chinese text.

However, the method is sensitive to the hyperparameter $k$ (number of subsets); excessively fragmented data can hinder the model's ability to learn effectively. Future work may explore extending this contextual similarity approach to Chinese Grammatical Error Correction (CGEC), where errors involve more complex structural issues. While the initial difficulty evaluation adds a pre-processing step, the resulting gains in accuracy and robustness make it a compelling strategy for advanced spell-checking systems.

Summary updated

Abstract

Chinese Spell Checking (CSC) task aims to detect and correct Chinese spelling errors. Recently, related researches focus on introducing character similarity from confusion set to enhance the CSC models, ignoring the context of characters that contain richer information. To make better use of contextual information, we propose a simple yet effective Curriculum Learning (CL) framework for the CSC task. With the help of our model-agnostic CL framework, existing CSC models will be trained from easy to difficult as humans learn Chinese characters and achieve further performance improvements. Extensive experiments and detailed analyses on widely used SIGHAN datasets show that our method outperforms previous state-of-the-art methods. More instructively, our study empirically suggests that contextual similarity is more valuable than character similarity for the CSC task.
BibTeX
@inproceedings{zhang2023contextual,
  author = {Ding Zhang and Yinghui Li and Qingyu Zhou and Shirong Ma and Yangning Li and Yunbo Cao and Hai-Tao Zheng},
  title = {{Contextual Similarity is More Valuable than Character Similarity: Curriculum Learning for Chinese Spell Checking}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095675},
}
Figure