Abstract
Chinese Spell Checking (CSC) typically addresses errors involving phonologically or visually similar characters. While traditional methods leverage confusion sets, this research argues that contextual information is a superior signal for error correction. We introduce a model-agnostic Curriculum Learning (CL) framework that prioritizes training samples based on "contextual similarity"—the cosine similarity between representations of erroneous and corrected sentences. By transitioning from easy, contextually-rich samples to harder, ambiguous ones, our framework consistently improves the performance of various CSC models. Experimental results on SIGHAN benchmarks demonstrate that our CL-enhanced MacBERT achieves a state-of-the-art F1 score of 78.4% on SIGHAN15, outperforming previous methods and underscoring the value of context over character appearance.
Introduction
Chinese Spell Checking (CSC) is a critical task in natural language processing, primarily dealing with "confusing characters" that share similar sounds or appearances. Most existing approaches utilize pre-defined confusion sets to guide models. However, these methods often overlook the power of context. For instance, in the phrase "wearing a [wrong character] hat," the surrounding word "hat" strongly implies the correct character "wear" (戴), regardless of the typo's visual or phonetic properties.
Furthermore, standard training regimes often present easy and difficult samples randomly. This paper proposes that models, much like humans, benefit from a structured curriculum that builds foundational knowledge on clear-cut cases before tackling complex ambiguities.
Methodology: The Curriculum Learning Framework
The proposed framework consists of two main components: Difficulty Evaluation and Curriculum Arrangement. This approach is model-agnostic and can be integrated with various encoders such as BERT, MacBERT, or Soft-Masked BERT.
Difficulty Evaluation
Instead of relying on human-annotated difficulty, the framework uses the model's own internal representations. It calculates the cosine similarity between the contextual embeddings of the source (erroneous) sentence and the target (corrected) sentence. High similarity indicates an "easy" sample where the context strongly supports the correct character, while low similarity denotes a "hard" sample with ambiguous or misleading context.
Curriculum Arrangement
The training data is partitioned into $k$ non-overlapping subsets based on their difficulty scores. During the "annealing" process, the model is initially exposed only to the easiest subsets. As training iterations progress, increasingly difficult subsets are introduced. Finally, the model is fine-tuned on the entire dataset to maintain the original data distribution.
Experimental Results
The framework was evaluated on the standard SIGHAN 13, 14, and 15 benchmarks, using a large-scale pseudo-dataset for initial training volume.
As shown in the results table, the CL framework provides a consistent performance boost across all base models. Notably, CL (MacBERT) achieved a correction-level F1 score of 78.4% on SIGHAN15, representing a +3.4% absolute improvement over the standard MacBERT baseline. This performance surpasses other competitive models like REALISE and SpellGCN.
Ablation studies confirmed that the contextual similarity metric (Difficulty Evaluation) is the primary driver of these gains, contributing more to the final performance than the curriculum arrangement alone.
Discussion and Conclusion
The success of this framework highlights that contextual similarity is a more valuable metric for training CSC models than traditional character-level similarity. By mimicking the human learning process, the curriculum approach allows models to better capture the nuances of Chinese text.
However, the method is sensitive to the hyperparameter $k$ (number of subsets); excessively fragmented data can hinder the model's ability to learn effectively. Future work may explore extending this contextual similarity approach to Chinese Grammatical Error Correction (CGEC), where errors involve more complex structural issues. While the initial difficulty evaluation adds a pre-processing step, the resulting gains in accuracy and robustness make it a compelling strategy for advanced spell-checking systems.
Summary updated