Abstract
This paper proposes Confusionset-guided Pointer Networks for the Chinese Spell Check (CSC) task. Our approach utilizes an off-the-shelf confusionset to guide character generation. To this end, our novel Seq2Seq model jointly learns to copy a correct character from an input sentence through a pointer network or generate a character from the confusionset rather than the entire vocabulary. We conduct experiments on three human-annotated datasets, and results demonstrate that our proposed generative model outperforms all competitor models by a large margin of up to 20% F1 score, achieving state-of-the-art performance on three datasets.
Introduction
Chinese Spelling Check (CSC) is significantly more complex than English spell-checking. In Chinese, there are no word delimiters (spaces), and errors are almost always context-dependent rather than simple dictionary misses. Most mistakes occur because a writer chooses a character that sounds or looks like the intended one (phonological or visual similarity).
Existing methods often struggle with the massive search space of the Chinese vocabulary—comprising thousands of characters—or treat the problem as a simple sequence labeling task. A key intuition driving this research is that in a misspelled sentence, the vast majority of characters are actually correct. Therefore, an effective model should have a strong inductive bias toward copying the input while being surgically precise about when and how to generate a replacement.
Methods
The proposed Confusionset-guided Pointer Network utilizes a standard Encoder-Decoder (Seq2Seq) architecture with several critical modifications designed for the CSC task.
1. Encoder
The model uses a bidirectional LSTM (BiLSTM) to capture the context of each character in the input sentence, providing a rich representation for the decoder.
2. Decoder with Copy Mechanism
Instead of just generating characters from a vocabulary, the decoder uses a Pointer Network. At each step, it decides whether to copy a character directly from the input (using an attention-based copy mechanism) or generate a new one. This acknowledges that CSC is primarily a "copying" task, allowing the generative component to focus only on the few characters that require correction.
3. Confusionset Guidance
When the model identifies that a character needs replacement, it does not search the entire Chinese vocabulary. Instead, it utilizes a Confusionset matrix ($M$). This matrix acts as a filter, limiting the decoder’s output to only those characters known to be commonly confused with the original input character. This pruning of the search space prevents the model from "hallucinating" irrelevant characters that might fit the local context but lack phonological or visual similarity to the error.
Datasets
The model was trained and evaluated on several key benchmarks:
- Training Data: A large-scale annotated corpus of approximately 281,000 lines, including automatically generated errors and human-annotated data from previous SIGHAN bakes-offs.
- Evaluation Data: Three standard benchmark datasets from the SIGHAN (2013, 2014, and 2015) shared tasks.
- Confusionset: A lexicon of 4,922 characters, with an average of 7.8 confused candidates per character.
Key Results
The proposed model, referred to as Ours+, significantly outperformed existing baselines across all metrics.
- Performance Gain: On the SIGHAN 2013 dataset, the model achieved a 70.1% F1 score at the detection level, outperforming the N-gram baseline (LMC) and standard sequence labeling (SL) approaches.
- Correction Accuracy: The model showed dramatic gains at the correction level, outperforming the LMC baseline by nearly 20% F1 on the SIGHAN 2014 and 2015 datasets.
- Ablation Study: Comparing "Ours-" (no confusionset) to "Ours+" (with confusionset) proved that the confusionset guidance was responsible for a massive jump in precision and overall F1, validating that narrowing the search space is essential for this task.
Conclusion and Future Work
By combining pointer networks with domain-specific confusionsets, this architecture effectively balances the need to preserve correct text with the need to accurately correct errors. However, the current model is designed for 1-to-1 character mapping, meaning it cannot handle errors involving insertions or deletions. Future research may explore incorporating confusionsets into more flexible architectures that can handle varying output lengths.
Summary updated