Introduction: The Incoherence Problem
Chinese Spelling Check (CSC) is a critical task focused on detecting and correcting errors in Chinese text. Most errors (over 80%) are phonological or visual in nature. While current state-of-the-art models utilize BERT in a non-autoregressive manner, they suffer from a significant flaw: the incoherence problem.
Because these models predict each character independently, they often produce "Frankenstein" words—combinations of characters that are individually plausible but collectively nonsensical. For example, a model might identify that a user intended to write "confused" (糊涂 - hú tu) but, due to independent predictions, output a non-existent word like "尴涂" (gān tú).
Methodology: Dynamic Connected Networks (DCN)
To bridge this gap, the authors introduce the Dynamic Connected Network (DCN). This architecture models dependencies between adjacent characters to ensure linguistic coherence.
1. Pinyin Enhanced Candidate Generator
Instead of relying solely on character embeddings, the DCN incorporates phonetic information. A Convolutional Layer processes Pinyin sequences, allowing the model to "hear" the user's intent. This stage generates the top-$k$ most likely candidates for every position in the sentence.
2. Dynamic Connected Scorer (DCScorer)
The core innovation is the DCScorer, which calculates a "connection score" for every pair of adjacent candidates. Unlike traditional Conditional Random Fields (CRFs) that use a static transition matrix, the DCScorer uses an attention mechanism to compute context-aware transitions on the fly. This allows the model to determine how well two specific characters fit together within the global context of the sentence.
3. Viterbi Decoding
Once candidate and connection scores are established, the model treats the sentence as a pathfinding problem. It employs the Viterbi algorithm to find the single path through the candidates with the highest total score, ensuring the final output is coherent.
Key Results and Insights
The DCN achieved new state-of-the-art results across the industry-standard SIGHAN benchmarks (2013, 2014, and 2015), outperforming models like SpellGCN and Soft-Masked BERT.
- Performance Gains: On SIGHAN 13, the model achieved a Correction-level F1 of 81.0%, a significant improvement over BERT's 74.6%.
- Ablation Success: Removing the DCScorer resulted in a nearly 2% drop in F1 score, confirming that modeling character connections is vital for accuracy.
- Efficiency: By restricting the search space to $k$ candidates and using Viterbi decoding, DCN provides the coherence of autoregressive models (like GPT) while maintaining the speed advantages of non-autoregressive architectures.
Conclusion
The Dynamic Connected Network effectively solves the incoherence problem in Chinese Spelling Check by moving beyond independent character predictions. By calculating dynamic, context-aware transitions and leveraging Pinyin as a strong prior, the model ensures that corrections are not just individually accurate, but collectively meaningful. Future work may involve incorporating visual (glyph) information to better address OCR-related errors and exploring longer-distance dependencies beyond adjacent characters.
Summary updated