Dynamic Connected Networks for Chinese Spelling Check

Baoxin Wang, Wanxiang Che, Dayong Wu, Shijin Wang, Guoping Hu, Ting Liu

Introduction: The Incoherence Problem

Chinese Spelling Check (CSC) is a critical task focused on detecting and correcting errors in Chinese text. Most errors (over 80%) are phonological or visual in nature. While current state-of-the-art models utilize BERT in a non-autoregressive manner, they suffer from a significant flaw: the incoherence problem.

Because these models predict each character independently, they often produce "Frankenstein" words—combinations of characters that are individually plausible but collectively nonsensical. For example, a model might identify that a user intended to write "confused" (糊涂 - hú tu) but, due to independent predictions, output a non-existent word like "尴涂" (gān tú).

Methodology: Dynamic Connected Networks (DCN)

To bridge this gap, the authors introduce the Dynamic Connected Network (DCN). This architecture models dependencies between adjacent characters to ensure linguistic coherence.

The architecture of the Dynamic Connected Network (DCN), illustrating the Pinyin-enhanced generator and the Dynamic Connected Scorer.

1. Pinyin Enhanced Candidate Generator

Instead of relying solely on character embeddings, the DCN incorporates phonetic information. A Convolutional Layer processes Pinyin sequences, allowing the model to "hear" the user's intent. This stage generates the top-$k$ most likely candidates for every position in the sentence.

2. Dynamic Connected Scorer (DCScorer)

The core innovation is the DCScorer, which calculates a "connection score" for every pair of adjacent candidates. Unlike traditional Conditional Random Fields (CRFs) that use a static transition matrix, the DCScorer uses an attention mechanism to compute context-aware transitions on the fly. This allows the model to determine how well two specific characters fit together within the global context of the sentence.

3. Viterbi Decoding

Once candidate and connection scores are established, the model treats the sentence as a pathfinding problem. It employs the Viterbi algorithm to find the single path through the candidates with the highest total score, ensuring the final output is coherent.

A visualization of the DCN decoding process, showing how the model selects the optimal path through candidate characters to reach the target sentence.

Key Results and Insights

The DCN achieved new state-of-the-art results across the industry-standard SIGHAN benchmarks (2013, 2014, and 2015), outperforming models like SpellGCN and Soft-Masked BERT.

  • Performance Gains: On SIGHAN 13, the model achieved a Correction-level F1 of 81.0%, a significant improvement over BERT's 74.6%.
  • Ablation Success: Removing the DCScorer resulted in a nearly 2% drop in F1 score, confirming that modeling character connections is vital for accuracy.
  • Efficiency: By restricting the search space to $k$ candidates and using Viterbi decoding, DCN provides the coherence of autoregressive models (like GPT) while maintaining the speed advantages of non-autoregressive architectures.

Conclusion

The Dynamic Connected Network effectively solves the incoherence problem in Chinese Spelling Check by moving beyond independent character predictions. By calculating dynamic, context-aware transitions and leveraging Pinyin as a strong prior, the model ensures that corrections are not just individually accurate, but collectively meaningful. Future work may involve incorporating visual (glyph) information to better address OCR-related errors and exploring longer-distance dependencies beyond adjacent characters.

Summary updated

BibTeX
@inproceedings{wang2021dynamic,
  author = {Baoxin Wang and Wanxiang Che and Dayong Wu and Shijin Wang and Guoping Hu and Ting Liu},
  title = {{Dynamic Connected Networks for Chinese Spelling Check}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.216},
}
Figure