VisCGEC: Benchmarking the Visual Chinese Grammatical Error Correction

Xiaoman Wang, Dan Yuan, Xin Liu, Yike Zhao, Xiaoxiao Zhang, Xizhi Chen, Yunshi LanNAACL 2025 · DOI

Introduction

Traditional Chinese Grammatical Error Correction (CGEC) has long relied on clean, digital text. However, for learners of Chinese as a Foreign Language (CFL), the primary mode of practice is handwriting. This introduces unique challenges that text-only models cannot address, such as visual confusion between similar characters and the creation of "faked" characters—non-existent symbols resulting from stroke errors. This paper introduces VisCGEC, the first benchmark specifically designed for Visual Chinese Grammatical Error Correction.

The VisCGEC Dataset

The core of this research is a meticulously curated dataset consisting of 2,451 images of handwritten sentences. These were collected from 581 essays written by 303 CFL learners across 39 countries, spanning proficiency levels from HSK4 to HSK6.

Example of VisCGEC dataset annotation, showing bounding boxes, source text, and target corrections.

Unlike previous datasets, VisCGEC includes detailed annotations: bounding boxes for every character, the recognized (often erroneous) text, and the corrected target text. A significant finding is that approximately 14% of sentences contain "faked" characters, which pose a major hurdle for standard OCR systems that tend to "auto-correct" them into valid but incorrect characters.

Methodology

The authors formalize the Visual CGEC task as generating a corrected text sequence $T$ from an input image $I$. They evaluated two main architectural frameworks:

  1. Two-stage Pipeline: This approach first performs character recognition (using standard OCR or CLIP-based visual matching) and then passes the text to a correction model (e.g., GECToR, SynGEC, or a fine-tuned Qwen2-7B LLM).
  2. End-to-End Systems: This utilizes Multimodal Large Language Models (MLLMs) like GPT-4o and Qwen-VL to process the image and output the correction in a single step.
The Visual CGEC workflow, illustrating the transition from handwritten input to corrected text output.

Key Insights and Results

The study reveals a substantial performance gap in current AI capabilities. The best-performing model—a two-stage pipeline using OCR and a fine-tuned Qwen2-7B—achieved an $F_{0.5}$ score of only 28.59%.

Several critical insights emerged:

  • OCR Limitations: While OCR is highly accurate (96.77%), its tendency to map "faked" characters to valid ones strips away visual cues essential for grammatical correction.
  • LLM Superiority: Fine-tuned LLMs outperformed specialized GEC models, demonstrating better robustness to the "noisy" input characteristic of language learners.
  • MLLM Challenges: Surprisingly, end-to-end MLLMs like GPT-4o performed poorly, likely due to a lack of domain-specific fine-tuning on handwritten Chinese.

Conclusion

VisCGEC highlights the immense difficulty of multimodal error correction in handwriting. Future work must focus on better multimodal fusion strategies where visual stroke features directly inform the grammatical correction process, rather than relying on a lossy intermediate OCR step.

Summary updated

BibTeX
@inproceedings{wang2025viscgec,
  author = {Xiaoman Wang and Dan Yuan and Xin Liu and Yike Zhao and Xiaoxiao Zhang and Xizhi Chen and Yunshi Lan},
  title = {{VisCGEC: Benchmarking the Visual Chinese Grammatical Error Correction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.261},
}
Figure