A Chinese Writing Correction System for Learning Chinese as a Foreign Language

Yow-Ting Shiue, Hen-Hsen Huang, Hsin-Hsi ChenCOLING 2018

TL;DR

This paper introduces the first neural machine translation (NMT)-based system for correcting grammatical errors in Chinese written by foreign language learners. Beyond just providing corrections, it also offers relevant example sentences with English translations to aid understanding, addressing a significant gap in readily available Chinese GEC tools.

What Problem Does This Paper Address?

Learning Chinese as a foreign language presents unique challenges, particularly in writing. While English has seen significant advancements in grammatical error correction (GEC) tools, readily usable and effective systems for Chinese are scarce. Existing Chinese GEC efforts often focus on specific error types or detection rather than comprehensive correction. This paper aims to bridge this gap by developing a system that not only identifies and corrects errors but also helps learners understand why a correction is needed and how to express themselves correctly, drawing inspiration from successful NMT approaches in English GEC.

Methods

The core of the system is a Neural Machine Translation (NMT) model that treats Chinese error correction as a translation task: transforming erroneous Chinese sentences into well-formed ones. They leverage an encoder-decoder architecture, specifically a bidirectional Long-short Term Memory (LSTM) encoder and a two-layer LSTM decoder, with global attention. A key design choice was to use a word-based NMT model over a character-based one, finding that word-based models achieved better performance despite potential segmentation errors in learner text. This is attributed to the decoder's ability to "complete" sentences based on partial source information.

Another crucial aspect is the use of pre-trained word embeddings. To overcome the limited availability of parallel Chinese learner data, they initialize their NMT model with word vectors pre-trained on a large corpus of well-formed Chinese text. Experiments showed that using pre-trained embeddings significantly improves performance, with Struct-SG embeddings yielding the best results.

Beyond correction, the system incorporates an Example Sentence Retrieval module. This module retrieves ten example Chinese sentences from a large corpus (UM-Corpus) that are semantically related to the user's input. These examples are ranked using character bigram overlaps to ensure relevance to the words and grammar patterns in the input sentence.

Datasets

The system was trained and evaluated using datasets from the NLPTEA 14-17 CGED shared tasks. The training data comprised approximately 38,554 sentences from NLPTEA 14-17. For validation and testing, they used the test data from NLPTEA 14 (1,783 sentences) and NLPTEA 15 (1,000 sentences). They note that NLPTEA 16 and 17 test data were not used as they only provided error type labels, not corrections. The example sentence retrieval module utilizes the UM-Corpus, a large English-Chinese parallel corpus.

Key Insights and Ideas

  • NMT for Chinese GEC is viable and effective: The paper demonstrates that NMT, a powerful translation paradigm, can be successfully adapted for Chinese grammatical error correction, offering a more robust approach than classification-based methods.
  • Word-based NMT outperforms character-based for Chinese GEC: This is a significant finding, as it suggests that even with imperfect word segmentation in learner text, the NMT decoder can effectively learn to correct errors.
  • Pre-trained embeddings are crucial for low-resource scenarios: The substantial performance gains from using pre-trained word embeddings highlight their importance when parallel learner data is scarce, a common issue in GEC research.
  • Example sentences enhance learning: Providing relevant example sentences alongside corrections offers a valuable pedagogical component, helping learners understand the nuances of correct usage beyond just fixing mistakes.

Key Results

The word-based NMT model with pre-trained Struct-SG embeddings achieved the best performance. On the NLPTEA 15 CGED shared task test set, the system reported an accuracy of 0.232 and a char. GLEU of 0.668, with a word GLEU of 0.580. These results represent a significant improvement over random embeddings and a baseline character-based model. The example sentence retrieval system successfully retrieves relevant examples, as demonstrated by the example where sentences using "每個月" (every month) were correctly identified for an input containing that phrase.

Limitations and Future Work

The authors acknowledge that the evaluation metric (exact match to ground-truth) can be strict, potentially penalizing minor variations that are still correct. They also report the GLEU metric, which is more forgiving of minor differences.

Future work could involve:

  • Further research on the framework to improve performance.
  • Leveraging the web interface to facilitate user evaluation and gather more data for incremental system improvement.
  • Exploring more advanced retrieval models that consider semantic similarities beyond surface-form overlaps.

Summary updated

BibTeX
@inproceedings{shiue2018chinese,
  author = {Yow-Ting Shiue and Hen-Hsen Huang and Hsin-Hsi Chen},
  title = {{A Chinese Writing Correction System for Learning Chinese as a Foreign Language}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure