Adapting Grammatical Error Correction Based on the Native Language of Writers with Neural Network Joint Models

Shamil Chollampatt, Duc Tam Hoang, Hwee Tou NgEMNLP 2016 · OpenReview

Abstract: Shamil Chollampatt, Duc Tam Hoang, Hwee Tou Ng. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.

2016.

Introduction

When learning a second language (L2), a writer's native language (L1) often leaves a distinct "fingerprint" on their grammatical errors. For instance, a Chinese speaker might struggle with verb inflections due to their absence in Chinese, while a Spanish speaker might have specific difficulties with prepositions. Despite these systematic differences, most modern Grammatical Error Correction (GEC) systems are "one-size-fits-all," trained on massive, mixed-background datasets. These systems often fail to capture the subtle, L1-specific error patterns that could lead to more accurate corrections.

The primary challenge in building L1-aware systems is data scarcity. L1-specific datasets are typically too small to train deep learning models from scratch without causing overfitting, where the model "forgets" general grammar rules in favor of the small sample's idiosyncrasies.

Methodology: L1-Aware Adaptation

The researchers proposed a method to adapt a Neural Network Joint Model (NNJM) within a Statistical Machine Translation (SMT) framework. The NNJM predicts the next correct word by considering both the corrected context and the original erroneous source words.

To make the model L1-aware without overfitting, they employed a two-step process:

  1. General Training: The NNJM is first trained on a massive, general corpus of learner text.
  2. KL-Divergence Regularized Adaptation: The model is then fine-tuned on a small set of L1-specific data.

To prevent the model from drifting too far from its general grammatical knowledge, they added a Kullback-Leibler (KL) divergence term to the objective function. This term acts as a "leash," ensuring the adapted model's probability distribution remains anchored to the original general model while still learning specific L1 error patterns.

Figure 1: Effect of KL-divergence regularization strength (λ) on F0.5 scores across different L1 backgrounds. Note how performance peaks with moderate regularization compared to unregularized fine-tuning (λ=0).

Datasets and Evaluation

The study utilized a combination of large-scale and specialized corpora:

  • General Training: The NUCLE (NUS Corpus of Learner English) and Lang-8 corpora, totaling over 2 million sentence pairs.
  • L1-Specific Adaptation: Subsets of the Lang-8 corpus filtered specifically for Chinese, Russian, and Spanish writers.
  • Evaluation: The CLC-FCE (Cambridge English First Certificate) corpus and the CoNLL-2014 shared task benchmark were used to measure performance.

Key Results

The effectiveness of the adaptation was measured using the $F_{0.5}$ score, which prioritizes precision to avoid over-correction. The results demonstrated that L1-adaptation consistently outperforms general-purpose models.

Table 6: Performance comparison of unadapted (Baseline) and L1-adapted GEC systems. The adapted models show significant gains in F0.5 across all three target languages.

As shown in the results, the adapted system achieved significant $F_{0.5}$ gains over the unadapted baseline: +1.27 for Chinese, +0.88 for Russian, and +0.64 for Spanish. Notably, the L1-Chinese adapted model outperformed other state-of-the-art SMT-based systems on the CoNLL-2014 benchmark.

Analysis of error types revealed that the gains were tied to specific linguistic hurdles. For example, adaptation significantly improved the correction of determiners for Chinese and Russian writers, and prepositions for Spanish writers, reflecting the unique challenges these learners face.

Conclusion and Future Work

This research proves that even small amounts of L1-specific data (as few as 50–100 essays) can significantly improve GEC performance when combined with proper regularization. The adaptation process is also computationally efficient, taking only 20–30 minutes on a GPU.

However, the current system depends on knowing the writer's L1 in advance. Future work could involve integrating automatic Native Language Identification (NLI) to make the system more versatile for anonymous users. Additionally, while this study focused on an SMT framework, the principle of L1-adaptation via KL-divergence regularization remains highly relevant for modern Transformer-based architectures.

Summary updated

BibTeX
@inproceedings{chollampatt2016adapting,
  author = {Shamil Chollampatt and Duc Tam Hoang and Hwee Tou Ng},
  title = {{Adapting Grammatical Error Correction Based on the Native Language of Writers with Neural Network Joint Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  url = {https://openreview.net/forum?id=2119HwDYJZB},
}
Figure