Abstract
This paper presents a novel approach to Grammatical Error Correction (GEC) by leveraging neural sequence-labelling models to re-rank the outputs of traditional Statistical Machine Translation (SMT) systems. By employing a bidirectional LSTM to detect errors at the token level and incorporating character-level gating, the proposed method achieves state-of-the-art results on multiple benchmarks using a minimal set of non-linguistic features.
Introduction
Grammatical Error Correction (GEC) is frequently framed as a translation task: converting ungrammatical "learner" English into correct "native" English. While Statistical Machine Translation (SMT) has historically been the standard approach, it often fails to capture the subtle nuances of grammaticality. Previous attempts to enhance SMT systems involved re-ranking the top $N$ hypotheses using complex, hand-crafted linguistic features such as part-of-speech tags or dependency parses. This work introduces a more streamlined, data-driven approach that utilizes neural networks to score hypotheses based on their likelihood of containing errors.
Neural Sequence-Labelling Architecture
The core of this methodology is a Neural Sequence-Labelling Model designed specifically for error detection rather than direct translation.
- Bi-LSTM Network: The model uses a Bidirectional Long Short-Term Memory (Bi-LSTM) network to predict the probability of each token being "correct" or "incorrect" within its context.
- Character-Level Gating: To address out-of-vocabulary words and misspellings, a character-level Bi-LSTM is integrated. An adaptive gating mechanism combines these character-level insights with standard word embeddings, allowing the model to analyze the internal structure of words.
- Multi-task Learning: The model is simultaneously trained as a Language Model, predicting adjacent words. This auxiliary task improves the internal representation of English grammar.
Re-ranking Strategy
The neural model scores the top 10 hypotheses from an SMT system using three primary features:
- Sentence Probability: The sum of log-probabilities for all tokens being "correct."
- Levenshtein Distance (LD): A measure of the alignment between SMT corrections and the neural model's predicted error map.
- True/False Positives: A ratio comparing the SMT's changes against the neural model's flagged errors.
Experimental Results
The hybrid SMT + Neural Re-ranker was evaluated on several standard GEC benchmarks, including the FCE (exam essays), NUCLE (advanced student essays), and JFLEG (fluency-focused edits).
The system achieved state-of-the-art performance across all test sets. On the FCE dataset, the $F_{0.5}$ score improved from a baseline of 52.90 to 55.60. Significant gains were also observed in the CoNLL 2014 benchmark (37.33 to 42.44) and the JFLEG GLEU score (46.10 to 47.72). Ablation studies confirmed that Sentence Probability was the most influential feature in the re-ranking process.
Discussion and Conclusion
One of the most significant insights of this research is that a model trained purely for error detection can be highly effective at selecting optimal corrections. By identifying which SMT hypothesis contains the fewest remaining predicted errors, the system significantly outperforms raw SMT output. Furthermore, the model's simplicity—relying on only four features—makes it more portable and efficient than previous feature-heavy re-rankers.
However, the system is constrained by the "N-best ceiling"; if the correct correction is not within the SMT's top 10 hypotheses, the re-ranker cannot select it. Future work may involve integrating the neural detection model directly into the SMT decoding process to overcome this limitation and further refine performance on specific error types like contractions and verb forms.
Summary updated