Abstract
Standard Grammatical Error Correction (GEC) models, typically based on autoregressive sequence-to-sequence (seq2seq) architectures, suffer from significant efficiency bottlenecks due to redundant token generation. Most source sentences are largely correct, yet models spend considerable computation copying correct text. We present a two-stage approach: Erroneous Span Detection (ESD) and Erroneous Span Correction (ESC). By identifying specific error spans and generating corrections only for those segments, our method reduces inference time by over 50% while maintaining or exceeding state-of-the-art accuracy on standard benchmarks.
Introduction
The current gold standard for GEC involves seq2seq models, usually Transformer-based. However, these models are autoregressive, generating the entire output token-by-token. In practice, the vast majority of a source sentence is grammatically correct. The model's "thinking time" is largely wasted on copying correct text from source to target. This makes real-time deployment in applications like word processors or chat apps computationally expensive and slow.
Methodology: The Detect-then-Edit Pipeline
We propose a pipeline that decouples error detection from correction, utilizing two specialized components:
Erroneous Span Detection (ESD)
Instead of a heavy seq2seq model, we employ an efficient sequence tagging model (fine-tuned from RoBERTa or BERT). This model performs binary classification on every token: 0 for correct and 1 for incorrect. This stage identifies the specific "spans" requiring modification.
Erroneous Span Correction (ESC)
The ESC is a Transformer-big seq2seq model that receives the original sentence with detected spans marked by special tags (e.g., <s1> erroneous text </s1>). Crucially, the decoder is trained to output only the corrected text for these specific spans, bypassing the need to rewrite the entire sentence.
To ensure robustness against imperfect detections, we utilize a training strategy inspired by SpanBERT. During training, we randomly select spans to "correct," teaching the model to handle various boundaries and even validate spans that may already be correct.
Experimental Setup and Datasets
The approach was evaluated on English and Chinese benchmarks:
- English: BEA-2019 and CoNLL-2014 test sets, with training data from FCE, Lang-8, NUCLE, and W&I+LOCNESS.
- Chinese: NLPCC-2018 shared task dataset.
- Pre-training: We synthesized a dataset of 260 million sentence pairs using back-translation and sentence corruption to achieve state-of-the-art performance.
Results and Analysis
Efficiency Gains
The "copying" bottleneck was significantly reduced. For 872 incorrect sentences, a standard model required 21,065 decoding steps, whereas our ESC model required only 7,647—a nearly 3x reduction in the most expensive part of the pipeline. Overall, this translates to a 50% reduction in total inference time across various batch sizes.
Performance Comparison
Despite the speedup, the model remains highly competitive. On the BEA-19 benchmark, our pre-trained model achieved an $F_{0.5}$ score of 66.9, outperforming the standard seq2seq baseline of 65.5. The ESD model also acts as a high-speed gatekeeper; if no errors are detected, the system bypasses the heavy ESC model entirely, saving massive amounts of compute on clean text.
Controllability
The ESD model outputs probabilities, allowing for an adjustable detection threshold. Users can tune the system to be "conservative" (higher precision) or "aggressive" (higher recall), a level of control difficult to achieve with standard black-box seq2seq models.
Conclusion
Our two-stage GEC framework addresses the efficiency bottleneck of traditional models by focusing computation only on erroneous spans. This approach is language-independent and offers a significant speedup without sacrificing accuracy. Future work includes exploring model distillation for the ESC component and applying span-specific decoding to other text-rewriting tasks like simplification or style transfer.
Summary updated