Leveraging What’s Overfixed: Post-Correction via LLM Grammatical Error Overcorrection

Taehee Park, Heejin Do, Gary LeeEMNLP 2025

TL;DR

This paper introduces PoCO, a novel approach to grammatical error correction (GEC) that tackles the precision-recall trade-off. By first leveraging Large Language Models (LLMs) to intentionally overcorrect text (maximizing recall) and then using smaller, fine-tuned models to refine these corrections (improving precision), PoCO achieves a better balance and overall higher quality GEC.

What Problem Does This Paper Address?

Grammatical Error Correction (GEC) is crucial for language learning and writing assistance. Traditional GEC models, often smaller supervised models (sLMs), excel at precision (making correct edits) but struggle with recall (missing potential errors). Conversely, powerful Large Language Models (LLMs) tend to overcorrect, generating many edits but often introducing errors or altering the original meaning, leading to low precision. This paper aims to bridge this gap by combining the strengths of both: the broad error identification of LLMs and the reliability of sLMs. Existing methods to curb LLM overcorrection often sacrifice recall, and simply ensembling models doesn't fully solve the low recall issue inherent in sLMs.

Methods

The core of PoCO is a two-stage process:

  1. Triggering Overcorrection: This stage uses LLMs (specifically GPT-3.5-Turbo-0125 in their experiments) with a modified prompt. The prompt encourages the LLM to "find as many errors as you can," intentionally inducing overcorrection to maximize the chances of identifying all errors (high recall). This is a deliberate step, unlike the accidental overcorrection seen in standard LLM usage.
  2. Post Correction (Refining Overcorrection): The overcorrected output from the LLM is then refined. This is achieved through a novel "double-target" training strategy using fine-tuned sLMs (based on T5). The model is trained on two types of targets:
  3. Gold Target: The original, human-annotated correct sentence. This helps the model learn to correct errors the LLM missed and refine overcorrections.
  4. Recovered Target: A specially constructed target that selectively incorporates correct edits from the LLM's output, but only within the regions the LLM modified. This target helps the model learn to reverse LLM overcorrections while preserving valid edits, thereby improving precision.

PoCO integrates these targets in different ways (e.g., PoCO-Seq, PoCO-Mix) to balance recall and precision.

Datasets

  • Training:
    • Clang-8: Used for pre-training the T5-based models.
  • W&I + LOCNESS: Used for fine-tuning the models with LLM-generated overcorrected outputs.
  • Evaluation:
    • CoNLL-14 test set
    • BEA-19 development and test sets

Key Insights and Ideas

  • Intentional Overcorrection as a Feature: Instead of treating LLM overcorrection as a problem to be solved, PoCO leverages it as a deliberate step to boost recall.
  • The "Recovered Target" is Key: The novel "recovered target" is crucial for improving precision. By teaching the model to specifically undo LLM overcorrections while keeping good edits, it directly addresses the precision drop caused by the first stage.
  • LLMs Alone Struggle with Post-Correction: Experiments showed that even advanced LLMs (GPT-4o-mini, GPT-4-Turbo) struggled to effectively perform post-correction on their own overcorrected outputs, often leading to a drop in precision. This reinforces the need for a hybrid approach.
  • Data Efficiency: PoCO demonstrates strong performance even when fine-tuned on a relatively small dataset (W&I + LOCNESS), suggesting it can achieve good results without requiring massive synthetic data generation.

Key Results

  • PoCO consistently achieved superior recall compared to robust supervised fine-tuned models across all evaluations.
  • It maintained competitive precision on BEA19 development and test sets.
  • PoCO achieved higher F0.5 scores (a harmonic mean of precision and recall) than both supervised models and LLMs, indicating a better overall balance.
  • In comparisons with LLM baselines, PoCO achieved the highest precision on BEA-19 test and dev sets.
  • When integrated into ensemble systems (like ESC), PoCO demonstrated high-performing results, sometimes achieving better performance with fewer models.
  • Reference-free evaluations using GPT-4.1 showed PoCO outputs had high linguistic quality and semantic fidelity, with scores around 4.7-4.9 across dimensions like Fluency, Grammaticality, and Meaning Preservation.

Limitations and Future Work

  • Prompting Optimization: The paper acknowledges that the LLM prompting techniques used were adopted from prior studies and may not be fully optimized for the specific task of mitigating overcorrection. Future work could explore more targeted and adaptive prompting strategies.
  • LLM Overcorrection Nuances: There's a lack of dedicated studies on how different prompting techniques influence LLMs' ability to handle overcorrection. Further research in this area is needed.

The authors suggest that exploring more advanced prompting methods could enhance LLM performance in post-correction tasks, leading to further improvements in GEC.

Summary updated

BibTeX
@inproceedings{park2025leveraging,
  author = {Taehee Park and Heejin Do and Gary Lee},
  title = {{Leveraging What’s Overfixed: Post-Correction via LLM Grammatical Error Overcorrection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure