Abstract
Grammatical Error Correction (GEC) has traditionally relied on either massive synthetic datasets or hand-crafted linguistic rules. While synthetic data approaches are effective, they are computationally expensive to train. This paper introduces AMR-GEC, a novel approach that incorporates Abstract Meaning Representation (AMR) to provide models with a semantic map of the input sentence. By utilizing a denoising mechanism to mitigate the impact of unreliable graphs generated from erroneous text, the proposed model achieves performance comparable to state-of-the-art systems while reducing training time by 32% compared to standard T5-based synthetic data approaches.
Introduction
In the field of Grammatical Error Correction (GEC), the primary objective is to transform an error-prone sentence into a grammatically correct version while strictly preserving its original meaning. Modern high-performing models typically achieve this through "brute force"—training on millions of synthetic sentence pairs to learn error patterns. However, this reliance on massive data leads to significant computational overhead.
An alternative is "knowledge enhancement," where models are supplemented with linguistic structures like dependency trees. This research proposes that Abstract Meaning Representation (AMR)—a rooted, labeled graph representing semantic concepts ("who is doing what to whom")—is a superior source of knowledge because it focuses on core semantics rather than surface-level syntax. The challenge lies in the "garbage in, garbage out" problem: automated AMR parsers often produce noisy or incorrect graphs when processing ungrammatical text. AMR-GEC addresses this through a specialized denoising framework.
Methodology
The AMR-GEC architecture is a Denoising Semantic Aggregated GEC Model consisting of three core components:
1. Sequence-AMR Graph Construction
Rather than treating text and graphs as separate entities, the model constructs a unified structure that fuses the word sequence with the AMR semantic graph. The researchers utilize the JAMR alignment tool to map specific words in the sentence to their corresponding nodes in the semantic graph, ensuring the model maintains a clear link between surface tokens and abstract concepts.
2. Graph Aggregation Encoder
To process this fused structure, a Graph Neural Network (GNN) is integrated on top of a standard Transformer encoder. This GNN updates word representations based on their neighbors within the semantic graph. This allows the model to effectively "see" long-distance relationships—such as a subject and verb separated by a lengthy parenthetical clause—which are often difficult for standard sequence models to resolve.
3. Denoising Function
To prevent the model from over-relying on potentially flawed AMR graphs generated from erroneous input, the authors implement two masking strategies during training:
- Node/Edge Level Mask: Random noise is added to nodes or edges within the graph.
- Subgraph Level Mask: Entire sections of the graph are randomly removed.
These strategies force the model to remain robust, learning to identify and utilize only the most helpful parts of the semantic structure while ignoring parser-induced noise.
Experimental Setup
The model was evaluated on two benchmark datasets:
- CoNLL-2014: A standard shared task dataset for GEC.
- BEA-2019: A diverse dataset including the "Write & Improve" and LOCNESS corpora.
The researchers used the SPRING parser to generate AMRs and JAMR for word-to-node alignment during the pre-processing phase.
Results and Discussion
Efficiency and Performance
AMR-GEC demonstrates that semantics can serve as a computational shortcut. Compared to a T5 model trained on massive synthetic data, AMR-GEC reduced training time by 32% while maintaining comparable inference speeds. On the BEA-2019 test set, the single AMR-GEC model achieved an $F_{0.5}$ score of 68.4, outperforming several strong baselines that rely heavily on synthetic data.
Error-Specific Improvements
The inclusion of AMR proved particularly beneficial for specific error types. The model showed significant gains in Punctuation (PUNCT) and Prepositions (PREP), with F-score increases of 4–6 points over a standard T5-GEC baseline. This suggests that the semantic clarity provided by AMR helps the model reconstruct correct sentence structures where surface-level patterns might be ambiguous.
Handling Long-Distance Dependencies
Standard sequence models often struggle when an error's correction depends on a word located far away in the sentence. Because AMR explicitly connects related concepts with edges regardless of their linear distance in the text, it provides a structural shortcut that allows the model to resolve these dependencies more accurately.
Limitations and Future Work
Despite its success, the system's performance remains tied to the quality of the external AMR parser (SPRING). If the parser fails to generate a coherent graph, the GEC model's primary advantage is diminished. Additionally, while the single model is highly effective, the authors noted that more sophisticated ensemble strategies, such as R2L reranking, could further boost performance.
Currently, the research is focused on English. However, since AMR is designed to be largely language-agnostic, extending this denoising semantic framework to low-resource languages represents a promising avenue for future study.
Conclusion
By bridging the gap between semantic representation and sequence-to-sequence learning, AMR-GEC offers a more efficient path to high-quality grammatical error correction. The introduction of denoising strategies successfully mitigates the inherent noise in parsing ungrammatical text, proving that deep semantic knowledge can reduce the industry's reliance on massive, computationally expensive synthetic datasets.
Summary updated