Introduction
Grammatical Error Correction (GEC) is a cornerstone of natural language processing, yet most existing systems are optimized for general-domain texts like student essays. Researchers from the National University of Singapore (NUS) addressed this limitation by participating in the Helping Our Own (HOO) 2011 Pilot Shared Task. This challenge focused on a highly specialized niche: technical papers written by non-native authors within the NLP community itself. Academic writing presents unique hurdles, including specialized jargon and complex syntactic structures, which the NUS team tackled using a sequential pipeline targeting spelling, articles, and prepositions.
Methodology: The Sequential Pipeline
The NUS system employs a three-stage sequential architecture, where each stage refines the output of the previous one. A critical "Sanity Filter" using a 5-gram Language Model (LM) trained on the Google Web 1T corpus governs every stage. The system only accepts a proposed correction if it increases the sentence's normalized probability score, ensuring a conservative approach that prioritizes precision.
1. Spelling Correction
The pipeline begins with Aspell, an open-source spellchecker. To adapt it to the technical domain, the authors integrated a custom dictionary derived from the ACL Anthology. This prevents the system from erroneously "correcting" domain-specific terms like "lemmatization."
2. Article Correction
Article usage (a, an, the, or null) is modeled as a multi-class classification problem. For every noun phrase, a linear classifier examines contextual features—such as surrounding words, part-of-speech tags, and head words—to predict the appropriate article. Notably, the "null" article is treated as a valid class, allowing the system to suggest deletions.
3. Preposition Correction
Similar to articles, preposition correction uses a classifier to select from 11 common candidates (e.g., at, in, on, to, with). Currently, the system focuses on substitutions rather than insertions or deletions of prepositions.
Data Resources
The system was developed and evaluated using a mix of shared task data and large-scale external corpora. The ACL Anthology was instrumental in providing in-domain vocabulary, while the CL-JOURNAL dataset provided high-quality training examples for the classifiers.
Results and Analysis
Performance was measured using $F_1$ scores across three categories: Detection (identifying an error exists), Recognition (identifying the error type), and Correction (fixing the error). The evaluation considered both strict matches and "bonus" matches where multiple corrections might be valid.
The results on the HOO-TEST set demonstrate the cumulative benefit of the pipeline. As shown in the performance data, the inclusion of article correction provided the most significant performance boost, raising the Correction $F_1$ score from 0.1611 (after spelling) to 0.2209. The final pipeline, including preposition substitution, achieved a Correction $F_1$ of 0.2274 and a Detection $F_1$ of 0.2840.
While the system showed steady improvement, prepositions remained the most challenging category. Their idiomatic nature makes them difficult to correct through classification alone. Furthermore, the sequential nature of the pipeline means that errors in early stages (like spelling) can propagate and negatively impact subsequent article or preposition classification.
Conclusion
The NUS approach highlights the importance of in-domain knowledge and conservative correction strategies in specialized GEC tasks. By leveraging the ACL Anthology and a robust language model filter, the system effectively targets common errors in academic writing. Future work will focus on expanding the scope of preposition correction to include insertions and deletions, as well as optimizing confidence thresholds to better balance error detection and false positives.
Summary updated