Correcting Semantic Collocation Errors with L1-induced Paraphrases

Daniel Dahlmeier, Hwee Tou NgEMNLP 2011 · OpenReview

Abstract

We present a novel approach for automatic collocation error correction in learner English which is based on paraphrases extracted from parallel corpora. Our key assumption is that collocation errors are often caused by semantic similarity in the first language (L1-language) of the writer. An analysis of a large corpus of annotated learner English confirms this assumption. We evaluate our approach on real-world learner data and show that L1-induced paraphrases outperform traditional approaches based on edit distance, homophones, and WordNet synonyms.

Introduction

Collocations are groups of words that naturally co-occur in a language, such as "strong tea" or "watch a movie." For English as a Foreign Language (EFL) students, mastering these combinations is a significant challenge. Traditional grammar checkers often fail to address these errors because they focus on orthographic or phonetic similarities (e.g., spelling or homophones). However, many collocation errors are semantic in nature, stemming from "L1-transfer," where a learner applies the semantic mappings of their native language to English. For instance, a Chinese speaker might say "look movies" because the Chinese word kàn (看) encompasses looking, watching, and reading.

Methodology

The proposed system treats collocation correction as a specialized translation task within a Statistical Machine Translation (SMT) framework.

  1. L1-induced Paraphrases: Using a Chinese-English parallel corpus (FBIS), the researchers identify English words that share a common Chinese translation. These are treated as semantic paraphrases.
  2. Pivot Technique: The probability of one English phrase being a valid substitute for another is calculated by "pivoting" through the shared L1 translation.
  3. SMT Decoding: The system utilizes the Moses decoder. Unlike standard translation, the decoder is constrained to find the best correction for a specific erroneous phrase while keeping the rest of the sentence fixed.
  4. Feature Integration: The model combines L1-induced paraphrases with traditional features like edit distance, homophones, and WordNet synonyms into a log-linear model.

Datasets

The study utilizes the NUCLE (NUS Corpus of Learner English), a one-million-word corpus of essays by native Chinese speakers, annotated with error tags. The FBIS corpus provides the parallel data for paraphrase extraction, and the English Gigaword corpus is used to train a 5-gram language model for fluency assessment.

Analysis of Error Sources

Analysis of the NUCLE corpus confirms that L1-transfer is the primary driver of collocation errors. As shown in the table below, L1-transfer errors significantly outnumber those caused by spelling, homophones, or synonyms. This supports the hypothesis that learners often "think" in their native language and map those concepts directly onto English.

Distribution of suspected error sources in the NUCLE corpus.

Evaluation and Results

The L1-induced system was compared against a baseline of traditional correction methods. In automatic evaluations using Mean Reciprocal Rank (MRR), the "All" system (incorporating L1 paraphrases) achieved a score of 17.21%, more than doubling the baseline's 7.61%.

Performance comparison of different models on collocation error correction.

Human evaluation further validated these results, with the "All" system reaching a Precision@1 of 38.20%. The L1-induced paraphrases were able to correct 906 errors that were otherwise undetectable by spelling or synonym-based checkers.

Conclusion and Future Work

While the L1-induced approach shows great promise, challenges remain. Domain mismatches between training data (news) and test data (essays) can lead to overly formal suggestions. Additionally, the system currently assumes errors have already been identified. Future work will focus on integrating error detection and expanding the vocabulary coverage using larger parallel corpora.

Summary updated

Abstract

We present a novel approach for automatic collocation error correction in learner English which is based on paraphrases extracted from parallel corpora. Our key assumption is that collocation errors are often caused by semantic similarity in the first language (L1-language) of the writer. An analysis of a large corpus of annotated learner English confirms this assumption. We evaluate our approach on real-world learner data and show that L1-induced paraphrases outperform traditional approaches based on edit distance, homophones, and WordNet synonyms.
BibTeX
@inproceedings{dahlmeier2011correcting,
  author = {Daniel Dahlmeier and Hwee Tou Ng},
  title = {{Correcting Semantic Collocation Errors with L1-induced Paraphrases}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
  url = {https://openreview.net/forum?id=HyZx8zzdbS},
}
Figure