A Classifier-Based Approach to Preposition and Determiner Error Correction in L2 English

Rachele De Felice, Stephen G. PulmanCOLING 2008

TL;DR

This paper presents a novel classifier-based approach to automatically correct preposition and determiner errors in non-native English writing. By leveraging contextual features and a maximum entropy model trained on native English corpora, the system achieves promising accuracy in identifying and suggesting corrections for these notoriously difficult grammatical elements for L2 learners.

What Problem Does This Paper Address?

Writing in a second language (L2) often presents significant challenges for learners, particularly with grammatical elements that have complex and often idiosyncratic usage rules. Prepositions and determiners are identified as two of the most frequent sources of error for L2 English speakers. Unlike more straightforward grammatical rules, the correct usage of prepositions and determiners is heavily influenced by context, making them difficult to master even for native speakers. Existing NLP tools for error correction often struggle with these specific parts of speech, necessitating the development of specialized approaches that can learn the subtle contextual cues governing their use.

Methods

The core of the proposed approach is a maximum entropy classifier. This statistical model is trained on large corpora of grammatically correct English (L1 data) to learn the patterns of correct preposition and determiner usage. The key innovation lies in the sophisticated feature set used to represent the context surrounding these words. These features go beyond simple word co-occurrence and include:

  • Syntactic analysis: Information derived from a full syntactic parse of the sentence, capturing grammatical relations between words.
  • Lexical information: Word types, WordNet categories, and subcategorization frames.
  • Semantic information: Broad semantic categories for nouns and verbs.
  • Contextual features: Information about surrounding words (POS tags of adjacent words), modifications (adjectives, adverbs, other prepositions), and discourse-level features (e.g., whether a noun has been mentioned previously).

The system first learns models of correct usage from L1 data and then applies these models to detect errors in L2 writing.

Datasets

  • British National Corpus (BNC): Used as the primary dataset for training and evaluating the models on L1 (native English) data.
  • Cambridge Learner Corpus (CLC): A subsection of this corpus was used to evaluate the system's performance on L2 (non-native English) writing. The CLC contains a large collection of learner essays.

Key Insights and Ideas

  • Context is King: The success of the model hinges on its ability to capture rich contextual information, demonstrating that prepositions and determiners are not used in isolation but are deeply embedded within the syntactic and semantic structure of a sentence.
  • L1 Models for L2 Correction: Training models on native speaker data (L1) can yield effective tools for identifying errors in non-native writing, suggesting that L2 errors often represent deviations from established L1 patterns.
  • "Learnability" Varies: Not all prepositions or determiners are equally easy to learn. The model's performance varies significantly across different prepositions, suggesting some are more inherently ambiguous or context-dependent than others.
  • L2 Challenges Beyond Grammar: The performance drop on L2 data is attributed not just to the inherent difficulty of prepositions/determiners but also to the nature of learner text itself, including spelling errors, ungrammaticality, and differences in text type.

Key Results

  • L1 Performance:
    • Prepositions: Achieved an accuracy of 70.06% on L1 data.
  • Determiners: Achieved an accuracy of 92.15% on L1 data.
  • L2 Performance (Initial Results):
    • Prepositions: Accuracy on correct L2 instances was 66.7%, and on incorrect instances (identifying an error) was 70%. However, the accuracy of suggested corrections was only 51.5%.
  • Determiners: Accuracy on L2 data was 92.2% for correct instances, with less than 10% accuracy on incorrect instances.

Limitations and Future Work

  • L2 Correction Accuracy: While the system can detect errors, the accuracy of its suggested corrections for prepositions on L2 data is still relatively low (51.5%), limiting its immediate usefulness.
  • Spelling and Grammar Errors: The presence of spelling mistakes and general ungrammaticality in L2 text significantly hinders the parser and thus the classifier's ability to make correct decisions.
  • Text Type Mismatch: The training data (BNC) may not sufficiently cover topics or lexical items common in learner essays, impacting performance on specific words (e.g., "internet").
  • Future Work:
    • Fine-tuning: Further fine-tuning of the classifier and experimenting with confidence thresholds.
  • L2 Data Preprocessing: Developing methods to factor out spelling and other errors from L2 data before applying the models.
  • Alternative Training Data: Exploring training data that is more similar in text type to learner essays or using alternative data sources.
  • Lexical Resource Creation: Mining the training data to create a lexical resource describing statistical tendencies for prepositions and determiners.
  • Discourse Features: Incorporating discourse-level features to improve determiner selection.

Summary updated

BibTeX
@inproceedings{felice2008classifier,
  author = {Rachele De Felice and Stephen G. Pulman},
  title = {{A Classifier-Based Approach to Preposition and Determiner Error Correction in L2 English}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2008},
}
Figure