Negation Scope Resolution for Chinese as a Second Language

Mengyu Zhang, Weiqi Wang, Shuqiao Sun, Weiwei SunWorkshop 2021

TL;DR

This paper introduces the first corpus and neural system for Negation Scope Resolution (NSR) in Chinese as a Second Language (CSL). They found that learners' native language (English vs. Japanese) significantly impacts both annotation difficulty and model performance, with Japanese learners' data being harder to annotate but leading to better model transfer.

What Problem Does This Paper Address?

Automatically understanding natural language is crucial, and a key component is negation. Negation Scope Resolution (NSR) aims to identify the part of a sentence that a negation word (like "not") affects. While NSR has been studied for native languages, processing second language (L2) data presents unique challenges. L2 language systems are distinct from both the native (L1) and target languages, meaning standard NLP models often fall short. This paper tackles the problem of NSR specifically for Chinese as a Second Language (CSL), recognizing that learners' native language backgrounds likely influence how they express negation and how models can learn from their language.

Methods

The authors developed a novel neural NSR system. Their approach is based on sequence labeling, where each word in a sentence is assigned a label indicating whether it falls within the scope of a negation cue. They leverage pre-trained language models like BERT and ELMo for word embeddings, combined with embeddings for Part-of-Speech (PoS) tags and the negation cue itself. These combined features are fed into a BiLSTM encoder, followed by a feedforward layer and a softmax classifier to predict the scope for each word. They also conducted a thorough error analysis to understand the linguistic reasons behind model performance variations.

Datasets

The core contribution is a newly annotated moderate-sized corpus for CSL. This corpus contains approximately 8,000 sentences from learners whose native languages are primarily English and Japanese. The annotations include negation cues and their corresponding scopes. For evaluation, they also used the standard English CD-SCO dataset.

Key Insights and Ideas

  • L2 is not just "broken" L1: CSL exhibits unique characteristics that necessitate specialized NLP approaches, rather than simply adapting native language models.
  • Native language matters (a lot): The L1 background of CSL learners has a significant and non-obvious impact on NSR.
  • Japanese learners' data is trickier, but better for transfer: Surprisingly, data from L1-Japanese CSL learners was found to be more difficult to annotate but led to better cross-lingual transfer performance when training models compared to L1-English CSL learners.
  • Cross-lingual transfer is unequal: The influence of a learner's native language on their L2 acquisition and subsequent model performance is not uniform across all language pairs.

Key Results

  • Their neural NSR system achieved state-of-the-art accuracy on the English benchmark CD-SCO dataset.
  • On their newly created CSL corpus, they observed that models trained on L2-Chinese with L1-Japanese backgrounds performed better than models trained on L2-Chinese with L1-English backgrounds.
  • Inter-annotator agreement on their CSL corpus was high, indicating the robustness of their annotation guidelines and the data quality.

Limitations and Future Work

  • Limited L1 backgrounds: The corpus primarily focuses on English and Japanese L1 backgrounds, leaving other L1 influences unexplored.
  • Annotation difficulty: The authors acknowledge that the inherent complexity of L2 language and potential non-native expressions can lead to annotation disagreements.
  • Future work: The paper suggests further linguistic analysis to better understand the cross-lingual transfer phenomena and to develop more linguistically informed NLP models for second languages. They also highlight the need for more research into NLP for L2 learners, considering the unique characteristics of their language systems.

Summary updated

Abstract

This paper studies Negation Scope Resolution (NSR) for Chinese as a Second Language (CSL), which shows many unique characteristics that distinguish itself from “standard” Chinese. We annotate a new moderate-sized corpus that covers two background L1 languages, viz. English and Japanese. We build a neural NSR system, which achieves a new state-of-the-art accuracy on English benchmark data. We leverage this system to gauge how successful NSR for CSL can be. Different native language backgrounds of language learners result in unequal cross-lingual transfer, which has a significant impact on processing second language data. In particular, manual annotation, empirical evaluation and error analysis indicate two non-obvious facts: 1) L2-Chinese, L1-Japanese data are more difficult to analyze and thus annotate than L2-Chinese, L1-English data; 2) computational models trained on L2-Chinese, L1-Japanese data perform better than models trained on L2-Chinese, L1-English data.
BibTeX
@inproceedings{zhang2021negation,
  author = {Mengyu Zhang and Weiqi Wang and Shuqiao Sun and Weiwei Sun},
  title = {{Negation Scope Resolution for Chinese as a Second Language}},
  booktitle = {Workshop},
  year = {2021},
}
Figure