TL;DR
This paper introduces NLI-PT, the first comprehensive dataset for Native Language Identification (NLI) in Portuguese, comprising 1,868 student essays from 15 different L1 backgrounds. This resource aims to advance research in NLI and related educational NLP applications for Portuguese, a language previously underserved in this domain.
What Problem Does This Paper Address?
Native Language Identification (NLI) is the task of predicting an author's first language (L1) based on their writing in a second language (L2). This is valuable for understanding language acquisition and developing targeted educational tools. While NLI research has flourished for English, other languages, including Portuguese, have lacked dedicated, well-annotated datasets. Existing Portuguese learner corpora are often fragmented, have varying L1 distributions, and lack consistent linguistic annotations, making them unsuitable for systematic NLI studies. This paper addresses this gap by creating a unified, richly annotated Portuguese NLI dataset.
Methods
The authors compiled the NLI-PT dataset by unifying data from three existing Portuguese learner corpora: COPLE2, Leiria corpus, and PEAPL2. They applied a methodology to convert these diverse sources into a uniform format. Crucially, they performed linguistic annotation at multiple levels:
- Part-of-Speech (POS) Tagging: Both simple (word type) and fine-grained (word type + morphological features) POS tags were generated.
- Syntactic Parsing: Constituency and dependency parses were created.
For the NLI experiments, a simple lexical baseline was established using a bag-of-words model and a linear Support Vector Machine (SVM) classifier.
Datasets
The primary dataset introduced is NLI-PT. It comprises:
- 1,868 student essays written in European Portuguese.
- 15 different L1 backgrounds, including Chinese, English, Spanish, German, Russian, French, Japanese, Italian, Dutch, Tetum, Arabic, Polish, Korean, Romanian, and Swedish.
- Over 380,000 tokens in total.
- Four types of annotation: original text, POS, fine-grained POS, constituency parses, and dependency parses.
The dataset was constructed by merging data from:
- COPLE2: Learner Corpus of Portuguese as Second/Foreign Language.
- Leiria corpus: Recolha de dados de Aprendizagem do Português Língua Estrangeira.
- PEAPL2: Corpus de Produções Escritas de Aprendentes de PL2.
Key Insights and Ideas
- Unifying Learner Data is Crucial: The paper highlights the challenge of disparate data formats and annotations in existing learner corpora and demonstrates a practical approach to unifying them for a specific research task (NLI).
- Rich Annotation Enables Diverse Research: Beyond NLI, the multi-level annotations (POS, syntax) make NLI-PT valuable for a broader range of educational NLP tasks and second language acquisition research.
- L1 Influence is Detectable: Even with a simple bag-of-words model, a 70% accuracy on Portuguese NLI suggests that L1 influence on L2 writing is a strong signal that can be leveraged.
- Topic Bias is a Significant Challenge: The authors acknowledge that their baseline model might be learning topic associations rather than true L1 linguistic patterns, underscoring the need for more sophisticated features and balanced datasets in NLI.
Key Results
- The NLI-PT dataset contains 1,868 texts from 15 L1 backgrounds, totaling over 380,000 tokens.
- A simple lexical baseline for Portuguese NLI, using a bag-of-words model and linear SVM on a subset of the data (5 L1s), achieved 70% accuracy with stratified 10-fold cross-validation.
Limitations and Future Work
Limitations: * The initial baseline experiment suffered from topic bias, meaning the model might have learned to associate topics with L1s rather than genuine linguistic patterns. * The dataset, while diverse, is not perfectly balanced across all L1s or topics, which can impact model performance.
Future Work: * Incorporate more features: The authors plan to experiment with syntactic features (function words, syntactic relations, POS annotations) to reduce topic bias and improve NLI performance. * Expand the dataset: Future work includes incorporating data from the CAL2 corpus and potentially other sources to further enrich NLI-PT. * Explore other applications: The dataset is intended for broader use in areas like grammatical error correction, Computer-Aided Language Learning (CALL), and spellchecking for Portuguese learners.
Summary updated