Collecting the Teacher-Student Chatroom Corpus

Andrew Caines, Helen Yannakoudakis, Helena Edmondson, Helen Allen, Pascual Pérez-Paredes, Bill Byrne, Paula Buttery2020 · DOI

Abstract We describe the collection of a dialogue corpus for language learning, one which we believe to be the first research corpus of its kind, and one which enables both close linguistic analysis and machine learning experiments in dialogue system training. The corpus is a collection of written conversations from one-to-one lessons between teachers and learners of English. The lessons took place in an online chatroom and therefore involve more interactive, immediate and informal language than might be found in asynchronous exchanges such as email correspondence. The fact that the lessons were one-to-one means that the teacher was able to focus exclusively on the linguistic abilities and errors of the student, and to offer personalised exercises, scaffolding and correction. The TSCC so far contains more than one hundred lessons between two teachers and eight students, amounting to 13.5K conversational turns and 133K words.

Introduction

The Teacher-Student Chatroom Corpus (TSCC) is a novel dataset designed to bridge the gap between static learner corpora and complex spoken dialogue. It consists of 102 synchronous, one-to-one English language lessons conducted in a custom online environment. By providing rich annotations—including grammatical error corrections and conversation threading—it offers a new playground for pedagogical Natural Language Processing (NLP) and dialogue system research.

The Need for Synchronous Chat Data

In the fields of Second Language Acquisition (SLA) and NLP, researchers typically rely on two types of data: written essays, which lack interactivity, or transcribed spoken lessons, which are noisy and expensive to produce.

There is a significant "missing middle": synchronous text-based chat. Unlike emails or forum posts, chat is immediate and informal, yet remains structured enough for automated analysis. Furthermore, most existing corpora focus on classroom settings rather than one-to-one tutoring. The TSCC fills this void, capturing the specific "scaffolding" and personalized feedback that occurs when a teacher focuses exclusively on a single student’s needs in real-time.

Methodology and the ShinyChat Platform

The researchers developed a bespoke platform called ShinyChat to facilitate and record these interactions. The methodology focused on capturing the natural flow of a lesson while preparing the data for high-level NLP tasks.

The ShinyChat interface used for data collection

The core methodology involves a multi-layered annotation schema applied to the raw text:

  • Grammatical Error Correction (GEC): Identifying and fixing student errors.
  • Non-linear Threading: In chat, participants often respond to messages from several turns ago. The authors annotated "responding to" links to reconstruct the true logical flow of conversation.
  • Sequence Analysis: Categorizing turns based on a conversation analysis model (e.g., identifying the beginning, middle, and end of specific instructional actions).
  • Pedagogical Labeling: Tagging the teaching focus (e.g., grammar vs. vocabulary) and the use of external materials.

The Teacher-Student Chatroom Corpus (TSCC)

The TSCC provides a dense look at teacher-student dynamics through the following metrics:

  • Scale: 102 written lessons, 13,500 conversational turns, and 133,000 word tokens.
  • Participants: 2 experienced teachers and 8 intermediate-to-advanced English learners.
  • Diversity: Students represent a variety of first languages (L1s), including Japanese, Spanish, Italian, Ukrainian, Mandarin Chinese, and Thai.
  • Availability: The corpus is anonymized and available for research use via an application form.

Key Insights and Results

One of the most important insights is the necessity of "threading." Unlike a standard book or essay, chat logs are often disjointed. By annotating these links, the authors provide a dataset that can train models to understand context across multiple turns.

The corpus enables the study of how pedagogical strategies, such as "recasting" a student's error, play out in a digital environment. Furthermore, by assessing student contributions against the Common European Framework of Reference for Languages (CEFR), the dataset allows researchers to correlate specific linguistic features with proficiency levels. This data serves as a stepping stone toward developing dialogue systems that can act as "tutor bots," assisting or even simulating the teacher's role in language learning.

Conclusion and Future Directions

While the TSCC is a rich resource, the authors acknowledge its current limitations. The data currently involves only 2 teachers and 8 students, which lacks the breadth needed for broad group-level variation analysis. Additionally, while 133K words is substantial for linguistic analysis, more data is required to fully realize the potential for training robust, end-to-end dialogue systems.

Future work aims to expand the corpus to include more participants and explore how these chat-based interactions differ from traditional face-to-face or video-based learning.

Summary updated

Abstract

We describe the collection of a dialogue corpus for language learning, one which we believe to be the first research corpus of its kind, and one which enables both close linguistic analysis and machine learning experiments in dialogue system training. The corpus is a collection of written conversations from one-to-one lessons between teachers and learners of English. The lessons took place in an online chatroom and therefore involve more interactive, immediate and informal language than might be found in asynchronous exchanges such as email correspondence. The fact that the lessons were one-to-one means that the teacher was able to focus exclusively on the linguistic abilities and errors of the student, and to offer personalised exercises, scaffolding and correction. The TSCC so far contains more than one hundred lessons between two teachers and eight students, amounting to 13.5K conversational turns and 133K words.
BibTeX
@misc{caines2020collecting,
  author = {Andrew Caines and Helen Yannakoudakis and Helena Edmondson and Helen Allen and Pascual Pérez-Paredes and Bill Byrne and Paula Buttery},
  title = {{Collecting the Teacher-Student Chatroom Corpus}},
  year = {2020},
  doi = {10.33774/coe-2020-7thpv},
}
Figure