Neural Multi-task Learning for Teacher Question Detection in Online Classrooms

Gale Yan Huang, Jiahao Chen, Haochen Liu, Weiping Fu, Wenbiao Ding, Jiliang Tang, Songfan Yang, Guoliang Li, Zitao Liu

Abstract

Automating the analysis of classroom interactions is essential for providing scalable pedagogical feedback. This paper proposes an end-to-end neural framework for detecting and categorizing teacher questions in online K-12 classrooms. By employing a Transformer-based language model (NEZHA) within a multi-task learning (MTL) architecture, the system differentiates between various question types—such as knowledge-solicitation, procedural, and discourse-management—outperforming traditional machine learning baselines.

Introduction

Questioning is a cornerstone of effective teaching, serving to stimulate critical thinking and monitor student progress. However, manual evaluation of teacher questioning techniques is labor-intensive and difficult to scale. Previous automated approaches often relied on hand-crafted acoustic features or keyword matching, which struggle with the diversity of question types, speaker variability, and the complex semantic context of natural classroom dialogue.

Methodology

The proposed system follows a multi-stage pipeline to transform raw classroom audio into categorized question labels.

System Pipeline

The process begins with Voice Activity Detection (VAD) to segment the audio, followed by Automatic Speech Recognition (ASR) using a Deep-FSMN model to generate text transcripts. These transcripts are then processed by a neural Natural Language Understanding (NLU) module.

Figure 1: The end-to-end pipeline from raw audio recording to question type labels.

Neural Architecture

The core of the framework is a pre-trained Transformer-based model (NEZHA). To handle the nuances of different question types, the researchers implemented a Multi-Task Learning (MTL) strategy. Instead of a single multi-class classifier, the model treats each question category as a separate binary detection task. These tasks share lower Transformer layers to capture general linguistic structures while utilizing specialized Multi-Layer Perceptron (MLP) heads for specific categories.

Figure 2: The Multi-Task Learning architecture featuring shared Transformer layers and task-specific MLP heads.

Dataset and Experimental Setup

The model was trained and evaluated on a large-scale dataset of 548 real-world K-12 online classroom recordings, totaling 39,313 utterance segments. The data spans Mathematics, Physics, and Chemistry across elementary, middle, and high school levels. Utterances were categorized into five types:

  • Knowledge-solicitation (KQ): Factual or conceptual questions.
  • Open questions (OQ): Questions without a single deterministic answer.
  • Procedural (PQ): Logistics, equipment checks, or greetings.
  • Discourse-management (DQ): Conversational transitions (e.g., "Right?").
  • Non-question (NQ): Standard instructional statements.

Results

The MTL framework demonstrated significant improvements over baseline models like Random Forests, SVMs, and Bi-LSTMs.

  • Binary Detection: For the general question vs. non-question task, the model achieved an accuracy of 0.885 and an AUC of 0.933.
  • Multi-way Classification: The model reached a Macro F1-score of 0.785 across all categories.
  • Category Performance: Discourse-management questions were the easiest to detect due to their formulaic nature, while procedural questions proved the most challenging due to their high linguistic diversity.

Conclusion

This research highlights the superiority of deep semantic features over traditional acoustic cues for classroom analysis. The multi-task learning approach effectively balances shared linguistic knowledge with category-specific nuances. Future work aims to reduce dependency on ASR error rates and link these automated metrics to student learning outcomes, ultimately providing actionable recommendations for teacher professional development.

Summary updated

BibTeX
@inproceedings{huang2020neural,
  author = {Gale Yan Huang and Jiahao Chen and Haochen Liu and Weiping Fu and Wenbiao Ding and Jiliang Tang and Songfan Yang and Guoliang Li and Zitao Liu},
  title = {{Neural Multi-task Learning for Teacher Question Detection in Online Classrooms}},
  booktitle = {AIED},
  year = {2020},
  doi = {10.1007/978-3-030-52237-7\_22},
}
Figure