Understanding the Teaching Styles by an Attention based Multi-task Cross-media Dimensional modelling

Suping Zhou, Jia Jia, Yufeng Yin, Xiang Li, Yang Yao, Ying Zhang, Zeyang Ye, Kehua Lei, Yan Huang, Jialie Shen

Introduction

Teaching style is a critical factor in student academic success, yet its evaluation has historically been subjective, relying on manual observations or student surveys. This research introduces a novel framework to automatically quantify and analyze teaching styles using multi-modal data, including audio, video, and text. By mapping classroom behaviors to a continuous dimensional space, the authors provide a path toward objective, automated feedback for educators.

Quantifying Style: The Teaching Style Semantic Space (TSSS)

A primary challenge in this field is defining "teaching style" quantitatively. Rather than using rigid, discrete categories, the researchers developed the Teaching Style Semantic Space (TSSS). This 2D coordinate system is based on the psychological Pleasure-Arousal theory.

To populate this space, 41 common teaching style adjectives—such as "humorous," "strict," and "dull"—were selected from over 10,000 parent feedback questionnaires. These adjectives were then mapped onto the Pleasure (P) and Arousal (A) axes based on human annotations, creating a continuous landscape of pedagogical behavior.

The Teaching Style Semantic Space (TSSS) maps 41 adjectives onto Pleasure and Arousal dimensions.

The AMMDNN Architecture

To map raw classroom data to the TSSS, the authors proposed the Attention-based Multi-path Multi-task Deep Neural Network (AMMDNN). This architecture is designed to handle the high dimensionality of cross-media data while avoiding overfitting on limited datasets.

Key features of the AMMDNN include:

  • Multi-path Processing: Raw features are partitioned into smaller groups and processed by "local regressors" to manage dimensionality.
  • Acoustic-Guided Attention: The model uses high-level acoustic features to determine the attention weights for visual and textual features, effectively "listening" to the teacher to decide which visual or textual cues are most relevant.
  • Multi-task Learning: By predicting both Pleasure and Arousal dimensions simultaneously, the model leverages the inherent correlations between these two axes to improve overall generalization.
The AMMDNN architecture utilizes multi-path local regressors and acoustic-guided attention for cross-modal fusion.

Experimental Results and Insights

The model was evaluated on the TEG18 dataset, a benchmark containing 4,541 fully-annotated 10-second segments from real primary school classes. The AMMDNN significantly outperformed baseline regression models (including standard DNNs and SVMs), achieving a Concordance Correlation Coefficient (CCC) of 0.808 for arousal and 0.739 for pleasure—an average improvement of +0.0842 over baselines.

Beyond raw performance, the study yielded several pedagogical insights:

  • Subject-Specific Styles: Physics teachers generally exhibited lower "pleasure" (higher seriousness) compared to Chinese language teachers, reflecting different subject-matter demands.
  • Fatigue Tracking: The model captured the natural decline in teacher "arousal" over the course of a lesson, likely due to physical and mental exhaustion.
  • Engagement Correlation: Analysis of student head-pose data from the EduBrain.ai dataset suggested that styles characterized by high arousal and low pleasure (e.g., "passionate" or "severe") were most effective at maintaining student visual attention.

Conclusion

By combining dimensional psychological theory with advanced deep learning, this work provides a robust method for characterizing teaching styles. The ability to objectively measure classroom presence opens new doors for automated teaching quality analysis and personalized professional development for educators.

Summary updated

Abstract

Teaching style plays an influential role in helping students to achieve academic success. In this paper, we explore a new problem of effectively understanding teachers' teaching styles. Specifically, we study 1) how to quantitatively characterize various teachers' teaching styles for various teachers and 2) how to model the subtle relationship between cross-media teaching related data (speech, facial expressions and body motions, content et al.) and teaching styles. Using the adjectives selected from more than 10,000 feedback questionnaires provided by an educational enterprise, a novel concept called Teaching Style Semantic Space (TSSS) is developed based on the pleasure-arousal dimensional theory to describe teaching styles quantitatively and comprehensively. Then a multi-task deep learning based model, Attention-based Multi-path Multi-task Deep Neural Network (AMMDNN), is proposed to accurately and robustly capture the internal correlations between cross-media features and TSSS. Based on the benchmark dataset, we further develop a comprehensive data set including 4,541 full-annotated cross-modality teaching classes. Our experimental results demonstrate that the proposed AMMDNN outperforms (+0.0842 in terms of the concordance correlation coefficient (CCC) on average) baseline methods. To further demonstrate the advantages of the proposed TSSS and our model, several interesting case studies are carried out, such as teaching styles comparison among different teachers and courses, and leveraging the proposed method for teaching quality analysis.
BibTeX
@inproceedings{zhou2019understanding,
  author = {Suping Zhou and Jia Jia and Yufeng Yin and Xiang Li and Yang Yao and Ying Zhang and Zeyang Ye and Kehua Lei and Yan Huang and Jialie Shen},
  title = {{Understanding the Teaching Styles by an Attention based Multi-task Cross-media Dimensional modelling}},
  booktitle = {ACM Multimedia},
  year = {2019},
  url = {https://openreview.net/forum?id=B15wTJocgb1},
}
Figure