Classroom Quantitative Evaluation: A Method of Both Formative and Summative Evaluation

Yi Zhang, Xiaoxia Wu, Cheng Zhu, Jincheng ZhouSustainability 2023 · DOI

TL;DR

This paper introduces a novel quantitative framework for evaluating classroom teaching, combining formative and summative assessments. By analyzing classroom discourse using the Flanders Interaction Analysis System (FIAS) and a Projection Pursuit of Entropy (PPE) model, it offers a systematic way to understand teaching quality and provide actionable feedback for teacher development.

What Problem Does This Paper Address?

Traditional classroom evaluations often rely on subjective expert opinions, leading to inconsistent and one-sided feedback. This makes it difficult for teachers to get concrete, operational strategies for improvement, hindering their sustainable professional development. The paper aims to bridge this gap by providing a data-driven, systematic approach to evaluate classroom teaching quality, encompassing both ongoing improvement (formative) and overall assessment (summative).

Methods

The study employs a two-pronged approach:

  • Formative Evaluation: This uses the Flanders Interaction Analysis System (FIAS) to quantitatively analyze classroom verbal interactions. FIAS categorizes discourse into teacher talk, student talk, and silence/confusion, with 10 coding items. The analysis is further broken down into:
  • Matrix Analysis: Examining classroom teaching structure, teaching style, emotional atmosphere, variable analysis, and question innovation degree.
  • Dynamic Curve Analysis: Tracking the temporal patterns of teacher talk, student talk, and silence/confusion.
  • Summative Evaluation: This utilizes a Projection Pursuit of Entropy (PPE) model. The PPE model is a technique for projecting high-dimensional data into a lower-dimensional space to identify underlying structures. In this context, it's used to create a comprehensive grade evaluation for classroom teaching language based on the FIAS data.

Datasets

The study used video recordings of 10 high school math lessons taught by 5 expert teachers (ETs) and 5 novice teachers (NTs) in Gansu Province, China. The lessons covered topics like "the concept of function," "the solution of one variable quadratic," and "definition of derivative." Each lesson lasted 40 minutes and involved classes of approximately 40 students with similar academic achievement.

Key Insights and Ideas

  • Expert teachers foster more student-led discourse: ETs leave more "blank space" in their lessons, encouraging students to think, discuss, and explore independently. This contrasts with NTs, who tend to dominate the conversation.
  • Indirect influence is key for expert teachers: ETs use more indirect teacher talk (e.g., praise, acceptance of ideas, emotional support) compared to NTs, who rely more on direct instruction and control. This aligns with theories suggesting indirect teaching promotes student motivation and active learning.
  • Harmonious classroom atmosphere is common, but ETs show more positive integration: Both ETs and NTs generally create a positive emotional atmosphere. However, ETs show a slightly higher degree of positive integration, suggesting a more robust emotional connection with students.
  • Questioning styles differ significantly: ETs ask more innovative questions and exhibit a more diversified questioning pattern, which is crucial for cultivating students' creative thinking. NTs' questioning tends to be more singular and less innovative.
  • The PPE model offers a robust summative evaluation: By integrating multiple FIAS-derived indicators, the PPE model provides an objective and operational way to assign a grade to classroom teaching language, moving beyond subjective assessments.

Key Results

  • Teacher Talk: ETs spent less time talking (59.31%) compared to NTs (63.7%), with ETs using more indirect influence (35.2%) than direct influence (23.9%). NTs used more direct influence (29.9%) than indirect influence (33.8%).
  • Student Talk: ETs had a higher proportion of student talk (17.6%) compared to NTs (13.4%).
  • Silence/Confusion: ETs had a higher proportion of silence/confusion (23.32%) than NTs (22.9%), indicating more time for student thinking and exploration.
  • Classroom Emotional Atmosphere: ETs had a higher data intensity in the "defect block" (5.41%) compared to NTs (3.53%), suggesting a larger emotional communication gap, despite both groups performing well in positive integration.
  • Summative Evaluation (PPE Model): Expert teachers achieved a higher classroom teaching language evaluation level ("excellent" with a score of 0.852143) compared to novice teachers ("medium" with a score of 0.749417).

Limitations and Future Work

  • Contextual Limitations: The study acknowledges that conclusions may not be universally applicable due to variations in classroom context, culture, and teacher training across different countries.
  • Non-Verbal Communication: The analysis focused solely on verbal interactions. The paper suggests that future research should incorporate non-verbal cues like teacher and student posture, which also convey important information.
  • Further Study: The authors call for further research to explore these limitations and expand the understanding of classroom interaction.

Summary updated

Abstract

As the most important carrier of education and teaching, the quality of classroom teaching directly affects the quality of formal school education. Traditional classroom teaching evaluation is usually based on expert experience, which can only give sporadic and one-sided evaluation results, and it is difficult for teachers to obtain systematic and operational improvement schemes, which has caused great obstacles to teachers’ sustainable development. Based on the concept of teachers’ sustainable development and Flanders Interactive Analysis System (FIAS), this study evaluates classroom teaching from two aspects: formative evaluation and summative evaluation. From the perspective of formative evaluation, a quantitative comprehensive analysis framework of classroom teaching language is constructed from two aspects of matrix analysis and dynamic curve analysis. The matrix analysis makes an in-depth analysis of classroom teaching from five aspects: classroom teaching structure, teaching style, classroom atmosphere, variable analysis, and question innovation degree. The dynamic curve of the classroom is analyzed from three aspects: teacher talk, student talk, as well as silence and confusion. From the perspective of summative evaluation, this paper combines the projection pursuit of entropy (PPE) model to form a class teaching grade evaluation model. On the basis of the above model, this study selects 5 + 5 (five expert teachers and five novice teachers) lessons that have heterogeneous forms for the same subject with Chinese high school math teachers as the research objects. Based on this framework, the verbal information of classroom teaching is encoded and a thorough analysis is carried on from the perspective of formative assessment, while at the same time, the level of classroom teaching evaluation is given from the perspective of formative evaluation. The results show that expert teachers use indirect discourse more, they also pay more attention to blank space and have a more harmonious classroom atmosphere as well as ask more innovative questions compared to novice teachers. Under the PPE model, expert teachers have higher scores than novice teachers. This study provides a systematic and comprehensive analysis scheme for the comprehensive evaluation of classroom teaching, which not only provides comprehensive and operational information for teachers to improve their classroom teaching, but also guarantees the sustainable development of teachers’ specialization, and provides a solution for the summative evaluation of classroom teaching.
BibTeX
@inproceedings{zhang2023classroom,
  author = {Yi Zhang and Xiaoxia Wu and Cheng Zhu and Jincheng Zhou},
  title = {{Classroom Quantitative Evaluation: A Method of Both Formative and Summative Evaluation}},
  booktitle = {Sustainability},
  year = {2023},
  doi = {10.3390/su15031783},
}
Figure