Applying multimodal learning analytics to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming

Fan Ouyang, Xinyu Dai, Si ChenInternational Journal of STEM Education 2022 · DOI

TL;DR

This study utilizes Multimodal Learning Analytics (MMLA) to track how teacher interventions—or "scaffolding"—impact K-12 students during collaborative coding. The researchers found a striking "control paradox": while open-ended questions (low-control) spark the most immediate engagement, students are far more likely to reuse and apply direct answers (high-control) to solve problems later in the lesson.

What Problem Does This Paper Address?

Collaborative programming is a cornerstone of modern STEM education, but for novice K-12 students, it is notoriously difficult. Without help, groups often stall due to technical hurdles or social friction. Teachers provide "scaffolding" to help, but the research community has long debated which type of help is most effective.

Most existing research relies on a single data source, like chat logs or final grades, which misses the messy, real-time reality of a classroom. Furthermore, few studies distinguish between immediate uptake (did the kids listen right now?) and delayed use (did they actually learn the concept and apply it later?). This paper seeks to fill that gap by using MMLA to provide a high-resolution look at the temporal effects of teacher support.

Methods

The researchers followed 28 elementary students (grades 3–5) in China across an 11-week AI and programming course. To capture the full picture, they used a "multimodal" approach:

  • Data Sources: Four classroom cameras (video), nine group screen recordings (clickstreams), and lapel microphones (audio).
  • The 20-Second Window: The team analyzed the data in three phases: 20 seconds before a teacher intervention, 20 seconds after (immediate uptake), and the remainder of the class (delayed use).
  • Analytical Toolkit: They didn't just count behaviors; they used Social Network Analysis (SNA) to see how group dynamics shifted and Epistemic Network Analysis (ENA) to map how students' thought patterns evolved after receiving help.

Datasets

The study used a primary dataset collected specifically for this research.

  • Context: An extracurricular "Artificial Intelligence in Maker Education" course using Kittenblock (a Scratch-based visual programming environment).
  • Scale: 81 distinct session files (9 groups over 9 weeks of coding), with 62 files containing specific instructor scaffolding events.
  • Granularity: Data was transcribed and coded in 5-second intervals.

Key Insights and Ideas

  • The "Guide on the Side" vs. "Sage on the Stage" Trade-off: Low-control cognitive scaffolding (asking open-ended questions) was excellent for getting kids to talk and think in the moment. However, it rarely led to "delayed use." Conversely, high-control scaffolding (giving the answer or showing the code) was the only type that students consistently carried forward to solve subsequent problems.
  • Social Support Isn't a Silver Bullet: Interestingly, "social scaffolding"—where the teacher tries to build trust or ease tension—had the weakest effect on cognitive engagement. The authors suggest that if students are stuck on a technical "cognitive barrier," social encouragement doesn't help until the technical hurdle is cleared.
  • The Persistence of Ignored Advice: Even when groups appeared to ignore the teacher initially (low immediate uptake), they often "stored" that information and used it 5 to 8 minutes later when they hit a wall. This suggests that "failed" scaffolding might actually be a delayed success.

Key Results

  • Immediate Engagement: Low and medium-control scaffolding led to the highest levels of "high-level cognitive engagement" (students elaborating on ideas or asking follow-up questions).
  • Delayed Success: High-control cognitive scaffolding was used most frequently in a delayed way (5 instances) compared to metacognitive (2) or medium-control (4).
  • Social Dynamics: SNA showed that low-control scaffolding significantly increased student-to-student interaction, whereas high-control scaffolding tended to make the group focus silently on the screen to copy the teacher's solution.

Limitations and Future Work

The authors acknowledge several constraints:

  1. Sample Size: The study focused on a single class of 28 students, which limits how much we can generalize the results to other cultures or age groups.
  2. Lack of Experimental Control: This was an observational study in a "natural" classroom. Future work should use quasi-experimental designs to compare different scaffolding styles more strictly.
  3. Missing "Internal" Data: While video and audio are great, the authors suggest adding physiological data (like heart rate or eye-tracking) in future studies to better understand student frustration and cognitive load.
  4. Gender Balance: The participant pool was skewed toward male students (19 males vs. 9 females), which may influence collaborative dynamics.

Summary updated

Abstract

Abstract Background Instructor scaffolding is proved to be an effective means to improve collaborative learning quality, but empirical research indicates discrepancies about the effect of instructor scaffoldings on collaborative programming. Few studies have used multimodal learning analytics (MMLA) to comprehensively analyze the collaborative programming processes from a process-oriented perspective. This research conducts a MMLA research to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 education context with an aim to provide research, analytics, and pedagogical implications. Results The results indicated that the instructor provided five types of scaffoldings from the social, cognitive, and metacognitive dimensions, and groups had seven types of responses (i.e., immediate uptake and delayed use) to five instructor scaffoldings, ranging from the low-to-medium and high level of cognitive engagement. After the scaffolding was faded, groups used the content from the high-control cognitive scaffolding frequently to solve problems in a delayed way, but groups did not use the instructor’s scaffolding content from the social and low-control cognitive scaffoldings from the pedagogical perspective, instructors should consider scaffolding types, group states and characteristics, as well as the timing of scaffolding to better design and facilitate collaborative programming. From an analytical perspective, MMLA was proved to be conducive to understand collaborative learning from social, cognitive, behavioral, and micro-level dimensions, such that instructors can better understand and reflect on the process of collaborative learning, and use scaffoldings more skillfully to support collaborative learning. Conclusions Collaborative programming is encouraged to be integrated in STEM education to transform education from the instructor-directed lecturing to the learner-centered learning. Using MMLA methods, this research provided a deep understanding of the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 STEM education from a process-oriented perspective. The results showed that various instructor scaffoldings have been used to promote groups’ social and cognitive engagement. Instructor scaffoldings have delayed effects on promoting collaborative programming qualities. It is highly suggested that instructors should integrate scaffoldings to facilitate computer programming education and relevant research should apply MMLA to reveal details of the process of collaboration.
BibTeX
@article{ouyang2022applying,
  author = {Fan Ouyang and Xinyu Dai and Si Chen},
  title = {{Applying multimodal learning analytics to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming}},
  journal = {International Journal of STEM Education},
  year = {2022},
  doi = {10.1186/s40594-022-00361-z},
}
Figure