Comparing Different Learning Activities in a Global Neuroscience MOOC

Casey Haney, S. Zahra Atiq, Jennifer DeBoer, David Cox2016 · DOI

TL;DR

This study investigates how different types of learning activities in a neuroscience MOOC affect student performance, using the ICAP framework (Interactive, Constructive, Active, Passive). Preliminary findings suggest that more interactive activities correlate with higher grades, offering insights for designing more effective online STEM courses.

What Problem Does This Paper Address?

Massive Open Online Courses (MOOCs) often face criticism for lacking hands-on learning opportunities. While some MOOCs, like the "Fundamentals of Neuroscience" (MCB80x) course examined here, attempt to bridge this gap with virtual labs and DIY experiment videos, it's unclear how these varied activities impact student learning. This paper aims to understand the effectiveness of different learning activities within a MOOC context, specifically by categorizing student engagement using the ICAP framework and relating it to course outcomes.

Methods

The researchers employ Chi's ICAP framework to categorize student activities within the MCB80x MOOC. This framework distinguishes between four levels of cognitive engagement:

  • Interactive (I): Peer-to-peer knowledge co-construction (e.g., discussion forums).
  • Constructive (C): Interpreting content and generating new knowledge (e.g., virtual labs, DIY videos).
  • Active (A): Physical manipulation of learning materials (e.g., interacting with video controls).
  • Passive (P): Absorbing information without overt tasks (e.g., watching lecture videos with minimal interaction).

Student data, including clickstream logs and final exam grades, were collected. Students were then stratified into groups based on their grades (passed/failed) and their total time spent on the course (high/low). The study analyzes the time spent on each ICAP activity type within these strata to understand their relationship with student success.

Datasets

The study utilizes data from the "Fundamentals of Neuroscience" (MCB80x) MOOC offered on the edX platform. Data collected includes:

  • Student demographic information.
  • Clickstream data (milestones, interactions, login/logout times) from October 31, 2013, to January 25, 2014.
  • Final exam grades.

A subset of students (n=49) who attempted the final exam were included in this analysis.

Key Insights and Ideas

  • Beyond Page Views: Simply tracking the number of page views doesn't accurately predict student success. This highlights the need to analyze how students engage with content, not just how much.
  • ICAP Framework's Utility: The ICAP framework provides a valuable lens for dissecting complex online learning behaviors and understanding their differential impact on learning outcomes.
  • Interactive Activities Matter: Preliminary analysis suggests a positive correlation between time spent on interactive activities and higher grades, supporting Chi's hypothesis that deeper engagement leads to better learning.
  • Background Influences Outcomes: For some student groups, prior knowledge in neuroscience seemed to override the expected impact of ICAP activities on grades, suggesting that individual learner characteristics are crucial moderators.

Key Results

Preliminary analysis indicates that students in the "passed, high grade, high time spent" (p_hi_hi) stratum spend more time on all activity types compared to other strata, particularly proportionally more time on interactive activities. This aligns with the ICAP hypothesis that interactive engagement leads to better performance.

Conversely, the "failed, high grade" (f_hi) group, despite spending more overall time on activities than the "passed, low grade, low time spent" (p_lo_lo) group, achieved lower grades. This suggests that the type of activity, rather than just the duration, is a significant factor. The f_hi group spent a relatively equal amount of time on Active and Passive activities, while the p_lo_lo group focused more on Active activities.

The "passed, high grade, low time spent" (p_hi_lo) group showed results that did not align with the ICAP hypothesis. The authors suggest this might be due to their strong prior background in neuroscience, with some even teaching the subject.

Limitations and Future Work

  • Small Sample Size: The study acknowledges that the sample size of students who took the final exam (n=49) is small, limiting the ability to draw overarching conclusions.
  • Unrecorded Activities: The clickstream data only captures online activities. External activities, such as taking notes or using the at-home lab kits (for the treatment group), are not recorded, making it difficult to fully assess constructive activities.
  • Future Directions: The researchers plan to use descriptive and inferential statistics to further analyze the data and answer their research questions. They also intend to analyze data from the control group and compare it with the treatment group to understand the impact of the at-home lab kits. Additionally, qualitative data from interviews with students will be collected to gain a deeper understanding of their online and offline learning experiences. This work aims to inform instructional designers in creating more effective online and blended learning environments.

Summary updated

Abstract

Abstract A major criticism leveled at Massive Open Online Courses (MOOCs) is the lack of hands-on learning opportunities. ABCx (identifying information removed) offered by X University counters this criticism by offering highly interactive virtual labs and Do It Yourself (DIY) lab experiment videos in addition to regular lecture videos, quizzes, and a discussion forum. The first cohort of this course in Fall 2013 attracted learners from 143 countries. As part of this course, a subset of learners (5163 out of total 24,112 enrolled) volunteered to participate in a Randomized Control Trial (RCT). Of this set of volunteers, the treatment group (185 learners) received lab kits on which they could perform experiments using neuroscience and electrical engineering principles. The aim of the RCT was to evaluate the impact of using at-home physical lab kits with the online course. In this on-going study, we have already performed quantitative and qualitative analysis to understand the learning behaviors and interaction with the course website of learners in the treatment group, which showed: 1) There was no significant bivariate correlation between the number of page views and student grades, and 2) There were numerous “outlier” students who had high interaction with the course website and either had a low grade or did not attempt the final exam (reference removed for blinding purposes). This study will investigate students of the control group. As this course provides a variety of engaging learning opportunities to its students, we use Chi’s ICAP framework to generate our research question and to help us analyze our findings. This framework provides a hierarchical distinction between four types of learning activities: 1) INTERACTIVE activities support learners’ peer-to-peer co-construction of knowledge (e.g., interaction on discussion forums), 2) CONSTRUCTIVE learning activities require learners to interpret existing learning content and generate new knowledge (e.g., virtual lab experiments), 3) ACTIVE learning involves physical manipulation of learning material (e.g., manipulating controls of the video lecture questions), 4) PASSIVE learning activities do not require students to perform overt tasks (e.g., reading an e-textbook) (Chi, 2009; Chi & Wylie, 2014). This work in progress focuses on understanding the learning behaviors and website interaction of learners in the control group (4798 learners) and the comparative differences between the findings of the control and treatment groups. The data sources we utilize for this study include pre- and post-course surveys, discussion forum and clickstream data. We use descriptive and inferential statistics to answer our research question(s). For learners of ABCx: are the interactive activities better than constructive activities? Are constructive activities better than active activities, and Are active activities better than passive activities? Further, we perform a comparative analysis to investigate the differences in learning patterns of students in the treatment and control group, while they use these four types of activities. This study has major implications for STEM MOOCs with lab components. Understanding how users benefit from different kinds of learning activities may help MOOC designers develop activities to support higher engagement and increased student learning outcomes via online learning environments.
BibTeX
@misc{haney2016comparing,
  author = {Casey Haney and S. Zahra Atiq and Jennifer DeBoer and David Cox},
  title = {{Comparing Different Learning Activities in a Global Neuroscience MOOC}},
  year = {2016},
  doi = {10.18260/p.26525},
}

Loading related content...

Figure