TL;DR
This study investigates how different types of learning activities in a neuroscience MOOC affect student performance, using the ICAP framework (Interactive, Constructive, Active, Passive). Preliminary findings suggest that more interactive activities correlate with higher grades, offering insights for designing more effective online STEM courses.
What Problem Does This Paper Address?
Massive Open Online Courses (MOOCs) often face criticism for lacking hands-on learning opportunities. While some MOOCs, like the "Fundamentals of Neuroscience" (MCB80x) course examined here, attempt to bridge this gap with virtual labs and DIY experiment videos, it's unclear how these varied activities impact student learning. This paper aims to understand the effectiveness of different learning activities within a MOOC context, specifically by categorizing student engagement using the ICAP framework and relating it to course outcomes.
Methods
The researchers employ Chi's ICAP framework to categorize student activities within the MCB80x MOOC. This framework distinguishes between four levels of cognitive engagement:
- Interactive (I): Peer-to-peer knowledge co-construction (e.g., discussion forums).
- Constructive (C): Interpreting content and generating new knowledge (e.g., virtual labs, DIY videos).
- Active (A): Physical manipulation of learning materials (e.g., interacting with video controls).
- Passive (P): Absorbing information without overt tasks (e.g., watching lecture videos with minimal interaction).
Student data, including clickstream logs and final exam grades, were collected. Students were then stratified into groups based on their grades (passed/failed) and their total time spent on the course (high/low). The study analyzes the time spent on each ICAP activity type within these strata to understand their relationship with student success.
Datasets
The study utilizes data from the "Fundamentals of Neuroscience" (MCB80x) MOOC offered on the edX platform. Data collected includes:
- Student demographic information.
- Clickstream data (milestones, interactions, login/logout times) from October 31, 2013, to January 25, 2014.
- Final exam grades.
A subset of students (n=49) who attempted the final exam were included in this analysis.
Key Insights and Ideas
- Beyond Page Views: Simply tracking the number of page views doesn't accurately predict student success. This highlights the need to analyze how students engage with content, not just how much.
- ICAP Framework's Utility: The ICAP framework provides a valuable lens for dissecting complex online learning behaviors and understanding their differential impact on learning outcomes.
- Interactive Activities Matter: Preliminary analysis suggests a positive correlation between time spent on interactive activities and higher grades, supporting Chi's hypothesis that deeper engagement leads to better learning.
- Background Influences Outcomes: For some student groups, prior knowledge in neuroscience seemed to override the expected impact of ICAP activities on grades, suggesting that individual learner characteristics are crucial moderators.
Key Results
Preliminary analysis indicates that students in the "passed, high grade, high time spent" (p_hi_hi) stratum spend more time on all activity types compared to other strata, particularly proportionally more time on interactive activities. This aligns with the ICAP hypothesis that interactive engagement leads to better performance.
Conversely, the "failed, high grade" (f_hi) group, despite spending more overall time on activities than the "passed, low grade, low time spent" (p_lo_lo) group, achieved lower grades. This suggests that the type of activity, rather than just the duration, is a significant factor. The f_hi group spent a relatively equal amount of time on Active and Passive activities, while the p_lo_lo group focused more on Active activities.
The "passed, high grade, low time spent" (p_hi_lo) group showed results that did not align with the ICAP hypothesis. The authors suggest this might be due to their strong prior background in neuroscience, with some even teaching the subject.
Limitations and Future Work
- Small Sample Size: The study acknowledges that the sample size of students who took the final exam (n=49) is small, limiting the ability to draw overarching conclusions.
- Unrecorded Activities: The clickstream data only captures online activities. External activities, such as taking notes or using the at-home lab kits (for the treatment group), are not recorded, making it difficult to fully assess constructive activities.
- Future Directions: The researchers plan to use descriptive and inferential statistics to further analyze the data and answer their research questions. They also intend to analyze data from the control group and compare it with the treatment group to understand the impact of the at-home lab kits. Additionally, qualitative data from interviews with students will be collected to gain a deeper understanding of their online and offline learning experiences. This work aims to inform instructional designers in creating more effective online and blended learning environments.
Summary updated