A Systematic Review of Questionnaire-Based Quantitative Research on MOOCs

Mingxiao Lu, Tianyi Cui, Zhenyu Huang, Hong Zhao, Tao Li, Kai WangThe International Review of Research in Open and Distributed Learning 2021 · DOI

This systematic review provides a meta-analytical look at five years of quantitative MOOC research. By synthesizing 126 studies, the authors map out the psychological and technical drivers of learner success and introduce "Factors-Goals (F-G) graphs" as a new framework for designing future empirical studies.

The MOOC Paradox: High Access, High Attrition

Massive Open Online Courses (MOOCs) represent a paradox in modern education: they offer unprecedented access to knowledge but suffer from notoriously high dropout rates. While thousands of researchers have investigated why students quit or how they perform, the literature is fragmented across various theoretical models and methodologies. This study addresses the lack of a unified "map" for questionnaire-based research, identifying the most effective models and the specific factors—such as self-efficacy or system quality—that correlate with learner outcomes.

Methodology and Theoretical Framework

The authors conducted a systematic review of 126 articles retrieved from the Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) published between 2015 and 2020. A central finding is that researchers rarely use "off-the-shelf" models. Instead, they extend classical frameworks like the Technology Acceptance Model (TAM) or Self-Determination Theory (SDT) by adding niche factors such as "network externalities" or "social recognition."

Common theoretical models and their hypotheses in MOOC research.

Visualizing Research: The Factors-Goals (F-G) Graph

The paper’s primary methodological contribution is the introduction of Factors-Goals (F-G) graphs. These graphical summaries visualize the complex web of correlations between research goals (e.g., reducing dropout rates) and influencing factors (e.g., perceived usefulness). This approach moves beyond simple lists of variables to show the structural hypotheses common in the field.

Factors-Goals (F-G) graph for learners’ continuance intention or behavior to use MOOCs.

As shown in the F-G graph above, "Continuance Intention" is driven by a mix of subjective factors (learner preference, expectancy) and objective factors (course quality, system stability), drawing from multiple theories like UTAUT and SDT.

Key Insights and Results

The review identifies three primary research clusters:

  1. Learner Performance (35.71%): What drives engagement and grades?
  2. Dropout/Continuance Intention (28.57%): Why do students stay or leave?
  3. Assessment (23.02%): How do students and teachers evaluate the platform?

Methodologically, while descriptive statistics are common (48.41%), Structural Equation Modeling (SEM) has become the gold standard for testing complex hypotheses, appearing in over 42% of the reviewed studies. Software like IBM SPSS and AMOS remain the dominant tools for these analyses.

Limitations and Future Directions

Despite the growth in research, several "bottlenecks" persist:

  • Small Sample Sizes: Many studies lack the statistical power for universal conclusions.
  • Traditional Statistics Bias: Reliance on "snapshot" surveys may miss the dynamic nature of learning.
  • Lack of Diversity: Participants are often clustered in specific geographic or institutional silos.

To overcome these, the authors recommend integrating "big data" (clickstream data), standardizing data across platforms, and applying AI and deep learning to move from describing behavior to predicting it with personalized interventions.

Summary updated

Abstract

Massive open online courses (MOOCs) have attracted much interest from educational researchers and practitioners around the world. There has been an increase in empirical studies about MOOCs in recent years, most of which used questionnaire surveys and quantitative methods to collect and analyze data. This study explored the research topics and paradigms of questionnaire-based quantitative research on MOOCs by reviewing 126 articles available in the Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) databases from January 2015 to August 2020. This comprehensive overview showed that: (a) the top three MOOC research topics were the factors influencing learners’ performance, dropout rates and continuance intention to use MOOCs, and assessing MOOCs; (b) for these three topics, many studies designed questionnaires by adding new factors or adjustments to extant theoretical models or survey instruments; and (c) most researchers used descriptive statistics to analyze data, followed by the structural equation model, and reliability and validity analysis. This study elaborated on the relationship of research topics and key factors in the research models by building factors-goals (F-G) graphs. Finally, we proposed some directions and recommendations for future research on MOOCs.
BibTeX
@article{lu2021systematic,
  author = {Mingxiao Lu and Tianyi Cui and Zhenyu Huang and Hong Zhao and Tao Li and Kai Wang},
  title = {{A Systematic Review of Questionnaire-Based Quantitative Research on MOOCs}},
  journal = {The International Review of Research in Open and Distributed Learning},
  year = {2021},
  doi = {10.19173/irrodl.v22i2.5208},
}
Figure