Understanding Why: Statistical Techniques to Infer Causality are Underused in Computing Education Research

Seth Poulsen, Binglin Chen, Craig Zilles, Matthew WestZenodo 2023 · DOI · OpenAlex

Abstract

A common thread through education research is asking questions about how<br> treatments applied to students affect their education, career, and other<br> outcomes. For example: Will being taught in a certain way increase students'<br> learning? Will taking a computer science course lead to higher job satisfaction<br> in the future? Are remedial programs serving their intended purpose?<br> <br> The most robust way to establish the causal effects of treatments is<br> to perform randomized controlled trials. However, in the context of<br> education, it would frequently be unethical or logistically<br> impossible to simply assign students to take a certain class or<br> participate in a certain program for the purpose of research.<br> <br> As a result, we often take advantage of natural experiments or<br> quasi-experiments. In such situations, the traditional method of<br> analysis is to look at the correlation between the treatment and<br> outcome variables. However, this doesn't tell us whether the outcome<br> was caused by the treatment, as there are almost always substantial<br> selection biases or confounding variables.<br> <br> In the past few decades, advanced statistical methods have been<br> developed to analyze the assignment of subjects to treatments as if it<br> was random, allowing us to deduce the causal effect of the<br> treatment. Such methods include difference-in-differences,<br> instrumental variables, and regression discontinuity design.<br> <br> In this paper we argue that these methods have been underused in computing<br> education research. To encourage their increased use, we describe the methods<br> and present selected examples of education studies where they have allowed<br> researchers to bridge the gap from correlation to causation.
BibTeX
@inproceedings{poulsen2023understanding,
  author = {Seth Poulsen and Binglin Chen and Craig Zilles and Matthew West},
  title = {{Understanding Why: Statistical Techniques to Infer Causality are Underused in Computing Education Research}},
  booktitle = {Zenodo},
  year = {2023},
  doi = {10.5281/zenodo.7761613},
}

Loading related content...

Figure