AI and Parallelism in CS1: Experiences and Analysis

Steven BogaertsAAAI 2023 · DOI

TL;DR

This paper explores integrating Artificial Intelligence (AI) and parallelism into introductory Computer Science (CS1) courses to enhance student engagement and understanding of core programming concepts. By using AI and parallelism as a thematic context, the authors found statistically significant improvements in students' self-reported understanding of CS, particularly for parallelism, while motivation remained high, suggesting a successful approach to introducing advanced topics early.

What Problem Does This Paper Address?

Introducing advanced computer science topics like AI and parallelism in introductory CS1 courses presents a challenge: how to do so without overwhelming beginners or sacrificing essential foundational knowledge. Many CS1 courses focus solely on basic programming constructs, potentially missing opportunities to spark student interest in more complex areas. This paper addresses this by proposing and evaluating a curriculum that weaves AI and parallelism into traditional CS1 topics, aiming to provide a motivating context for learning fundamental concepts and offering a glimpse into advanced fields. The authors also investigate whether students' pre-existing perceptions of AI (often seen as more familiar) versus parallelism influence their learning and engagement.

Methods

The study integrates AI and parallelism into an "objects-first" Java CS1 curriculum through a series of weekly lab assignments and two major projects.

  • AI Integration:
    • Decision Trees (Project 1): Students build a game where the computer identifies cartoon characters using a decision tree. While the ID3 algorithm for building the tree is treated as a "black box," students learn about training, applying models, and the concept of generalization.
  • Supervised Learning (Lab 8): A game called "Find the Gnomes" introduces supervised learning concepts, focusing on training and testing sets, model fitting, prediction, and generalization using 1D arrays and loops.
  • Neural Network Unit (Project 2): Students implement a single unit of a neural network, learning about matrix operations, vectorized batch processing formulas (though derivations are omitted), and applying their model to real-world datasets.
  • Parallelism Integration (Lab 12):
    • Unplugged Activities: Analogies and non-programming exercises introduce concepts like speedup, overhead, race conditions, and locks.
  • Programming Exercises: Students implement a parallel matrix multiplication strategy using multithreading in Java, comparing its performance to sequential execution.

The effectiveness of these integrated materials was assessed by comparing survey data from students in "alternative" courses (using the AI/parallelism theme) with those in "original" courses (without the theme). Surveys were administered pre- and post-semester, measuring students' understanding, motivation, and interest in CS, AI, and parallelism using Likert scales. Statistical tests (t-tests) and effect size measures (Hedges' g) were used to analyze the data, with adjustments for multiple hypotheses.

Datasets

The paper mentions two real-world datasets used for evaluating the neural network unit:

  • Wisconsin Breast Cancer Dataset: This dataset was cleaned by removing a sample code number, standardizing attributes, and converting output values.
  • Titanic Survivors Dataset: This dataset was preprocessed by dropping several columns, one-hot-encoding "Embarked," converting "Sex" to numerical values, and standardizing other attributes.

No new benchmarks were introduced; existing datasets were adapted for the CS1 context.

Key Insights and Ideas

  • AI as a "Black Box" can be effective: Treating complex AI algorithms (like ID3 or neural network training formulas) as black boxes in CS1 allows students to grasp the application and purpose of these techniques without getting bogged down in intricate mathematical derivations. This approach successfully contextualizes fundamental CS1 concepts.
  • Parallelism benefits from a "blank slate": Students generally showed stronger gains in understanding and explaining parallelism compared to AI. This is attributed to AI often being perceived as more familiar, leading to higher initial motivation that leaves less room for significant increase, whereas parallelism, being less familiar, offered more potential for growth in understanding and interest.
  • Motivation Ceiling Effect: Students already have high intrinsic motivation for CS. This "ceiling effect" makes it difficult to show statistically significant increases in motivation, even when the integrated materials are engaging. The fact that motivation doesn't decrease significantly in the experimental group is seen as a positive outcome.
  • Preconceptions Matter: Students' prior familiarity with AI (often higher) versus parallelism influenced their responses, suggesting that introductory materials should be mindful of these existing attitudes.

Key Results

  • CS Understanding: Students in the alternative courses showed statistically significant increases in their self-reported ability to explain CS concepts compared to the original courses (e.g., [I CS Exp], p = 0.01, g = 0.66).
  • Parallelism Understanding: "Ability to explain" (Exp) for parallelism showed statistically significant improvements across all six comparisons in Table 1, with strong effect sizes (e.g., [IV Par Exp], p < 0.01, g = 1.54).
  • AI Understanding: AI "ability to explain" (Exp) also showed statistically significant increases in most comparisons, though generally with weaker significance than parallelism.
  • Motivation: While positive trends were observed for AI and parallelism motivation, they generally did not reach statistical significance, likely due to the high baseline motivation.
  • Performance: The parallel matrix multiplication implementation showed significant speedups for larger matrices (e.g., 300x300 matrices were nearly twice as fast as sequential).

Limitations and Future Work

  • Limitations:
    • The study was conducted during periods of emergency remote instruction due to COVID-19, leading to low response rates and uncontrolled variables.
  • The Benjamini-Hochberg adjustment suggested potential false positives for some statistically significant findings, warranting further study.
  • The "black box" approach for AI might limit deeper understanding for some students, though it was chosen to maintain level-appropriateness.
  • Future Work:
    • Conducting a future study to compare different integration strategies, such as one with "black boxes" versus one with no black boxes, to better understand their respective impacts.
  • Developing a revised survey instrument with more detail on student attitudes and potentially incorporating grade data for a more comprehensive evaluation.
  • Exploring other data sources, such as detailed student feedback and exam results, to gain deeper insights.
  • Comparing different designs and delivery methods for the integrated materials.

Summary updated

BibTeX
@inproceedings{bogaerts2023ai,
  author = {Steven Bogaerts},
  title = {{AI and Parallelism in CS1: Experiences and Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26876},
}
Figure