Making Sense of Machine Learning: Integrating Youth’s Conceptual, Creative, and Critical Understandings of AI

Luis Morales‐Navarro, Yasmin B. Kafai, Francisco Barnés De Castro, William D. Payne, Kayla DesPortes, Daniella DiPaola, Randi M. Williams, Safinah Ali, Cynthia Breazeal, Clifford Lee, Elisabeth Soep, Duri Long, Brian Magerko, Jaemarie Solyst, Amy Ogan, Cansu Tatar, Shiyan Jiang, Jie Chao, Carolyn Penstein Rosé, Sepehr VakilProceedings 2023 · DOI

Abstract

Understanding how youth make sense of machine learning (ML) is increasingly vital as they interact with ML-powered applications daily—while connecting with friends, listening to music, or attending school. This symposium presents diverse perspectives on supporting ML sensemaking in and out of school. By constructing applications and critically evaluating systems, youth develop integrated conceptual, creative, and critical understandings necessary for participating in a data-driven world.

Introduction: Opening the "Black Box"

Machine Learning is ubiquitous in the lives of young people, powering everything from TikTok feeds to educational tools. Despite this prevalence, ML often remains a "black box" to its users. Traditional computer science education, which frequently emphasizes code-heavy logic, often fails to address the data-driven paradigm of ML. This research addresses the challenge of supporting youth "sensemaking" by identifying pedagogical "sweet spots" where students can build mental models (conceptual), express themselves (creative), and evaluate societal impacts (critical).

Methodological Approaches to ML Literacy

This symposium synthesizes seven distinct research projects that employ qualitative and design-based methods to foster ML literacy:

  • Embodied Learning: Platforms like danceON allow students to "shuttle" between physical movement and abstract algorithmic logic, grounding technical concepts in bodily knowledge.
  • Constructionism: Students learn by building ML-powered media, journalism tools (StoryQ), and electronic textiles (eTextiles), moving from passive consumers to empowered designers.
  • Embedded Ethics: Rather than treating ethics as a separate subject, it is integrated into the design process using the "Use-Understand-Create" framework.
  • Failure Analysis: Researchers use "failure artifacts"—models with deliberate biases or bugs—to prompt students to investigate underlying mechanics when systems fail to meet expectations.
  • Tangible Interfaces: Physical tools like Creature Features use cards and tokens to make abstract data training processes concrete and visible.

Data Contexts and "Small Data"

Unlike industry ML which relies on massive datasets, these educational interventions focus on youth-generated and "small data" contexts:

  • Personalized Datasets: Youth create datasets of dance poses, gestures, and textual information for journalism.
  • Curated Classification Data: Feature-based bird data is used to teach classification through tangible interfaces.
  • Intentional Failures: Pre-designed "broken" models serve as catalysts for debugging and critical reflection.

Key Insights: The Mechanics of Sensemaking

The research highlights several critical mechanisms for learning:

  • The Power of "Shuttling": Learning is most effective when students move between disciplinary knowledge (e.g., dance or journalism) and technical ML concepts. This makes abstract ideas like "feature selection" or "occlusion" personally relevant.
  • Failure as a Pedagogical Tool: When a system fails (e.g., a pose detector misses a move due to clothing), it creates "productive friction." This forces students to look beyond "magic" and examine the training data.
  • Ethics as a Technical Reality: Students develop a nuanced understanding of fairness when they see how technical decisions—such as data inclusion—directly result in biased outcomes.
  • Agency through "Hacking": By manipulating or "hacking" existing algorithms (like Spotify's danceability ratings), youth transition from feeling disempowered by "intelligent" tech to seeing themselves as agents capable of redesigning these systems.

Results and Developmental Shifts

The interventions yielded significant shifts in how youth perceive AI:

  • Conceptual Evolution: High schoolers moved from viewing AI as "helpful robots" to defining it as technology that mimics human intelligence through feature selection and training data.
  • Sociotechnical Ethics: Middle schoolers progressed from an "agreeable" view of AI to a sociotechnical perspective, predicting how flawed datasets lead to harmful societal repercussions.
  • Interface Efficacy: Tangible interfaces were found to promote deeper "learning talk" compared to purely digital or ephemeral interfaces.
  • Developmental Trajectories: While younger children (ages 9-12) often define fairness as simple equality, older youth (12-14) are better prepared to engage with complex issues of power and privilege in algorithmic design.

Future Directions

While promising, the research identifies several areas for growth. There is a need for better scaffolding of creative work to ensure that "fun" interactions lead to lasting conceptual gains. Furthermore, scaling these multi-dimensional approaches into standard K-12 classroom constraints remains a challenge. Future work should also explore how different ML tasks, such as generative AI versus classification, require distinct types of support for student sensemaking.

Summary updated

Abstract

Understanding how youth make sense of machine learning and how learning about machine learning can be supported in and out of school is more relevant than ever before as young people interact with machine learning powered applications everyday-while connecting with friends, listening to music, playing games, or attending school.In this symposium, we present different perspectives on understanding how learners make sense of machine learning in their everyday lives, how sensemaking of machine learning can be supported in and out of school through the construction of applications, and how youth critically evaluate machine learning powered systems.We discuss how sensemaking of machine learning applications involves the development and integration of conceptual, creative, and critical understandings that are increasingly important to prepare youth to participate in the world.
BibTeX
@inproceedings{moralesnavarro2023making,
  author = {Luis Morales‐Navarro and Yasmin B. Kafai and Francisco Barnés De Castro and William D. Payne and Kayla DesPortes and Daniella DiPaola and Randi M. Williams and Safinah Ali and Cynthia Breazeal and Clifford Lee and Elisabeth Soep and Duri Long and Brian Magerko and Jaemarie Solyst and Amy Ogan and Cansu Tatar and Shiyan Jiang and Jie Chao and Carolyn Penstein Rosé and Sepehr Vakil},
  title = {{Making Sense of Machine Learning: Integrating Youth’s Conceptual, Creative, and Critical Understandings of AI}},
  booktitle = {Proceedings},
  year = {2023},
  doi = {10.22318/icls2023.930603},
}
Figure