Smartphone-Based Game Development to Introduce K12 Students in Applied Artificial Intelligence

Sara Guerreiro-Santalla, Alma Mallo, Tamara Baamonde, Francisco BellasAAAI 2022 · DOI

Introduction

As Artificial Intelligence (AI) becomes increasingly woven into the fabric of daily life, there is a growing need to move K12 education beyond abstract theory and simplified demonstrations. A significant gap exists between recognizing AI's presence and understanding how to build reliable, embedded systems that interact with the physical world. This paper presents a hands-on curriculum designed to transform the ubiquitous smartphone into a laboratory for AI education, targeting students aged 15-17.

The proposed approach addresses three primary challenges in AI education: hardware accessibility, student engagement, and technical complexity. By leveraging smartphones—devices already owned by most students—the curriculum democratizes access to AI tools. Engagement is fostered through project-based learning (PBL), where students create a functional "Scavenger Hunt" game, providing a clear purpose for the underlying technology. Finally, the use of MIT App Inventor balances technical rigor with accessible, block-based programming.

Methodology and Curriculum Design

The curriculum is structured around a Project-Based Learning approach, divided into two primary phases: Supervised Learning and Multi-agent Systems.

Figure 1: Overview of the 'CAPTURE IT!' curriculum structure, detailing the time allocation for main and additional activities.

Supervised Learning (TU4)

In the first phase, students utilize the "Personal Image Classifier" (PIC) extension for MIT App Inventor. This tool enables the creation of a multilayer perceptron (MLP) model without traditional text-based coding. Students navigate the entire machine learning (ML) pipeline:

  1. Data Acquisition: Capturing images of real-world objects (e.g., apples, bottles).
  2. Hyperparameter Tuning: Adjusting layers, neurons, and learning rates.
  3. Model Validation: Testing the model's accuracy in real-time.

Multi-agent Systems (TU5)

An optional extension introduces collective intelligence. By utilizing CloudDB, students synchronize game states across multiple devices. This allows players to monitor each other's progress, introducing the complexities of remote sensing, actuation, and multi-agent coordination.

The "Capture it!" Application

The core of the curriculum is the development of Capture it!, an intelligent scavenger hunt game. The application serves as an "embedded intelligence" platform where the AI runs locally on the phone, utilizing sensors (camera) and actuators (screen, speaker) to create a perception-action loop.

Figure 2: User interface and flow of the 'Capture it!' application, showing the transition from the start screen to object capture and results.

The game flow consists of several distinct sections:

  • Screen 1: The entry point where users can start the game or exit.
  • Section 1 (Starting): A countdown display to prepare the player.
  • Section 2 (ObjectCapture): The main gameplay area where the camera visor is active, the target object name is displayed, and the player initiates classification.
  • Section 3 (Results): Feedback screens indicating whether the correct object was found and displaying the total elapsed time.

Data-Driven Learning and Insights

Unlike traditional AI courses that use static benchmark datasets, this curriculum emphasizes student-generated data. Students must grapple with data quality, representativity, and diversity. For instance, they quickly learn that a model trained on a single apple from one angle fails to generalize to other apples or lighting conditions. This provides a practical, immediate lesson in overfitting and the importance of robust training sets.

Key pedagogical insights include:

  • AI as an Intelligent Agent: Framing AI as a system that receives percepts and performs actions makes abstract concepts tangible.
  • Reliability as a Bottleneck: Students realize that the game's success depends entirely on the model's performance; a poor model cannot be masked by a polished user interface.
  • Democratization: Using personal smartphones reduces the digital divide, making high-level AI education accessible to schools with varying economic resources.

Results and Impact

The curriculum was piloted in 2021 with over 50 students across six European countries (Spain, Italy, Slovenia, Lithuania, and Finland). Qualitative and quantitative feedback indicated that students successfully mastered the four stages of the ML process. Notably, this work served as the foundation for a new official high school subject, "Introduction to AI," launched in the Galician region of Spain for the 2021/22 academic year.

Conclusion and Future Work

While successful, the project highlighted that a baseline of computational thinking is necessary for students to navigate block-based programming effectively. Future iterations of the AI+ project plan to expand into autonomous robotics using the "Robobo" platform and the Internet of Things (IoT) via "Home Assistant," further broadening the scope of applied AI in K12 education.

Summary updated

BibTeX
@inproceedings{guerreirosantalla2022smartphone,
  author = {Sara Guerreiro-Santalla and Alma Mallo and Tamara Baamonde and Francisco Bellas},
  title = {{Smartphone-Based Game Development to Introduce K12 Students in Applied Artificial Intelligence}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21554},
}
Figure