Abstract
This paper describes a hands-on workshop designed to introduce non-experts to deep learning through the creation of a "Rock-Paper-Scissors" classifier. By engaging participants in the full machine learning workflow—from data collection to model evaluation—the activity illustrates fundamental concepts such as overfitting, data augmentation, and generalization.
Introduction
Despite the ubiquity of Machine Learning (ML), it remains a "black box" to most people outside the field. Traditional educational approaches often lean heavily on theory or require significant programming background, creating a barrier to entry. The authors address this gap by designing a guided experimental activity that requires zero prior knowledge or coding. The goal is to move beyond abstract definitions and let participants experience the actual workflow of an ML engineer—including the "unanticipated setbacks" that occur when a model fails to generalize.
Methodology
The activity follows a supervised classification pipeline using a Deep Neural Network (DNN), which is treated as a black box to keep the focus on high-level concepts. Key technical steps include:
- Compact Representation: Participants discuss image resolution, eventually settling on downsampling images to 64x64 pixels to balance computational efficiency with recognizability.
- Black-Box Training: The model is trained using Python and Keras (with a TensorFlow backend) within a Jupyter notebook. The facilitator executes cells one by one, focusing on the results rather than the code.
- Pedagogical "Trap": The activity is intentionally structured to fail initially. By training on a small, audience-generated dataset without regularization, the model achieves 100% training accuracy but performs poorly on unseen data, vividly illustrating overfitting.
- Data Augmentation: To fix the overfitting, the authors introduce synthetic data generation—rotating, flipping, and perturbing the color of existing images—to teach the network to recognize the underlying gesture rather than memorizing specific photos.
Datasets
The activity uses two distinct datasets:
- Evaluation Set: A pre-collected, "gold standard" set of 100 images used to test the model’s performance.
- Training Set: Crowdsourced live from the audience. Participants use their cellphones to take 80–200 images per class (Rock, Paper, Scissors). This creates a sense of ownership and demonstrates the importance of "representative" data (varying skin tones, lighting, and backgrounds).
Key Insights and Ideas
- The Power of Failure: The most effective teaching moment occurs when the audience sees a "perfect" 100% training accuracy crash to <50% on the evaluation set. This makes the concept of generalization visceral rather than theoretical.
- Human vs. Machine Perception: By scrolling through low-resolution 64x64 images, participants realize that what looks like "noise" to a human can still contain enough features for a classifier to function.
- Probabilistic Thinking: By visualizing the output as three bars (class probabilities), the activity teaches that neural networks don't provide "the truth," but rather a statistical estimate of likelihood.
Key Results
The primary quantitative finding highlights the impact of data augmentation on model performance. Without augmentation, the model typically hits 100% training accuracy but fails on the evaluation set (often <50% accuracy). After implementing data augmentation, the training accuracy grows more slowly (indicating the model is working harder to learn), but the evaluation accuracy jumps significantly, typically reaching around 75%.
Conclusion
The authors note that the final accuracy (~75%) is limited by the relatively small size of the training set collected during a short session. While the current version avoids coding to remain accessible, it could be easily adapted for Computer Science students by requiring them to implement the Keras layers or the data augmentation logic themselves. Future iterations could also explore more complex gesture sets or different types of sensor data.
Summary updated