TL;DR
This paper introduces a five-step framework to help high school students systematically audit machine learning (ML) applications, moving beyond ad-hoc observations to more rigorous evaluation. Through a case study of youth auditing peer-designed TikTok filters, the authors demonstrate how this scaffolded approach empowers young people to critically understand the limitations and potential biases of AI.
What Problem Does This Paper Address?
As AI and ML become increasingly integrated into young people's lives, there's a growing need for them to become critical users, designers, and evaluators of these technologies. However, existing AI education often focuses on basic literacy or direct instruction, with little emphasis on how youth can empirically assess system behaviors and their implications. The "black-box" nature of ML models further complicates this, making it difficult to understand their limitations and potential biases. Algorithm auditing, a method used by experts to investigate opaque systems, offers a promising avenue, but existing research hasn't effectively translated this into accessible, systematic learning activities for younger audiences. This paper aims to bridge that gap by proposing and evaluating a structured, five-step auditing process tailored for youth.
Methods
The authors developed a five-step framework for algorithm auditing based on common practices used by expert auditors and user-driven auditing research. These steps are:
- Developing a Hypothesis: Formulating an educated guess about how the system might behave.
- Generating a Set of Systematic, Thorough, and Thoughtful Inputs: Designing specific inputs to rigorously test the hypothesis.
- Running a Test: Systematically querying the algorithm with the designed inputs and recording the outputs.
- Analyzing the Data: Examining the collected outputs to draw conclusions.
- Reporting the Results: Communicating the findings to inform potential changes or raise awareness.
They piloted this framework in a two-week after-school workshop with 14 high school students (ages 14-15) who designed and then audited peer-created generative AI TikTok filters. The study employed a case study methodology, analyzing youth-created artifacts (like input organizers and audit reports) and video/screen recordings to understand their engagement with each step.
Datasets
Not discussed in the paper. The focus was on youth-created TikTok filters and their outputs.
Key Insights and Ideas
- Structured Auditing is Key for Youth: Simply observing a system's behavior isn't enough. A systematic, step-by-step approach like the proposed five-step model is crucial for youth to move from casual observations to more in-depth, critical evaluations.
- Bridging Expert and User Auditing: The framework successfully integrates elements of both expert-driven auditing (systematicity, rigor) and user-driven auditing (connecting to everyday experiences, ad-hoc hypothesis generation). This blend makes auditing more accessible and relevant to young people.
- Scaffolding is Essential: Providing concrete tools and guidance at each step (e.g., input organizers, structured tables for recording data) significantly supports youth in navigating the complexities of algorithm auditing.
- Hypothesis Generation Benefits from Exploration: Youth initially explored the TikTok filters through open-ended play, which was instrumental in helping them develop hypotheses about potential biases. This highlights the value of unstructured exploration before formal hypothesis formulation.
- Input Generation is a Major Challenge: Creating a diverse and systematic set of inputs proved to be a significant hurdle for the youth. This suggests a need for further scaffolding or collaborative approaches to data collection.
- Reporting as a Learning Opportunity: Having youth report their findings to their peers (the filter designers) provided a clear audience and purpose, framing the audit as a constructive feedback mechanism.
Key Results
The case study of Ishmael and Ziyi demonstrated that high school youth could successfully engage with the five-step auditing process. They audited a TikTok filter designed to give users red hair, red clothes, and a cloudy background. Their hypothesis was that the filter exaggerated feminine characteristics. Through their audit, they found that the filter indeed transformed all input images into female-presenting individuals, adding features like blush and lipstick, and altering skin tones. They reported their findings in a humorous TikTok video, demonstrating their understanding of the audit process and its outcomes. The study highlights that even with a relatively small dataset (33 images), youth could uncover systematic biases.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Dataset Size: The youth generated a dataset of only 33 images, which is smaller than what expert audits typically use. Future work could explore class-wide collaborative dataset building or leveraging existing datasets.
- Input Generation Complexity: Youth struggled with generating a sufficiently diverse and systematic set of inputs. Future iterations should further scaffold this step.
- Data Analysis Sophistication: While youth could identify patterns and calculate percentages, more advanced statistical analysis and data visualization techniques could be explored.
- Reporting Formats: While TikTok videos were engaging, exploring other computational communication methods like infographics or data visualization projects could be beneficial.
- Classroom Integration: The paper calls for exploring how these five steps can be integrated into classroom settings, considering factors like AI system choice, collaborative arrangements, and teacher professional development.
- Platform Accessibility: The reliance on TikTok and Effect House presents challenges due to potential legal issues and platform restrictions in school environments. Future work needs to identify alternative platforms and community spaces for such activities.
Summary updated