Introduction
The gaming industry is undergoing a paradigm shift, moving away from hard-coded logic toward systems that embed machine learning (ML) for gameplay analysis, procedural generation, and autonomous agents. However, teaching advanced concepts like Deep Learning (DL) and Reinforcement Learning (RL) presents a significant pedagogical challenge. Traditional courses often focus on toy problems or require heavy prerequisites. This paper details a graduate-level course at the University of Southern California designed to bridge the gap between theoretical AI and practical applications by using video games as a primary vehicle for learning.
Pedagogical Methods
The course employs a "learning by doing" model structured over a 15-week semester. The curriculum is split into two halves: the first covers DL fundamentals (CNNs, RNNs, GANs), while the second focuses on RL (Q-Learning, Policy Gradients).
Central to the course is project-based learning. Students form groups of up to eight to tackle a single, semester-long research project. These projects are "direction-agnostic," allowing for research, industry, or product-focused goals. Deliverables include an Engineering Design Document (EDD), a technical paper draft, a project website, and a public presentation. To support these efforts, students are provided with cloud credits and access to high-end GPU workstations.
Datasets and Environments
Rather than a single static dataset, the course leverages a wide array of open-source benchmarks and complex simulators.
Key platforms include:
- Reinforcement Learning: OpenAI Gym, Arcade Learning Environment (ALE), and Facebook’s ELF.
- Complex Simulators: StarCraft II (PySC2), Project Malmo (Minecraft), and Unity ML-Agents.
- NLP/Text: TextWorld for text-based adventure games.
- Custom Data: Students also collect their own data, such as Twitch chat logs or motion capture data for specialized analysis.
Key Insights
- Intrinsic Motivation: Allowing students to choose their own projects (e.g., training a League of Legends bot) encourages deeper engagement with complex algorithms than assigned homework.
- Interdisciplinary Collaboration: The course attracts a diverse cohort (63% CS, 37% other engineering fields), fostering unique problem-solving approaches through peer effects.
- Games as AGI Testbeds: Games serve as a "sandbox" for Artificial General Intelligence, where vision, language, and decision-making must converge.
Outcomes and Student Feedback
The course has proven scalable, covering 292 students across various modalities. Direct research output includes three published conference papers. Notable project successes include a Pokemon Showdown agent with an 85% win rate and an LSTM-based procedural generator for Super Mario Bros levels.
Survey data highlights the course's impact: 60% of students gave a 5/5 recommendation, and 51% felt the course was highly instrumental in securing professional opportunities like internships or full-time jobs.
Limitations and Future Work
Challenges remain in balancing heavy mathematical theory with hands-on implementation. As the course moved online, the need for more visual aids and live demonstrations became apparent. Future iterations will include formal paper-reading sessions and live-coding to further strengthen the research pipeline and help students transition to academic publication.
Summary updated