Applied Machine Learning for Games: A Graduate School Course

Yilei Zeng, Aayush Shah, Jameson Thai, Michael ZydaAAAI 2021 · DOI

Introduction

The gaming industry is undergoing a paradigm shift, moving away from hard-coded logic toward systems that embed machine learning (ML) for gameplay analysis, procedural generation, and autonomous agents. However, teaching advanced concepts like Deep Learning (DL) and Reinforcement Learning (RL) presents a significant pedagogical challenge. Traditional courses often focus on toy problems or require heavy prerequisites. This paper details a graduate-level course at the University of Southern California designed to bridge the gap between theoretical AI and practical applications by using video games as a primary vehicle for learning.

Pedagogical Methods

The course employs a "learning by doing" model structured over a 15-week semester. The curriculum is split into two halves: the first covers DL fundamentals (CNNs, RNNs, GANs), while the second focuses on RL (Q-Learning, Policy Gradients).

Central to the course is project-based learning. Students form groups of up to eight to tackle a single, semester-long research project. These projects are "direction-agnostic," allowing for research, industry, or product-focused goals. Deliverables include an Engineering Design Document (EDD), a technical paper draft, a project website, and a public presentation. To support these efforts, students are provided with cloud credits and access to high-end GPU workstations.

Datasets and Environments

Rather than a single static dataset, the course leverages a wide array of open-source benchmarks and complex simulators.

Example of a complex game environment used for multi-agent reinforcement learning projects.

Key platforms include:

  • Reinforcement Learning: OpenAI Gym, Arcade Learning Environment (ALE), and Facebook’s ELF.
  • Complex Simulators: StarCraft II (PySC2), Project Malmo (Minecraft), and Unity ML-Agents.
  • NLP/Text: TextWorld for text-based adventure games.
  • Custom Data: Students also collect their own data, such as Twitch chat logs or motion capture data for specialized analysis.

Key Insights

  • Intrinsic Motivation: Allowing students to choose their own projects (e.g., training a League of Legends bot) encourages deeper engagement with complex algorithms than assigned homework.
  • Interdisciplinary Collaboration: The course attracts a diverse cohort (63% CS, 37% other engineering fields), fostering unique problem-solving approaches through peer effects.
  • Games as AGI Testbeds: Games serve as a "sandbox" for Artificial General Intelligence, where vision, language, and decision-making must converge.

Outcomes and Student Feedback

The course has proven scalable, covering 292 students across various modalities. Direct research output includes three published conference papers. Notable project successes include a Pokemon Showdown agent with an 85% win rate and an LSTM-based procedural generator for Super Mario Bros levels.

Survey results indicating student motivation, learning outcomes, and course recommendations.

Survey data highlights the course's impact: 60% of students gave a 5/5 recommendation, and 51% felt the course was highly instrumental in securing professional opportunities like internships or full-time jobs.

Limitations and Future Work

Challenges remain in balancing heavy mathematical theory with hands-on implementation. As the course moved online, the need for more visual aids and live demonstrations became apparent. Future iterations will include formal paper-reading sessions and live-coding to further strengthen the research pipeline and help students transition to academic publication.

Summary updated

BibTeX
@inproceedings{zeng2021applied,
  author = {Yilei Zeng and Aayush Shah and Jameson Thai and Michael Zyda},
  title = {{Applied Machine Learning for Games: A Graduate School Course}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17849},
}
Figure