TL;DR
Game2Player is a scalable framework that co-develops game environments and AI players to generate high-quality, verifiable training data for visual agents. By automating the adaptation of over 500 web games and using a novel progress-weighted fine-tuning method, the researchers produced a 9B-parameter model that doubles the performance of base models on complex gaming benchmarks.
What Problem Does This Paper Address?
Training general-purpose visual agents to play video games is a "grand challenge" for AI because it requires a rare mix of sharp perception, long-horizon planning, and precise motor control. However, the field faces a massive data bottleneck.
Human gameplay is prohibitively expensive to collect at scale. While the internet is full of gameplay videos, these lack the explicit action labels (keystrokes, mouse clicks) needed for supervised learning. Furthermore, existing game datasets are often narrow or lack "verifiable" outcomes—it’s hard to train an agent if the environment can't programmatically confirm whether a task was actually completed. Building these interactive, state-verified environments manually is a slow, engineering-heavy process that doesn't scale.
Methods
The authors introduce the Game2Player co-development loop, which treats environment building and agent training as a unified, self-correcting process.
The framework involves three specialized roles:
- The Builder: An LLM-powered agent that "scouts" in-the-wild web games (Unity, Godot, Flash, etc.) and instruments them. It exposes the internal game state (the "state probe") and defines specific tasks with measurable progress and success conditions.
- The Players: Two types of agents attempt these tasks. A Coding Player writes Python scripts to play using the exposed internal state, while a VLM Player plays directly from pixels, providing natural language reasoning for its actions.
- The Feedback Loop: If the players fail, they file "defect reports" (e.g., "this menu blocks the screen") or "observation requests" (e.g., "I need to see the player's health variable"). The Builder uses this feedback to repair the environment until it is verified as solvable.
- Progress-Weighted SFT: Once environments are stable, the system collects thousands of trajectories. To train the final visual agent, the authors use progress-weighted Supervised Fine-Tuning (SFT). Instead of treating every frame equally, this method assigns higher importance to actions that directly lead to a state-based increase in task progress, effectively filtering out "noise" from the training signal.
Datasets
The paper introduces a massive new resource for the research community:
- Game2Player Collection: A dataset of 568 web games spanning over 25 genres, featuring 3,436 state-verified tasks and 15.7K trajectories (roughly 307 hours of gameplay).
- PlayerBench: A new, challenging evaluation benchmark consisting of 50 held-out games and 287 tasks, categorized by difficulty tiers from "Easy" to "Very Hard."
Key Insights and Ideas
- Agents as Quality Assurance: The most striking idea is using agents to debug environments. By having agents "complain" to the builder when they can't solve a task, the system automatically scales the creation of high-quality, bug-free training grounds.
- The Power of Progress: Standard imitation learning often gets bogged down in frames where an agent is just wandering or waiting. By weighting the loss function based on internal game state progress, the model learns to prioritize the "clutch" moments that actually drive a task toward completion.
- Complementary Teachers: The authors found that Coding Players are incredibly efficient (22x faster than VLM players at generating data) and provide "perfect" successful demonstrations. Meanwhile, VLM players provide the "why" through reasoning notes, which helps the final model generalize its decision-making process.
Key Results
The resulting model, Game2Player-9B (fine-tuned from Qwen3.5), showed dramatic improvements over existing open-weight and even some proprietary models:
- Doubled Performance: On the new PlayerBench, it improved task progress from 14.3% to 31.2% compared to the base model.
- Strong Generalization: On the external GameWorld benchmark, it reached 53.1% progress, outperforming the published results of several larger proprietary models.
- Loop Effectiveness: In controlled tests, the co-development process increased the share of correctly implemented tasks from 65% to 72%, proving that the feedback loop is essential for generating valid data.
Limitations and Future Work
Despite its success, the authors acknowledge several limitations:
- Solvability: Even with the feedback loop, about 28% of tasks in their controlled study remained incorrectly implemented or unsolved by the automated agents.
- Technical Scope: The framework currently focuses on single-player web games. Scaling to massive 3D titles or multiplayer environments would require handling significantly more complex engine hooks and network latency.
- Real-time Latency: The VLM Player currently pauses the game to "think" at each step. Future work will need to address acting under real-time constraints to truly mimic human-level performance in fast-paced games.
Summary updated