Abstract
This paper presents a "zero-dependency" reinforcement learning (RL) framework designed to solve JumpIN’, a complex spatial puzzle game. By utilizing a modular Python-based architecture and a novel backtracking update algorithm, the RL agent successfully solves all 60 official puzzles, often reaching near-optimal solutions within 20 iterations. Analysis of the resulting Q-tables provides human-interpretable strategic insights, such as the utility-based role of fox pieces and the effectiveness of bunny-prioritization heuristics.
Introduction
The researchers address the challenge of solving JumpIN’, a spatial puzzle game by SmartGames played on a 5x5 grid. The game involves moving bunnies (which must jump over obstacles), foxes (which slide), and stationary mushrooms to get all bunnies into burrows. While the rules are simple, the state space is large, and puzzles can require up to 100 moves, making traditional "greedy" look-ahead strategies computationally expensive. The authors sought to create a transparent, modular system that could not only solve the game but also distill human-readable strategies from the machine's "experience."
Methodology
The researchers built a custom RL environment in Python without external machine learning libraries. Their approach features three key components:
- Modular Code Structure: The system is split into independent modules for Game Mechanics, Solution (tree-growth algorithm), and Training.
- Backtracking Q-Value Updates: Instead of standard step-by-step updates, the agent traces the solution path backward from the goal once a win is achieved. This allows the final reward to propagate through states much faster.
- Efficiency-Based Rewards: Rewards are calculated as the reciprocal of the number of moves ($1/n$), incentivizing the agent to find the shortest path.
Experimental Setup
The study utilized the 60 official sample puzzles from the JumpIN’ manual, ranging from "Starter" to "Master" difficulty. During large-scale training, the agent played each puzzle 10,000 times, generating a Q-table containing 92,183 unique states.
Results and Analysis
The agent demonstrated rapid convergence, typically reaching near-optimal solutions within 20 training iterations. The choice of exploration rate ($\epsilon$) significantly impacted performance stability.
As shown in the figure above, a high exploration rate ($\epsilon = 0.99$) allowed the agent to find solutions quickly but caused oscillations in performance. Conversely, a low exploration rate ($\epsilon = 0.1$) resulted in slow but steady, monotonic improvement. After training, test solutions for all 60 puzzles were within 5 moves of the known minimum path length.
Analysis of the Q-table provided insights into game strategy. The agent's most successful strategy matched a human heuristic: prioritizing moves that maximize the number of bunnies in burrows. Furthermore, the data revealed that fox moves are generally ranked lower than bunny moves, suggesting foxes act primarily as "utility" pieces.
The researchers also discovered a predictive relationship between piece composition and solution requirements. As illustrated in the figure above, the fraction of foxes on the board is a strong predictor of the proportion of fox moves required in the final solution.
Conclusion and Future Work
The "zero-dependency" approach proved effective for the 5x5 JumpIN’ board, achieving high performance while maintaining transparency. However, the standard Q-table approach faces scaling challenges due to the curse of dimensionality. Future work will explore Deep Q-Learning (DQN) to generalize across states, symmetry exploitation to reduce state space, and scaling the framework to larger grid dimensions.
Summary updated