Abstract
The maze-dataset library is a highly configurable Python package designed to generate, manipulate, and visualize maze-solving tasks for machine learning research. It addresses the need for systematic testbeds to study out-of-distribution generalization and model interpretability. By offering fine-grained control over generation algorithms and supporting diverse output formats—including raw pixels and text-based tokens—the library allows researchers to evaluate models across different architectures on identical logical tasks.
Introduction
A central challenge in modern machine learning is understanding how models handle distributional shifts—situations where testing data differs structurally from training data. Mazes serve as an ideal "sandbox" for this research because different generation algorithms produce distinct patterns, such as long winding paths versus dense branching networks.
Existing maze generation tools are often too rigid, lacking the ability to easily switch between algorithms or support the "tokenized" formats required by modern Transformer models. The maze-dataset library fills this gap, providing a robust framework for creating custom benchmarks where the underlying "rules" of data generation can be precisely controlled.
Methods and Implementation
The maze-dataset library is an extensible Python package that integrates directly with the PyTorch ecosystem. It implements several classic algorithms, including Randomized Depth-First Search (RDFS), Wilson’s algorithm (for uniform spanning trees), and Percolation (for non-acyclic grids).
Key technical highlights include:
- Efficient Internal Representation: The library uses a compact boolean array (
connection_list) rather than memory-heavy adjacency matrices, allowing for fast lookups and low memory overhead. - Post-Generation Filtering: Users can apply filters to ensure data meets specific criteria, such as minimum path length or Manhattan distance between start and end points.
- Multi-Format Outputs: A standout feature is the ability to export a single maze as ASCII for debugging, RGB pixel arrays for CNNs, or textual sequences (tokens) for autoregressive Transformers. All formats are derived from a single unified internal representation, ensuring consistency across model evaluations.
Key Insights for Research
The library's primary value lies in its utility for studying distributional generalization. By training a model on RDFS-generated mazes and testing it on Wilson-generated mazes, researchers can quantify how structural changes affect a model's reasoning.
Additionally, the library proposes a coordinate-based tokenization scheme. By ordering coordinates consistently, a model trained on small mazes (e.g., $3 \times 3$) can potentially handle the spatial logic of larger mazes (e.g., $5 \times 5$) without its vocabulary breaking. This facilitates research into spatial scaling and model transferability.
Results and Reproducibility
Performance benchmarks indicate that RDFS is highly efficient, averaging ~31ms for large mazes, whereas Wilson’s algorithm is significantly more computationally expensive (~767ms). To solve the common reproducibility headache in ML research, maze-dataset saves all generation parameters—including seed, algorithm, and version—directly into human-readable metadata files alongside the generated data.
Conclusion and Future Work
While currently focused on 2D rectangular lattices, maze-dataset provides a flexible foundation for the AI safety and interpretability communities. Future developments aim to include more algorithms (such as Prim’s or Kruskal’s), support for non-lattice "shortcuts," and expansion into periodic (wrap-around) maze structures. The library is intended to be a living resource that evolves with the needs of researchers studying complex model behaviors.
Summary updated