A Configurable Library for Generating and Manipulating Maze Datasets

Michael Igorevich Ivanitskiy, Rusheb Shah, Alex F. Spies, Tilman Räuker, Dan Valentine, Can Rager, Lucia Quirke, Chris Mathwin, Guillaume Corlouer, Cecilia G. Diniz Behn, Samy Wu FungarXiv 2023 · DOI · arXiv

Abstract

The maze-dataset library is a highly configurable Python package designed to generate, manipulate, and visualize maze-solving tasks for machine learning research. It addresses the need for systematic testbeds to study out-of-distribution generalization and model interpretability. By offering fine-grained control over generation algorithms and supporting diverse output formats—including raw pixels and text-based tokens—the library allows researchers to evaluate models across different architectures on identical logical tasks.

Introduction

A central challenge in modern machine learning is understanding how models handle distributional shifts—situations where testing data differs structurally from training data. Mazes serve as an ideal "sandbox" for this research because different generation algorithms produce distinct patterns, such as long winding paths versus dense branching networks.

A representative 2D maze with start (green), end (red), and the shortest path (blue) identified.

Existing maze generation tools are often too rigid, lacking the ability to easily switch between algorithms or support the "tokenized" formats required by modern Transformer models. The maze-dataset library fills this gap, providing a robust framework for creating custom benchmarks where the underlying "rules" of data generation can be precisely controlled.

Methods and Implementation

The maze-dataset library is an extensible Python package that integrates directly with the PyTorch ecosystem. It implements several classic algorithms, including Randomized Depth-First Search (RDFS), Wilson’s algorithm (for uniform spanning trees), and Percolation (for non-acyclic grids).

A sample of diverse mazes generated by the library, illustrating different structural properties.

Key technical highlights include:

  • Efficient Internal Representation: The library uses a compact boolean array (connection_list) rather than memory-heavy adjacency matrices, allowing for fast lookups and low memory overhead.
  • Post-Generation Filtering: Users can apply filters to ensure data meets specific criteria, such as minimum path length or Manhattan distance between start and end points.
  • Multi-Format Outputs: A standout feature is the ability to export a single maze as ASCII for debugging, RGB pixel arrays for CNNs, or textual sequences (tokens) for autoregressive Transformers. All formats are derived from a single unified internal representation, ensuring consistency across model evaluations.

Key Insights for Research

The library's primary value lies in its utility for studying distributional generalization. By training a model on RDFS-generated mazes and testing it on Wilson-generated mazes, researchers can quantify how structural changes affect a model's reasoning.

Additionally, the library proposes a coordinate-based tokenization scheme. By ordering coordinates consistently, a model trained on small mazes (e.g., $3 \times 3$) can potentially handle the spatial logic of larger mazes (e.g., $5 \times 5$) without its vocabulary breaking. This facilitates research into spatial scaling and model transferability.

Results and Reproducibility

Performance benchmarks indicate that RDFS is highly efficient, averaging ~31ms for large mazes, whereas Wilson’s algorithm is significantly more computationally expensive (~767ms). To solve the common reproducibility headache in ML research, maze-dataset saves all generation parameters—including seed, algorithm, and version—directly into human-readable metadata files alongside the generated data.

Conclusion and Future Work

While currently focused on 2D rectangular lattices, maze-dataset provides a flexible foundation for the AI safety and interpretability communities. Future developments aim to include more algorithms (such as Prim’s or Kruskal’s), support for non-lattice "shortcuts," and expansion into periodic (wrap-around) maze structures. The library is intended to be a living resource that evolves with the needs of researchers studying complex model behaviors.

Summary updated

Abstract

Understanding how machine learning models respond to distributional shifts is a key research challenge. Mazes serve as an excellent testbed due to varied generation algorithms offering a nuanced platform to simulate both subtle and pronounced distributional shifts. To enable systematic investigations of model behavior on out-of-distribution data, we present $\texttt{maze-dataset}$, a comprehensive library for generating, processing, and visualizing datasets consisting of maze-solving tasks. With this library, researchers can easily create datasets, having extensive control over the generation algorithm used, the parameters fed to the algorithm of choice, and the filters that generated mazes must satisfy. Furthermore, it supports multiple output formats, including rasterized and text-based, catering to convolutional neural networks and autoregressive transformer models. These formats, along with tools for visualizing and converting between them, ensure versatility and adaptability in research applications.
BibTeX
@misc{ivanitskiy2023configurable,
  author = {Michael Igorevich Ivanitskiy and Rusheb Shah and Alex F. Spies and Tilman Räuker and Dan Valentine and Can Rager and Lucia Quirke and Chris Mathwin and Guillaume Corlouer and Cecilia G. Diniz Behn and Samy Wu Fung},
  title = {{A Configurable Library for Generating and Manipulating Maze Datasets}},
  year = {2023},
  eprint = {2309.10498},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.10498},
}
Figure