From Rocks to Walls: A Model-Free Reinforcement Learning Approach To Dry Stacking With Irregular Rocks

André Menezes, Pedro Vicente, Alexandre Bernardino, Rodrigo VenturaWorkshops @ CVPR 2021 · DOI

Abstract

Autonomous construction using local materials is a cornerstone of future space exploration. This paper introduces a model-free reinforcement learning (RL) approach to "dry stacking"—the ancient art of building stable structures from irregular rocks without mortar. By employing a Deep Q-Network (DQN) with a novel Discounted Intersection over Union (DIoU) reward, the proposed agent learns to build stable walls from un-scanned, irregular rocks. The system demonstrates an ability to prioritize structural stability alongside geometric accuracy, outperforming traditional heuristics and paving the way for autonomous robotic masonry in unstructured environments.

Introduction: The Challenge of In-Situ Construction

For missions to the Moon or Mars, the cost of transporting building materials from Earth is a significant barrier. In-Situ Resource Utilization (ISRU) proposes using local materials, such as regolith and rocks, for construction. Dry stacking is an energy-efficient technique ideal for ISRU, but it presents a formidable challenge for robotics. Traditional robotic stacking relies on precise 3D models of every component to simulate physics and ensure stability. However, in rugged extraterrestrial settings, pre-scanning every rock is impractical. This research explores a model-free alternative: can a robot learn to build stable structures simply by observing the current state of the pile and the rock to be placed?

Methodology

The researchers framed dry stacking as a Reinforcement Learning task. The agent's goal is to place a sequence of irregular rocks within a target volume while ensuring the resulting structure remains stable.

State Representation and Neural Architecture

The agent perceives its environment through three 2D depth maps: an overhead view of the current structure, a bottom view of the next rock to be placed, and a map representing the target "goal" volume.

The core of the system is a Q-network with a two-branched architecture. A 5-level U-Net processes the structure and goal maps, while a shallower branch processes the individual rock. These features are merged via a cross-correlation layer, which identifies potential "pockets" or stable fitting locations across the construction area.

The Q-network architecture uses Siamese-inspired branches and cross-correlation to determine optimal rock placement coordinates.

The DIoU Reward Function

A key innovation is the Discounted Intersection over Union (DIoU) reward. While standard IoU measures the overlap between the built structure and the target volume, it ignores stability; a rock that falls over might still contribute to the volume. DIoU addresses this by applying a "discount" factor to the reward if a rock moves or rotates significantly after placement. This mechanism incentivizes the agent to value structural integrity, as unstable placements lead to lower cumulative rewards.

Results and Discussion

The DIoU-trained agent demonstrated superior performance compared to random baselines and sophisticated cross-correlation heuristics.

Performance Metrics

The agent achieved a higher final Intersection over Union (IoU) and maintained significantly better stability, as measured by the Average Discount (AD) metric. The AD metric serves as a proxy for stability, where a higher value indicates that rocks remained closer to their intended placement positions.

Performance comparison over 30 time steps showing the learned agent's superiority in both volume adherence (IoU) and stability (AD).

Emergent Strategies

Interestingly, the agent developed human-like stacking strategies without explicit instruction. It learned a "boundary-first" approach, placing rocks at the edges of the target area first. This creates an inward-sloping support structure that prevents subsequent rocks from sliding out, a foundational principle in traditional dry-stone masonry.

Conclusion and Future Work

This research proves that a model-free RL agent can develop an "intuition" for physics and stability in complex construction tasks. While successful, current limitations include a fixed orientation for rock placement and a fixed sequence of rocks. Future work will focus on allowing the agent to rotate rocks and select the most suitable piece from a pile, bringing us closer to fully autonomous robotic masons capable of building infrastructure on other worlds.

Summary updated

BibTeX
@inproceedings{menezes2021from,
  author = {André Menezes and Pedro Vicente and Alexandre Bernardino and Rodrigo Ventura},
  title = {{From Rocks to Walls: A Model-Free Reinforcement Learning Approach To Dry Stacking With Irregular Rocks}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  doi = {10.1109/cvprw53098.2021.00234},
}
Figure