Abstract
Autonomous construction using local materials is a cornerstone of future space exploration. This paper introduces a model-free reinforcement learning (RL) approach to "dry stacking"—the ancient art of building stable structures from irregular rocks without mortar. By employing a Deep Q-Network (DQN) with a novel Discounted Intersection over Union (DIoU) reward, the proposed agent learns to build stable walls from un-scanned, irregular rocks. The system demonstrates an ability to prioritize structural stability alongside geometric accuracy, outperforming traditional heuristics and paving the way for autonomous robotic masonry in unstructured environments.
Introduction: The Challenge of In-Situ Construction
For missions to the Moon or Mars, the cost of transporting building materials from Earth is a significant barrier. In-Situ Resource Utilization (ISRU) proposes using local materials, such as regolith and rocks, for construction. Dry stacking is an energy-efficient technique ideal for ISRU, but it presents a formidable challenge for robotics. Traditional robotic stacking relies on precise 3D models of every component to simulate physics and ensure stability. However, in rugged extraterrestrial settings, pre-scanning every rock is impractical. This research explores a model-free alternative: can a robot learn to build stable structures simply by observing the current state of the pile and the rock to be placed?
Methodology
The researchers framed dry stacking as a Reinforcement Learning task. The agent's goal is to place a sequence of irregular rocks within a target volume while ensuring the resulting structure remains stable.
State Representation and Neural Architecture
The agent perceives its environment through three 2D depth maps: an overhead view of the current structure, a bottom view of the next rock to be placed, and a map representing the target "goal" volume.
The core of the system is a Q-network with a two-branched architecture. A 5-level U-Net processes the structure and goal maps, while a shallower branch processes the individual rock. These features are merged via a cross-correlation layer, which identifies potential "pockets" or stable fitting locations across the construction area.
The DIoU Reward Function
A key innovation is the Discounted Intersection over Union (DIoU) reward. While standard IoU measures the overlap between the built structure and the target volume, it ignores stability; a rock that falls over might still contribute to the volume. DIoU addresses this by applying a "discount" factor to the reward if a rock moves or rotates significantly after placement. This mechanism incentivizes the agent to value structural integrity, as unstable placements lead to lower cumulative rewards.
Results and Discussion
The DIoU-trained agent demonstrated superior performance compared to random baselines and sophisticated cross-correlation heuristics.
Performance Metrics
The agent achieved a higher final Intersection over Union (IoU) and maintained significantly better stability, as measured by the Average Discount (AD) metric. The AD metric serves as a proxy for stability, where a higher value indicates that rocks remained closer to their intended placement positions.
Emergent Strategies
Interestingly, the agent developed human-like stacking strategies without explicit instruction. It learned a "boundary-first" approach, placing rocks at the edges of the target area first. This creates an inward-sloping support structure that prevents subsequent rocks from sliding out, a foundational principle in traditional dry-stone masonry.
Conclusion and Future Work
This research proves that a model-free RL agent can develop an "intuition" for physics and stability in complex construction tasks. While successful, current limitations include a fixed orientation for rock placement and a fixed sequence of rocks. Future work will focus on allowing the agent to rotate rocks and select the most suitable piece from a pile, bringing us closer to fully autonomous robotic masons capable of building infrastructure on other worlds.
Summary updated