Abstract
Constructing large structures with robots is a challenging task with many potential applications that requires mobile manipulation capabilities. We present two systems for autonomous wall building that we developed for the Mohamed Bin Zayed International Robotics Challenge 2020. Both systems autonomously perceive their environment, find bricks, and build a predefined wall structure. While the UGV uses a 3D LiDAR-based perception system which measures brick poses with high precision, the UAV employs a real-time camera-based system for visual servoing. We report results and insights from our successful participation at the MBZIRC 2020 Finals, additional lab experiments, and discuss the lessons learned from the competition.
Introduction: The Challenge of Autonomous Construction
Autonomous construction is a "holy grail" for robotics, with applications ranging from disaster response to building structures on other planets. However, moving from a controlled lab to an outdoor construction site introduces massive hurdles. Robots must identify and precisely locate bricks under harsh, changing sunlight without external motion-capture systems. Furthermore, bricks are heavy (up to 2kg) and long (up to 1.8m), requiring grippers that are both strong and capable of handling misalignments. Finally, building a wall requires optimizing the sequence of movements to minimize battery drain and time.
The researchers addressed these issues for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020, where robots had to build complex walls from bricks of varying sizes and colors autonomously.
Robotic Platforms and Methodology
The team developed two distinct robotic platforms with specialized software stacks to handle the diverse requirements of the challenge.
The Ground Vehicle (UGV): "Bob"
The UGV is built on an omnidirectional base equipped with a UR10e robotic arm and a custom magnetic gripper. A standout feature is its actuated storage system, which allows the robot to carry up to 10 large bricks at once, drastically reducing travel time between the supply pile and the wall.
For perception, the UGV uses a wrist-mounted 3D LiDAR (Velodyne VLP-16). The pipeline employs Iterative Closest Point (ICP) for rough alignment of the brick pile, followed by a fine-grained optimization that considers the physical constraints of the wall structure. To ensure efficiency, a backtracking algorithm finds the time-optimal build order, minimizing base movement.
The Aerial Vehicle (UAV): "Lofty"
The UAV is a customized DJI Matrice 600 featuring a unique compliant magnetic gripper. This gripper uses telescopic carbon fiber rods that allow for horizontal and rotational "give" during the approach but become rigid once the brick is lifted. This "mechanical intelligence" simplifies the control problem, as the drone does not require perfect flight precision to achieve a successful grasp.
Perception for the UAV relies on three Intel RealSense D435 RGB-D cameras. It uses color-based segmentation and a Perspective-n-Point (PnP) solver for real-time 6D pose estimation. A "cone of descent" strategy is used for stable grasping and placement, even in windy conditions.
Key Results and Insights
The team's approach proved highly effective, securing a second-place finish in Challenge 2 of MBZIRC 2020.
- Precision and Efficiency: In lab tests, the UGV achieved manipulation precision of 1.3–1.9 cm in translation and under 1.3° in rotation. The backtracking optimization reduced the number of required robot base positions and saved an average of 2.3 meters of driving per wall segment.
- Storage as a Force Multiplier: While most construction robots move one brick at a time, the UGV’s "magazine" transformed the task into a high-speed assembly process.
- Context-Aware Perception: By using the known wall blueprint as a prior, the UGV’s perception system became significantly more robust against noise and occlusions.
Lessons Learned and Future Work
The competition highlighted a "reality gap" between lab testing and field deployment. The UGV faced challenges with arena slopes and gripper reliability that were not fully anticipated. Future work will focus on improving UGV navigation on uneven terrain and deepening the integration between the UAV and UGV, potentially using the ground robot as a mobile refueling or reloading station for the aerial team.
Summary updated