Abstract
This paper presents a robust approach for fusing data from multiple sensors—both on-board and remote—to train a robot using Deep Reinforcement Learning (DRL). By employing a "late fusion" architecture combined with a specialized DropPath regularization technique, the researchers created a system that not only outperforms single-sensor setups but remains functional even when external sensors fail or lose connection at runtime. The approach is validated through a "search and pick" task using a Kuka YouBot in both simulated and real-world environments.
Introduction
In traditional robotics, sensors and actuators are often treated as a static, hardwired system. However, as we move toward an "Internet of Things" (IoT) paradigm, robots may increasingly rely on external sensors, such as overhead cameras or corner-mounted lidars, connected via unreliable wireless networks. Current DRL models typically struggle with two primary challenges in this context:
- Scalability: Simply concatenating every sensor input (known as "early fusion") leads to a massive increase in neural network parameters, which complicates and slows down the training process.
- Robustness: If a model is trained to expect input from three sensors and one goes offline, the policy usually collapses because the input distribution has fundamentally changed.
This research aims to identify a neural network architecture that fuses multi-sensor data efficiently and handles sensor "dropouts" gracefully without requiring retraining.
Methodology
The researchers utilized a Deep Q-Network (DQN) to train a Kuka YouBot—an omnidirectional mobile platform equipped with a 5-DOF arm—for a navigation and manipulation task.
Fusion Architectures
The study compared two main strategies for integrating lidar data from the robot and the environment:
- Early Fusion: All lidar scans are treated as additional channels or planes at the very first layer of the network.
- Late Fusion: Each sensor has its own dedicated "head" consisting of convolutional layers. The features extracted from these heads are then merged. Three merging methods were evaluated:
- Concat: Stacking features, which results in a high parameter count.
- Conv: Using a 1x1 convolution to compress stacked features.
- Accumulation (Acc): Summing the feature maps from all sensors. This is the most efficient method as it does not add new weights when additional sensors are integrated.
DropPath Regularization
To address sensor failure, the authors introduced DropPath. During a "refinement" training phase, the network randomly "drops" the input from the environment-mounted sensors. This forces the robot to learn how to succeed using only its on-board sensor when necessary, while still utilizing the extra data when available. The authors found that applying DropPath from the start hindered learning; instead, they first trained the model to convergence with all sensors active before performing the refinement pass.
Experimental Setup
The study utilized a custom-built RL environment to bridge the gap between simulation and reality:
- Simulation: A digital twin of the physical arena was created in the V-REP simulator. It featured a 1.8m x 2.7m arena, the Kuka YouBot, and three Hokuyo lidar sensors (one on the robot and two mounted in the corners).
- Real-World: The trained policies were transferred to a physical Kuka YouBot setup to validate "sim-to-real" performance.
Key Results and Insights
- Late Fusion via Accumulation is Superior: Summing feature maps (Late-acc) proved highly efficient. It allows the network to learn a "universal" feature representation where any sensor can contribute to identifying critical features, such as target location.
- Overcoming "Blind Spots": A single on-board sensor cannot see behind the robot. Fusing data from corner-mounted sensors allows the robot to "see" the target even when its back is turned, leading to significantly more efficient navigation paths compared to the single-sensor baseline.
- Graceful Degradation: Without DropPath, a late-fusion model's performance dropped below a "random action" baseline if environment sensors were removed. With DropPath (the
Acc + DPmodel), the robot maintained performance nearly identical to a model trained specifically for a single sensor. - Sim-to-Real Success: The researchers successfully transferred the policy to the physical robot. They noted that adding standard Dropout (dropping individual neurons) in addition to DropPath helped the model handle the noise and reflections inherent in real-world lidar data.
Conclusion
This work demonstrates that late fusion via feature accumulation, combined with a two-stage DropPath training regime, provides a scalable and robust solution for multi-sensor robot control. Future work may explore continuous action spaces using algorithms like PPO or DDPG, as well as multi-modal fusion involving both lidar and RGB-D camera data.
Summary updated