Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation

Guan-Horng Liu, Avinash Siravuru, Sai P. Selvaraj, Manuela M. Veloso, George Kantor

Abstract

We proposed a multimodal end-to-end policy based on deep reinforcement learning (DRL) that leverages sensor fusion to reduced performance drops in noisy environment from 50% to 10% compared with traditional methods. By introducing Sensor Dropout, a novel stochastic regularization technique, we force the policy to remain functional even when specific sensors fail or become noisy.

Introduction

In autonomous navigation, sensor fusion is critical for reliability. Ideally, if a camera is blinded by glare, a vehicle should fall back on LiDAR or GPS. However, many modern end-to-end deep learning models struggle with this, often overfitting to the presence of all sensors. When one sensor fails, the entire policy collapses because the model never learned to navigate using only the remaining inputs. This paper addresses this vulnerability by proposing a robust multimodal system that learns an integrated latent representation of the environment.

Methods

The researchers enhance standard Deep Reinforcement Learning (DRL) algorithms, such as NAF and DDPG, with three primary components:

  1. Sensor Dropout (SD): Unlike standard dropout that targets individual neurons, Sensor Dropout zeros out entire sensor modules (e.g., camera feeds or laser scans) during training. This forces the network to develop "sub-policies" capable of navigating with any sensor combination.
  2. Informed Rescaling: To handle varying dimensionalities across sensors (e.g., small vectors vs. large images), dynamic rescaling ensures internal activations remain consistent regardless of active sensors.
  3. Auxiliary Loss for Variance Reduction: A penalty term encourages different sensor combinations to agree on the same action, mapping diverse inputs into a shared, consistent latent space.
Architecture of the multimodal policy with Sensor Dropout.

Experimental Setup

The study utilizes TORCS (The Open Racing Car Simulator) with three sensor modalities:

  • Physical: 10-DOF data (velocity, position, orientation).
  • Lasers: 4 consecutive laser range finder scans (19 readings each).
  • Images: 4 consecutive RGB front-view images (64x64 resolution).

The model was trained to navigate tracks autonomously, focusing on isolating the effects of multimodal perception.

Key Results and Insights

The proposed method demonstrates significant improvements in robustness and efficiency.

  • Robustness to Noise and Failure: When Gaussian noise was introduced, the SD-trained policy saw only a 9.8% performance drop, compared to a 51.7% drop for a standard baseline. The policy remained functional even when both camera and laser sensors failed completely.
  • True Fusion and Latent Alignment: PCA visualization revealed that SD clusters representations of different sensor combinations together, meaning the AI "sees" the same situation regardless of the input type. Gradient visualizations also showed that SD-trained models focus on salient features like road boundaries, whereas naive models pick up on noise.
  • Performance Gains: As shown in the comparative analysis, the inclusion of Sensor Dropout and auxiliary loss significantly improves the reward per step in both training and testing environments compared to baseline models without SD.
Performance comparison across different training and testing environments.

Conclusion

Sensor Dropout provides a powerful regularization technique for multimodal autonomous navigation. While currently tested in obstacle-free environments, it shows great promise for creating fault-tolerant policies. Future work will explore scaling to dense traffic, adaptive dropout distributions, and more complex driving behaviors like lane changing and collision avoidance.

Summary updated

Abstract

We proposed a multimodal end-to-end policy based on deep reinforcement learning (DRL) that leverages sensor fusion to reduced performance drops in noisy environment from 50% to 10% compared with th...
BibTeX
@inproceedings{liu2017learning,
  author = {Guan-Horng Liu and Avinash Siravuru and Sai P. Selvaraj and Manuela M. Veloso and George Kantor},
  title = {{Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation}},
  booktitle = {Conference on Robot Learning},
  year = {2017},
  url = {https://openreview.net/forum?id=mRYhj7lA7Nu},
}
Figure