Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation

Guan-Horng Liu, Avinash Siravuru, Sai P. Selvaraj, Manuela M. Veloso, George Kantor

Abstract

Multisensory policies are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called Sensor Dropout, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion.

Introduction

Autonomous navigation systems increasingly rely on multimodal sensor suites—combining cameras, Lidar, and physical sensors—to achieve robust state estimation. However, end-to-end "pixels-to-commands" controllers often suffer from over-dependence on specific modalities. For instance, a neural network might learn to rely exclusively on high-dimensional camera data, leading to catastrophic failure if that sensor is obscured, even if other sensors like Lidar remain functional.

This paper introduces Sensor Dropout (SD), a novel regularization technique designed to force agents to learn redundant, robust policies. By training the agent to handle missing sensor data during the learning phase, the proposed framework maintains high performance during partial sensor failures, significantly reducing performance degradation in noisy or failing environments.

Methods and Architecture

The core innovation is a modular architecture augmented with two key components: Sensor Dropout and an Auxiliary Loss for variance reduction.

Modular Sensor Fusion

The architecture processes each sensor modality through its own dedicated feature extraction pipeline. For example, CNNs are used for image data, while 1D-convolutions handle laser scans. These extracted features are then concatenated into a unified multimodal state representation before being passed to the policy network.

Sensor Dropout (SD)

Unlike standard Dropout, which randomly zeroes out individual neurons, SD operates at the sensor level. During training, entire blocks of sensor data (e.g., the full camera feed or all Lidar readings) are randomly zeroed out. This forces the network to develop "cross-connections," ensuring it can navigate effectively using any available subset of inputs.

The proposed architecture featuring modular feature extraction and the Sensor Dropout mechanism before the policy network.

Auxiliary Loss for Consistency

To address the "jerk" problem—where abrupt sensor deactivation causes sudden, erratic changes in steering or acceleration—the authors introduced an auxiliary loss term. This term penalizes inconsistency between sub-policies (actions chosen when only some sensors are active) and the optimal policy identified during full-sensor training. This ensures smooth transitions during sensor failures.

Experimental Setup

The framework was evaluated using TORCS (The Open Racing Car Simulator). Three distinct sensor modalities were defined:

  1. Physical: 10-dimensional data including velocity, position, and orientation.
  2. Lasers: 4 consecutive laser range finder scans (19 readings each).
  3. Images: 4 consecutive RGB front-view images (64x64 resolution).

The researchers integrated these techniques into two popular Deep Reinforcement Learning (DRL) algorithms: Normalized Advantage Function (NAF) and Deep Deterministic Policy Gradient (DDPG).

Key Results

The inclusion of Sensor Dropout and the auxiliary loss yielded significant improvements in robustness and performance across both training and testing environments.

Resilience to Noise and Failure

In environments with Gaussian noise, baseline multimodal systems experienced a 51.7% performance drop. In contrast, systems trained with SD saw only a 9.8% drop. Furthermore, SD-trained agents successfully navigated tracks even when specific sensors were completely blocked, a scenario where naive agents typically failed.

Comparison of reward per step across different configurations. SD and the auxiliary loss significantly improve performance in both training and test environments compared to standard dropout or no dropout.

Balanced Sensor Sensitivity

Without SD, models tended to favor one modality over others by a wide margin. SD balanced this reliance, bringing the sensitivity ratio between different modalities closer to 1.0. This indicates that the model treats all sensors as valuable contributors to the navigation task, effectively transforming hardware redundancy into a functional software capability.

Discussion and Insights

Gradient visualizations revealed that agents trained with SD develop a "universal" latent representation. Whether the input is a visual image of a curve or a laser scan, internal neurons fire in similar patterns, suggesting the model learns the underlying concept of the track rather than just memorizing raw patterns.

While promising, the study was limited to clear tracks without obstacles. Future work includes exploring adaptive dropout distributions—where the system dynamically adjusts reliance based on real-time sensor reliability—and testing the framework in more complex traffic scenarios.

Summary updated

Abstract

Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called \textit{Sensor Dropout}, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: https://youtu.be/QAK2lcXjNZc.
BibTeX
@inproceedings{liu2017learning,
  author = {Guan-Horng Liu and Avinash Siravuru and Sai P. Selvaraj and Manuela M. Veloso and George Kantor},
  title = {{Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation}},
  booktitle = {Conference on Robot Learning},
  year = {2017},
  url = {https://openreview.net/forum?id=mRYhj7lA7Nu},
}
Figure