Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations

Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian A. Scherer, Ashish Kapoor

Abstract

Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to-end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real-world data. Simulated data, on the other hand, is easy to generate, but generally does not render safe behaviors in diverse real-life scenarios. In this work we propose a novel method for learning robust visuomotor policies for real-world deployment which can be trained purely with simulated data. We develop rich state representations that combine supervised and unsupervised environment data. Our approach takes a cross-modal perspective, where separate modalities correspond to the raw camera data and the system states relevant to the task, such as the relative pose of gates to the drone in the case of drone racing. We feed both data modalities into a novel factored architecture, which learns a joint low-dimensional embedding via Variational Auto Encoders. This compact representation is then fed into a control policy, which we trained using imitation learning with expert trajectories in a simulator. We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors. We also present real-world results for drone navigation through gates in different track configurations and environmental conditions. Our proposed method, which runs fully onboard, can successfully generalize the learned representations and policies across simulation and reality, significantly outperforming baseline approaches.

Introduction

Training complex robotic policies entirely in simulation for reliable real-world deployment remains a significant challenge. Deep learning for "visuomotor" tasks—mapping raw pixels directly to motor commands—typically suffers from the "reality gap," where visual differences between simulated and real environments cause policies to fail. Furthermore, end-to-end learning is often sample-inefficient, requiring dangerous amounts of real-world trial and error.

The proposed framework enables direct transfer of navigation policies from simulation to diverse real-world environments.

This work addresses these issues by learning a representation that is invariant to visual "fluff" but sensitive to task-relevant geometry. By using a Cross-Modal Variational Autoencoder (CM-VAE), we demonstrate that drones can learn to navigate high-speed racing tracks in challenging real-world conditions without ever seeing a labeled real-world image during training.

Methodology

Cross-Modal Variational Autoencoder (CM-VAE)

The core innovation is the CM-VAE, which bridges the sim-to-real gap by forcing the network to learn a joint latent space ($z$) for two distinct modalities: raw RGB images from the drone's FPV camera and the relative geometric pose of the next gate.

The CM-VAE architecture learns a joint latent representation by reconstructing both raw images and geometric gate poses.

By requiring the latent space to reconstruct the gate's pose, the network is regularized to ignore background distractions like trees or shadows that do not affect the navigation task. This results in a smooth, compact manifold where geometric relationships are preserved, facilitating stable control.

Policy Training via Imitation Learning

Once the CM-VAE is trained, its encoder is frozen. We then employ Behavior Cloning to train a control policy. An expert algorithm in the simulator generates optimal trajectories, providing a mapping from the latent vector $z$ to velocity commands ($v_x, v_y, v_z, v_{yaw}$). Because the policy operates on a low-dimensional geometric abstraction rather than raw pixels, it generalizes naturally to the real world.

Key Results

  • Robustness and Generalization: In simulation, the cross-modal approach achieved a success rate five times higher than standard end-to-end learning when subjected to random gate displacements.
  • Real-World Deployment: The system successfully navigated complex S-tracks and circular circuits. In one instance, the drone traversed 26 consecutive gates without failure.
  • Extreme Conditions: Despite being trained only on "sunny day" simulation data, the policy successfully flew in heavy snow, 20km/h wind gusts, and indoor environments with completely different visual textures.
  • Comparison to Baselines: Pure regression models (mapping images directly to poses) failed in the real world due to visual noise sensitivity, whereas the CM-VAE's unsupervised reconstruction loss kept features grounded and robust.

Conclusion

This research demonstrates that cross-modal representation learning is a powerful tool for closing the sim-to-real gap in aerial navigation. By abstracting visual input into a geometrically-informed latent space, we can train robust policies purely in simulation that perform reliably in unpredictable real-world scenarios. Future work will explore incorporating temporal information to handle more complex maneuvers and using adversarial training to further align simulated and real latent distributions.

Summary updated

Abstract

Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to- end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real- world data. Simulated data, on the other hand, is easy to generate, but generally does not render safe behaviors in diverse real-life scenarios. In this work we propose a novel method for learning robust visuomotor policies for real-world deployment which can be trained purely with simulated data. We develop rich state representations that combine supervised and unsupervised environment data. Our approach takes a cross-modal perspective, where separate modalities correspond to the raw camera data and the system states relevant to the task, such as the relative pose of gates to the drone in the case of drone racing. We feed both data modalities into a novel factored architecture, which learns a joint lowdimensional embedding via Variational Auto Encoders. This compact representation is then fed into a control policy, which we trained using imitation learning with expert trajectories in a simulator. We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors. We also present real-world results for drone navigation through gates in different track configurations and environmental conditions. Our proposed method, which runs fully onboard, can successfully generalize the learned representations and policies across simulation and reality, significantly outperforming baseline approaches.
BibTeX
@misc{bonatti2019learning,
  author = {Rogerio Bonatti and Ratnesh Madaan and Vibhav Vineet and Sebastian A. Scherer and Ashish Kapoor},
  title = {{Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations}},
  year = {2019},
  eprint = {1909.06993},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.06993},
}
Figure