How hard is it to cross the room? - Training (Recurrent) Neural Networks to steer a UAV

Klaas Kelchtermans, Tinne TuytelaarsarXiv 2017 · OpenReview

  • Training (Recurrent) Neural Networks to steer a UAV

Abstract

This work explores the feasibility of steering a drone with a (recurrent) neural network, based on input from a forward looking camera, in the context of a high-level navigation task. We set up a generic framework for training a network to perform navigation tasks based on imitation learning. It can be applied to both aerial and land vehicles. As a proof of concept we apply it to a UAV (Unmanned Aerial Vehicle) in a simulated environment, learning to cross a room containing a number of obstacles. So far only feedforward neural networks (FNNs) have been used to train UAV control. To cope with more complex tasks, we propose the use of recurrent neural networks (RNN) instead and successfully train an LSTM (Long-Short Term Memory) network for controlling UAVs. Vision based control is a sequential prediction problem, known for its highly correlated input data. The correlation makes training a network hard, especially an RNN. To overcome this issue, we investigate an alternative sampling method during training, namely window-wise truncated backpropagation through time (WW-TBPTT). Further, end-to-end training requires a lot of data which often is not available. Therefore, we compare the performance of retraining only the Fully Connected (FC) and LSTM control layers with networks which are trained end-to-end. Performing the relatively simple task of crossing a room already reveals important guidelines and good practices for training neural control networks. Different visualizations help to explain the behavior learned.

Introduction

Teaching an Unmanned Aerial Vehicle (UAV) to navigate autonomously using only visual input is a significant challenge in robotics. While deep learning has revolutionized computer vision, its application to real-time control introduces unique hurdles. Navigation is inherently a sequential task where consecutive frames are highly correlated, a property that often destabilizes the training of Recurrent Neural Networks (RNNs). Furthermore, imitation learning—where a student network learns to copy an expert—suffers from "state-space shift." Small errors can lead the UAV into unfamiliar states where it lacks the knowledge to recover, typically resulting in a crash.

Methodology

The researchers developed a framework using a simulated environment where an automated expert, equipped with privileged information (depth and localization), generates training data. The student network aims to replicate this behavior using only a single forward-looking RGB camera.

Top-down view of the simulated room crossing environment with obstacles and a sample trajectory.

Model Architecture and Training

A key innovation in this work is the move away from end-to-end training of massive Convolutional Neural Networks (CNNs). Instead, the authors utilize a pre-trained Inception v3 network as a feature extractor, training only the final control layers. This approach is compared across two architectures: memory-less Fully Connected (FC) networks and Long Short-Term Memory (LSTM) networks.

To stabilize LSTM training on correlated video data, the authors introduced Window-Wise Truncated Backpropagation Through Time (WW-TBPTT). This method samples random time-windows of frames to decorrelate training data and prevent overfitting to specific sequences.

The imitation learning framework, showing the transition from expert-only data to supervised student flight.

Handling Drift

Two strategies for handling state-space shift were evaluated:

  1. DAgger: An iterative process where the student flies, the expert provides corrections, and the model is retrained.
  2. Recovery Cameras: Virtual cameras mounted at angles (left/right) that provide corrective steering labels, teaching the drone how to return to the center of a path without requiring real-time expert intervention.

Experimental Setup

The study utilized two synthetic datasets created in the Gazebo simulator:

  • Room Crossing One: 18 rooms with a fixed sequence of obstacles (block, wall, and overhang) but varying dimensions.
  • Room Crossing Two: A more challenging set of 15 rooms with randomized obstacle order, lighting, and textures.

Results and Discussion

The experiments yielded several critical insights for vision-based UAV control:

  • Efficiency of Transfer Learning: Retraining only control layers on top of Inception features took just 10 minutes on a modest GPU, compared to 6.5 hours for an end-to-end CNN. The transfer learning approach achieved a 100% success rate in the first dataset, while the end-to-end model succeeded only 22% of the time without heavy augmentation.
  • RGB vs. Depth: Surprisingly, RGB input outperformed perfect depth maps. RGB images provide perspective cues that allow for long-range path planning, whereas depth maps often only support reactive, last-second obstacle avoidance.
  • The Value of Memory: LSTMs consistently outperformed FC networks on the randomized dataset, demonstrating that temporal context is vital for navigating complex environments.
  • DAgger Limitations: In this specific high-level navigation task, DAgger actually degraded performance. The expert's corrective labels often conflicted with the student's existing trajectory. In contrast, "recovery cameras" proved more effective for teaching drift correction.

Conclusion

This research demonstrates that steering a UAV with recurrent neural networks is feasible and efficient when using pre-trained features and specialized training techniques like WW-TBPTT. While conducted in simulation, these findings provide a roadmap for developing robust, vision-based autonomous navigation systems. Future work will focus on validating these methods in real-world environments.

Summary updated

Abstract

This work explores the feasibility of steering a drone with a (recurrent) neural network, based on input from a forward looking camera, in the context of a high-level navigation task. We set up a generic framework for training a network to perform navigation tasks based on imitation learning. It can be applied to both aerial and land vehicles. As a proof of concept we apply it to a UAV (Unmanned Aerial Vehicle) in a simulated environment, learning to cross a room containing a number of obstacles. So far only feedforward neural networks (FNNs) have been used to train UAV control. To cope with more complex tasks, we propose the use of recurrent neural networks (RNN) instead and successfully train an LSTM (Long-Short Term Memory) network for controlling UAVs. Vision based control is a sequential prediction problem, known for its highly correlated input data. The correlation makes training a network hard, especially an RNN. To overcome this issue, we investigate an alternative sampling method during training, namely window-wise truncated backpropagation through time (WW-TBPTT). Further, end-to-end training requires a lot of data which often is not available. Therefore, we compare the performance of retraining only the Fully Connected (FC) and LSTM control layers with networks which are trained end-to-end. Performing the relatively simple task of crossing a room already reveals important guidelines and good practices for training neural control networks. Different visualizations help to explain the behavior learned.
BibTeX
@misc{kelchtermans2017how,
  author = {Klaas Kelchtermans and Tinne Tuytelaars},
  title = {{How hard is it to cross the room? - Training (Recurrent) Neural Networks to steer a UAV}},
  year = {2017},
  eprint = {1702.07600},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1702.07600},
}
Figure