Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness

Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith, Stan Birchfield

Abstract

Autonomous navigation of micro aerial vehicles (MAVs) in unstructured environments like forest trails presents significant challenges in stability and environmental awareness. This paper presents a robust system utilizing a specialized deep neural network, TrailNet, which predicts both orientation and lateral offset to ensure centered flight. Integrated with real-time object detection and monocular depth estimation on an NVIDIA Jetson TX1, the system achieves high levels of autonomy and smooth control. Experimental results demonstrate 100% autonomy on complex forest trails and stable flights of up to 1 km using off-the-shelf hardware.

Introduction

Navigating a forest trail is a "Goldilocks" problem for robots: they must stay centered enough to avoid thick brush but move fast enough to be useful. Most previous attempts at autonomous trail-following relied solely on predicting the drone's orientation. However, orientation-only systems fail to correct for lateral drift, leading to collisions. Furthermore, many deep learning controllers suffer from "overconfidence," resulting in jerky, oscillatory flight paths. This work addresses these issues by introducing a dual-prediction network and a stability-focused loss function.

System Architecture and Methods

The proposed system runs entirely on an NVIDIA Jetson TX1 mounted on a 3DR Iris+ quadcopter. The software stack is built on ROS and integrates several key components for navigation and awareness.

System architecture overview showing the integration of TrailNet, object detection, and the flight controller within the ROS framework.

TrailNet: A Specialized DNN

The core of the navigation system is TrailNet, a modified ResNet-18 architecture optimized for real-time performance and accuracy. Key modifications include the removal of batch normalization and the implementation of Shifted ReLUs (SReLU).

The s-ResNet-18 (TrailNet) architecture, featuring dual outputs for view orientation and lateral offset.

Unlike traditional models that only predict orientation, TrailNet utilizes a 6-category classification scheme. It predicts three categories for orientation (left, center, right) and three for lateral offset (left, center, right relative to the trail midline). This dual approach allows the MAV to actively center itself rather than just pointing in the correct direction.

Stability and Awareness

To prevent jerky flight, a stability-focused loss function incorporating label smoothing and an entropy reward was implemented. This encourages the network to maintain a degree of uncertainty, allowing the controller to blend probabilities for a smoother steering angle. For environmental awareness, the system employs a version of YOLO for detecting people or pets and Direct Sparse Odometry (DSO) for reactive obstacle detection via depth estimation.

Experimental Results

The system was evaluated on challenging forest trails in the Pacific Northwest. In a 250m "zigzag" test, TrailNet achieved 100% autonomy with zero human interventions, outperforming standard ResNet-18 (88%) and previous state-of-the-art models (80%). The MAV successfully completed autonomous flights of up to 1 km in unstructured environments. Disturbance tests confirmed that the dual-prediction model recovers to the trail center significantly faster than orientation-only models. The entire processing stack utilizes approximately 80% of the CPU and 100% of the GPU on the Jetson TX1.

Conclusion and Future Work

This research demonstrates that combining orientation and lateral offset predictions with probabilistic control leads to stable and autonomous MAV navigation in complex environments. While highly effective on defined trails, the system currently struggles in open fields lacking distinct features. Future work will focus on optimizing object detection frame rates and handling complex navigation decisions such as trail switching.

Summary updated

Abstract

We present a micro aerial vehicle (MAV) system, built with inexpensive off-the-shelf hardware, for autonomously following trails in unstructured, outdoor environments such as forests. The system introduces a deep neural network (DNN) called TrailNet for estimating the view orientation and lateral offset of the MAV with respect to the trail center. The DNN-based controller achieves stable flight without oscillations by avoiding overconfident behavior through a loss function that includes both label smoothing and entropy reward. In addition to the TrailNet DNN, the system also utilizes vision modules for environmental awareness, including another DNN for object detection and a visual odometry component for estimating depth for the purpose of low-level obstacle detection. All vision systems run in real time on board the MAV via a Jetson TX1. We provide details on the hardware and software used, as well as implementation details. We present experiments showing the ability of our system to navigate forest trails more robustly than previous techniques, including autonomous flights of 1 km.
BibTeX
@inproceedings{smolyanskiy2017toward,
  author = {Nikolai Smolyanskiy and Alexey Kamenev and Jeffrey Smith and Stan Birchfield},
  title = {{Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  url = {https://openreview.net/forum?id=rI6CqOA8vos},
}
Figure