Autonomous Flights in Dynamic Environments with Onboard Vision

Yingjian Wang, Jialin Ji, Qianhao Wang, Chao Xu, Fei Gao

Abstract

In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones into an occupancy grid map. To achieve dynamic avoidance, we design a planning method composed of modified kinodynamic path searching and gradient-based optimization. The method leverages manually constructed gradients without maintaining a signed distance field (SDF), making the planning procedure finished in milliseconds. We integrate the above methods into a customized quadrotor system and thoroughly test it in real-world experiments, verifying its effective collision avoidance in dynamic environments.

Introduction

Navigating a quadrotor through cluttered environments with moving obstacles—such as a room full of walking people—remains a significant challenge in robotics. This task requires a seamless integration of perception, environment representation, and real-time planning. Traditional methods often struggle with occlusions or rely on computationally expensive structures like Signed Distance Fields (SDFs), which are difficult to update dynamically on limited onboard hardware.

System overview showing real-world scene, depth map, and 3D trajectory planning around a dynamic obstacle modeled as an ellipsoid.

This work presents a robust, fully autonomous system that addresses these bottlenecks. By combining an occlusion-aware perception module with a lightning-fast, "ESDF-free" trajectory planner, the system enables drones to navigate complex scenes with planning latencies of less than 7 milliseconds.

Perception and Representation

The system processes raw depth images to distinguish between static and dynamic obstacles. A key innovation is the occlusion-aware Kalman Filter, which tracks objects even when they are temporarily hidden behind other structures, preventing the drone from "forgetting" an obstacle's presence.

To maintain map consistency, the researchers employ a "Re-Free" strategy. Initially, all detected clusters are treated as static. Once a cluster is confirmed as dynamic—based on consistent motion over a specific time horizon—the space it previously occupied is "freed" in the occupancy grid. This eliminates "ghost" obstacles caused by processing lag. For efficient computation, dynamic objects are modeled as 3D ellipsoids, providing a "Goldilocks" balance between accuracy and simplicity compared to complex meshes.

Trajectory Planning

The planning module follows a hierarchical approach to ensure both safety and physical feasibility (kinodynamics).

  1. Kinodynamic Path Searching: A modified A* algorithm finds a rough, time-optimal path that avoids static walls and the predicted future trajectories of moving ellipsoids.
  2. B-Spline Optimization: The rough path is refined into a smooth curve. Unlike traditional planners that rely on a global SDF, this system uses manually constructed gradients. This allows the drone to "feel" the repulsive forces of obstacles and adjust its trajectory in milliseconds.
Gradient-based trajectory optimization using anchor points to avoid both static and moving obstacles without a global SDF.

By avoiding the SDF bottleneck, the trajectory optimization finishes in roughly 5-7ms, significantly faster than competing methods that often exceed 14ms.

Experimental Results

The system was validated through extensive simulations and real-world experiments using a custom quadrotor equipped with an Intel RealSense D455 depth camera.

  • Perception Accuracy: In real-world benchmarks, the perception module reduced position estimation error to 0.18m (from 0.31m in previous methods) and velocity error to 0.29m/s (from 0.57m/s).
  • High-Speed Evasion: In live tests, the drone successfully navigated a cluttered room at 2.5m/s while dodging a person walking at 2m/s.
  • Computational Efficiency: The "ESDF-free" approach proved critical for running on limited onboard hardware, maintaining high-speed flight without sacrificing safety.

Conclusion and Future Work

While the system demonstrates state-of-the-art performance, it currently assumes obstacles move at a constant velocity. Future improvements could involve more complex motion models to handle erratic movements. Additionally, increasing the effective range of the depth sensors would allow for even higher flight speeds by extending the drone's reaction horizon.

Summary updated

Abstract

In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones into an occupancy grid map. To achieve dynamic avoidance, we design a planning method composed of modified kinodynamic path searching and gradient-based optimization. The method leverages manually constructed gradients without maintaining a signed distance field (SDF), making the planning procedure finished in milliseconds. We integrate the above methods into a customized quadrotor system and thoroughly test it in realworld experiments, verifying its effective collision avoidance in dynamic environments.
BibTeX
@inproceedings{wang2021autonomous,
  author = {Yingjian Wang and Jialin Ji and Qianhao Wang and Chao Xu and Fei Gao},
  title = {{Autonomous Flights in Dynamic Environments with Onboard Vision}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=RKnkDRQJnq},
}
Figure