Autonomous Flights in Dynamic Environments with Onboard Vision

Yingjian Wang, Jialin Ji, Qianhao Wang, Chao Xu, Fei Gao

TL;DR

This paper presents a robust, fully autonomous system for quadrotors to navigate cluttered environments with moving obstacles using only onboard vision. By combining an occlusion-aware perception module with a lightning-fast, "ESDF-free" trajectory planner, the researchers enabled drones to dodge walking people in real-time with planning latencies of less than 7 milliseconds.

What Problem Does This Paper Address?

Navigating a drone through a room full of moving people and static furniture is a "holy grail" challenge in robotics. It requires solving three massive problems simultaneously:

  1. Perception: Distinguishing between what is moving and what is still, even when objects briefly disappear behind others (occlusion).
  2. Representation: Modeling the world in a way that the drone’s "brain" can understand without hogging all the computational power.
  3. Planning: Finding a path that is not only safe but also physically possible for a drone to fly (kinodynamics) at high speeds.

Most existing solutions either assume they have perfect knowledge of the room or rely on computationally expensive maps called Signed Distance Fields (SDFs), which are slow to update when things move.

Methods

The authors propose a "complete system" divided into two main parts:

1. Occlusion-Aware Perception

The system takes raw depth images and clusters them into objects. To handle the "dynamic vs. static" problem, they use:

  • Occlusion-aware Kalman Filters: These track objects even when they are temporarily hidden, preventing the drone from "forgetting" a person is there just because they walked behind a pillar.
  • The "Re-Free" Strategy: Initially, the system treats everything as static. Once an object is confirmed as dynamic, the system "frees" the space it previously occupied in the map. This eliminates "ghost" obstacles caused by the lag in processing.

2. Hierarchical Planning

Instead of a single complex calculation, the planner works in two stages:

  • Kinodynamic Path Searching: A modified A* algorithm finds a rough, time-optimal path that avoids both static walls and the predicted future positions of moving ellipsoids.
  • B-Spline Optimization: The rough path is refined into a smooth curve. Crucially, they use manually constructed gradients instead of an SDF. This allows the drone to "feel" the push of obstacles and adjust its trajectory in milliseconds.

Datasets

The researchers evaluated their system using:

  • Simulation: A 40m x 10m x 3m virtual environment filled with static obstacles and multiple moving agents.
  • Real-World Experiments: Conducted using a custom quadrotor equipped with an Intel RealSense D455 depth camera.
  • Benchmark Comparison: They recorded a custom dataset of a human walking in a cluttered environment to compare their perception accuracy against existing state-of-the-art methods (specifically referencing work by Lin et al.).

Key Insights and Ideas

  • Ellipsoids for Efficiency: Modeling humans as moving 3D ellipsoids is a "Goldilocks" solution—it’s more accurate than a single point but far less computationally taxing than a full 3D mesh.
  • Ditching the SDF: By avoiding the creation of a global Signed Distance Field, the team removed the biggest bottleneck in traditional drone planning. This is what allows the system to run on limited onboard hardware.
  • Consistency is Key: A cluster is only labeled "dynamic" if it consistently shows motion over a specific time horizon ($t_h$), which prevents sensor noise from triggering unnecessary evasive maneuvers.

Key Results

  • Higher Accuracy: In real-world tests, their perception module reduced position estimation error to 0.18m (down from 0.31m in previous methods) and velocity error to 0.29m/s (down from 0.57m/s).
  • Blazing Speed: The trajectory optimization finishes in roughly 5-7ms. Competing methods often take 14-17ms or more, which is an eternity when a drone is flying at high speeds.
  • High-Speed Evasion: In live tests, the drone successfully navigated through a cluttered room at 2.5m/s while dodging a person walking at 2m/s.

Limitations and Future Work

  • Constant Velocity Assumption: The current Kalman Filter assumes obstacles move at a constant velocity. If a person suddenly lunges or changes direction erratically, the prediction might fail.
  • Sensor Range: The system is limited by the ~4-meter effective range of the depth camera. At higher flight speeds, the drone would need "longer-range eyes" to react in time.
  • Complex Geometry: While ellipsoids work well for humans, they might over-simplify more complex moving shapes (like a swinging boom arm), leading to overly conservative paths.

Summary updated

Abstract

In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones into an occupancy grid map. To achieve dynamic avoidance, we design a planning method composed of modified kinodynamic path searching and gradient-based optimization. The method leverages manually constructed gradients without maintaining a signed distance field (SDF), making the planning procedure finished in milliseconds. We integrate the above methods into a customized quadrotor system and thoroughly test it in real-world experiments, verifying its effective collision avoidance in dynamic environments.
BibTeX
@inproceedings{wang2021autonomous,
  author = {Yingjian Wang and Jialin Ji and Qianhao Wang and Chao Xu and Fei Gao},
  title = {{Autonomous Flights in Dynamic Environments with Onboard Vision}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=RKnkDRQJnq},
}

Loading related content...

Figure