Perception-Aware Trajectory Planning for Agile Quadrotors

This direction is merged into another direction: perception-aware trajectory optimization. Its last version stays readable and citable here.

Version 1 of 116 papers22 conceptsfrozen 2026-10-04

In the world of autonomous flight, there is a fundamental conflict between speed and sight. For a quadrotor to accelerate forward, it must tilt its entire body. However, because cameras are typically fixed to the drone's frame, this aggressive tilting often points the "eyes" of the robot toward the ground or the sky, away from the very gates or landmarks it needs to see to navigate.

Perception-aware planning is the research direction dedicated to solving this conflict. Instead of treating perception and motion as independent stages—where a planner decides where to go and a vision system tries to keep up—this approach treats visibility as a core constraint of the flight trajectory itself.

The Visibility-Agility Trade-off

Standard trajectory planners often prioritize time-optimality, finding the fastest path through a set of waypoints. As noted in AlphaPilot: Autonomous Drone Racing by Philipp Foehn and Davide Scaramuzza, these planners can be "perception-blind." They may command a maneuver so aggressive that the drone loses sight of the next gate, leading to a total failure in state estimation and a subsequent crash.

The core intuition of perception-aware planning is that a robot should never move into "unobserved space." Sikang Liu and Vijay Kumar addressed this in Towards Search-based Motion Planning for Micro Aerial Vehicles by coupling the vehicle's heading with its velocity vector. By forcing the drone to "look where it’s going," the system ensures that directional sensors maintain coverage over the intended flight path.

Integrating Field-of-View as a Constraint

To make this work during high-speed maneuvers, researchers integrate the camera's Field-of-View (FOV) directly into the mathematical optimization of the trajectory. This is often done in one of two ways:

  1. Hard Constraints: In Flying through Moving Gates without Full State Estimation, Ralf Römer and Angela P. Schoellig model the FOV as a hard geometric constraint. The optimizer is forbidden from choosing any orientation that would put the target gate outside the camera's view.
  2. Linearized Visibility: Because FOV constraints are non-linear and computationally "heavy," Jeffrey Mao and Giuseppe Loianno utilize Taylor series expansion to linearize the relationship between camera orientation and target visibility in Robust Active Visual Perching with Quadrotors on Inclined Surfaces. This allows for real-time replanning during aggressive perching maneuvers.

Proactive Perception in Racing

In autonomous drone racing, the stakes are higher. Drones must not only see the gate they are currently passing but also "look ahead" to the next one to minimize drift. Hyunchul Shim demonstrates this in Drift-Corrected Monocular VIO and Perception-Aware Planning for Autonomous Drone Racing, where the drone proactively orients its sensors toward upcoming gates to provide a continuous stream of visual data for the Visual-Inertial Odometry (VIO) system.

Similarly, Fangguo Zhao and Shuo Li developed Pairwise Model Predictive Control (PMPC) to handle multi-drone scenarios. In Online Time-Optimal Trajectory Generation for Two Quadrotors with Multi-Waypoints Constraints, they use a "Magnetic Induction Line" approach to guide the drone through gates while ensuring the heading remains perception-aware, preventing the loss of waypoints during high-speed tilts.

Learning to Look

While classical optimization is powerful, it can be slow. Recent work has shifted toward end-to-end learning of these behaviors. In Learning Perception-Aware Agile Flight in Cluttered Environments, Yunlong Song and Davide Scaramuzza trained a neural network policy that implicitly learns the trade-off between speed and visibility. By using a "learning-by-cheating" framework—where the agent has access to perfect information during training—the resulting policy naturally learns to slow down or adjust its tilt in cluttered spaces to ensure the camera stays pointed at the future path.

By treating perception as a dynamic requirement rather than a static sensor feed, these researchers have enabled quadrotors to navigate narrow gaps, perch on walls, and race at speeds exceeding 50 km/h, all while relying solely on onboard vision.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_perception_aware_trajectory_planning_for_a7d1ee94,
  title = {Perception-Aware Trajectory Planning for Agile Quadrotors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/perception-aware-trajectory-planning-for-agile-quadrotors/rdr_a7d1ee94da40479e86b9541876a78937},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_perception_aware_trajectory_planning_for_a7d1ee94_v1,
  title = {Perception-Aware Trajectory Planning for Agile Quadrotors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/perception-aware-trajectory-planning-for-agile-quadrotors/rdr_a7d1ee94da40479e86b9541876a78937/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}