Directing Cinematographic Drones

Quentin Galvane, Christophe Lino, Marc Christie, Julien Fleureau, Fabien Servant, François-Louis Tariolle, Philippe Guillotel

TL;DR

This paper introduces a high-level control system that allows a single user to act as both pilot and cinematographer for one or more drones. By mapping complex drone physics and framing constraints into a specialized "Drone Toric Space," the system enables real-time path planning, interactive sketching of trajectories, and automated multi-drone coordination to capture professional-grade shots of dynamic targets.

What Problem Does This Paper Address?

In professional filmmaking, drone shots typically require two experts: a pilot to handle flight and a cinematographer to control the camera. For amateurs or solo creators, synchronizing these tasks is incredibly difficult. While existing tools can automate "follow-me" behaviors or static waypoints, they often fail to:

  1. Respect Cinematographic Principles: Maintaining specific framing, distances, or angles on moving targets.
  2. Ensure Physical Feasibility: Generating paths that a quadrotor can actually fly without jerky movements or violating gimbal limits.
  3. Handle Dynamic Environments: Avoiding collisions with moving obstacles while keeping the "shot" intact.
  4. Coordinate Multiple Drones: Ensuring multiple cameras provide complementary views without filming each other.

Methods

The authors propose a multi-layered architecture centered on a novel mathematical representation:

  • Drone Toric Space (DTS): This is a re-parameterization of the "Toric Space" (a spindle torus surface where every point maintains a specific angle between two targets). The authors adapt this for drones by embedding safety distances ($d_S$) and physical constraints (like gimbal tilt limits) directly into the coordinate system.
  • Real-time Path Planning: The system uses a visibility-aware roadmap. It searches for the shortest path in a 4D $\tau$-space $(\alpha, \phi, \theta, z)$ using an A* algorithm. This ensures that the drone doesn't just take the shortest physical path, but the "shortest" path in terms of maintaining visual properties.
  • $C^4$ Continuous Trajectories: To ensure smooth flight, the system generates piecewise polynomial trajectories of degree 5. Unlike standard $C^2$ splines, $C^4$ continuity accounts for the "snap" (the fourth derivative of position), which is critical for stable quadrotor dynamics.
  • Sketching Interface: Users can draw a 2D path on a screen, which the system then projects into the 3D environment, refines for height, and optimizes into a collision-free flight path.
  • Multi-Drone Coordination: Using a master/slave relationship, the system employs a "min-conflict" optimization to assign different "framing instances" (e.g., over-the-shoulder, apex shot) to slave drones while ensuring they stay out of the master drone's view.

Datasets

The paper does not use standard large-scale datasets (like ImageNet). Instead, it evaluates the system using:

  • Simulated Environments: Various 3D scenes created in Unity 5 with different obstacle densities and target behaviors.
  • Real-world Lab Setup: A $15 \times 10 \times 5$ meter flight volume equipped with a Vicon motion capture system for high-precision tracking of drones and human actors.

Key Insights and Ideas

  • Constraints as Geometry: By building safety buffers and gimbal limits directly into the Drone Toric Space, the "search space" for valid shots is pre-filtered. If a user tries to move the camera into an "illegal" position (e.g., too close to a target), the geometry of the DTS naturally pushes the drone to the nearest safe location.
  • Visual Property Interpolation: Traditional path planning interpolates between $(x, y, z)$ coordinates. This paper interpolates between visual properties (like screen size and angle). This results in much smoother-looking footage where the subject's framing evolves naturally rather than jumping.
  • The "Director" Abstraction: The system treats the user as a high-level director. Instead of "fly left," the user says "keep this framing while I draw this path," and the system handles the low-level motor commands and obstacle avoidance.

Key Results

  • User Performance: In a study with 12 novices, the proposed tools significantly outperformed manual drone controllers. Users were able to achieve precise framing and fluid motions that were impossible for them to replicate manually.
  • Computational Efficiency: The path planning and optimization algorithms consistently run in under 200ms, making the system responsive enough for live, dynamic scenes.
  • Smoothness: Quantitative analysis showed that the $C^4$ trajectories and DTS-based planning resulted in significantly fewer variations in target size and orientation compared to standard Euclidean shortest-path methods.

Limitations and Future Work

  • Hardware Constraints: The real-world tests used Parrot ARDrone2 units, which lack stabilized gimbals. The authors note that the system would perform even better with modern, gimbal-equipped drones.
  • Reactive vs. Proactive: The current system is largely reactive. It responds to target movement but does not yet "anticipate" where a target is going to move next.
  • Complex Obstacles: While it handles dynamic obstacles, extremely cluttered environments (like a dense forest) might still challenge the roadmap-based planner.
  • Future Direction: The authors suggest integrating more advanced "anticipation schemes" and exploring more complex multi-drone behaviors, such as automated "editing" where the system chooses the best camera feed in real-time.

Summary updated

Abstract

Quadrotor drones equipped with high quality cameras have rapidely raised as novel, cheap and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth -- and cinematographic -- control of a camera drone remains challenging for most users, despite recent tools that either automate part of the process or enable the manual design of waypoints to create drone trajectories. This paper proposes to move a step further towards more accessible cinematographic drones by designing techniques to automatically or interactively plan quadrotor drone motions in 3D dynamic environments that satisfy both cinematographic and physical quadrotor constraints. We first propose the design of a Drone Toric Space as a dedicated camera parameter space with embedded constraints and derive some intuitive on-screen viewpoint manipulators. Second, we propose a specific path planning technique which ensures both that cinematographic properties can be enforced along the path, and that the path is physically feasible by a quadrotor drone. At last, we build on the Drone Toric Space and the specific path planning technique to coordinate the motion of multiple drones around dynamic targets. A number of results then demonstrate the interactive and automated capacities of our approaches on a number of use-cases.
BibTeX
@article{galvane2018directing,
  author = {Quentin Galvane and Christophe Lino and Marc Christie and Julien Fleureau and Fabien Servant and François-Louis Tariolle and Philippe Guillotel},
  title = {{Directing Cinematographic Drones}},
  journal = {ACM Trans. Graphics},
  year = {2018},
  doi = {10.1145/3181975},
}

Loading related content...

Figure