Evaluating Robust, Perception Based Control with Quadrotors

Laura Jarin-Lipschitz, Rebecca Li, Ty Nguyen, Vijay Kumar, Nikolai Matni

Abstract

Traditionally, controllers and state estimators in robotic systems are designed independently. Controllers are often designed assuming perfect state estimation. However, state estimation methods such as Visual Inertial Odometry (VIO) drift over time and can cause the system to misbehave. While state estimation error can be corrected with the aid of GPS or motion capture, these complementary sensors are not always available or reliable. Recent work has shown that this issue can be dealt with by synthesizing robust controllers using a data-driven characterization of the perception error, and can bound the system's response to state estimation error using a robustness constraint. We investigate the application of this robust perception-based approach to a quadrotor model using VIO for state estimation and demonstrate the benefits and drawbacks of using this technique in simulation and hardware. Additionally, to make tuning easier, we introduce a new cost function to use in the control synthesis which allows one to take an existing controller and "robustify" it. To the best of our knowledge, this is the first robust perception-based controller implemented in real hardware, as well as one utilizing a data-driven perception model. We believe this as an important step towards safe, robust robots that explicitly account for the inherent dependence between perception and control.

Introduction

In robotics, the separation principle typically dictates that controllers and state estimators are designed independently. Controllers often operate under the assumption of receiving perfect state estimates. However, for lightweight robots in GPS-denied environments, state estimation relies heavily on Visual-Inertial Odometry (VIO), which is prone to drift and noise based on movement and lighting. When perception degrades, standard controllers can overreact to noisy estimates, leading to instability.

This paper bridges the gap between robust control theory and real-world robotics by implementing the first robust, perception-based controller on a physical quadrotor. By introducing a novel "imitation cost" function within the System Level Synthesis (SLS) framework, we provide a practical way to take a well-tuned, standard controller and "robustify" it against the inevitable drift of VIO.

The Snapdragon Flight quadrotor platform used for hardware validation.

Methods

The quadrotor is modeled as a Linear Time-Invariant (LTI) system linearized around hover. We utilize System Level Synthesis (SLS) to parameterize achievable closed-loop behaviors through three main steps:

  1. Data-Driven Error Modeling: Perception error is characterized using an $S$-slope derived from ground-truth vs. VIO data, acting as a worst-case bound on error degradation.
  2. Robustness Constraint: This $S$-slope is injected into the SLS optimization as a hard constraint to ensure mathematically bounded responses to perception errors.
  3. Imitation Cost Function: To simplify tuning, we introduce an imitation cost that minimizes the difference between the robust controller and a pre-existing nominal controller (e.g., a PD controller), subject to the robustness constraint.

Experiments were conducted in simulation (ROS/Gazebo) and on a 256g Snapdragon Flight quadrotor using VICON for ground truth.

Key Results and Insights

Our implementation represents the first successful hardware deployment of a robust, perception-based controller using a data-driven error model. In well-lit environments, the $L_1$ robust imitation controller matched the tracking performance of a highly tuned PD controller. Under degraded conditions, the robust controller maintained tracking and bounded perception error, whereas standard PD controllers suffered significantly.

Comparison of X position error across various controllers, highlighting the performance of the novel imitation cost approach.

A key insight was the "softness" of robust control; the system naturally learns to be more conservative. When state estimation spikes occur, the robust controller damps the response rather than aggressively chasing noisy measurements. However, we found that strict theoretical bounds (the $S$-slope) were often too conservative for hardware, requiring manual relaxation to achieve feasibility. While perception error stayed below theoretical maximums (e.g., 3.172m in hardware), these bounds are often larger than desired for precise navigation.

Conclusion and Future Work

The framework currently forces a trade-off that leans heavily toward minimizing perception error at the expense of tight trajectory tracking. Future work will explore Linear Time-Varying (LTV) System Level Synthesis to incorporate yaw into the control model. This would enable "active sensing," allowing the quadrotor to dynamically orient its sensors toward feature-rich areas to actively reduce perception uncertainty.

Summary updated

Abstract

Traditionally, controllers and state estimators in robotic systems are designed independently. Controllers are often designed assuming perfect state estimation. However, state estimation methods such as Visual Inertial Odometry (VIO) drift over time and can cause the system to misbehave. While state estimation error can be corrected with the aid of GPS or motion capture, these complementary sensors are not always available or reliable. Recent work has shown that this issue can be dealt with by synthesizing robust controllers using a data-driven characterization of the perception error, and can bound the system's response to state estimation error using a robustness constraint. We investigate the application of this robust perception-based approach to a quadrotor model using VIO for state estimation and demonstrate the benefits and drawbacks of using this technique in simulation and hardware. Additionally, to make tuning easier, we introduce a new cost function to use in the control synthesis which allows one to take an existing controller and "robustify" it. To the best of our knowledge, this is the first robust perception-based controller implemented in real hardware, as well as one utilizing a data-driven perception model. We believe this as an important step towards safe, robust robots that explicitly account for the inherent dependence between perception and control.
BibTeX
@inproceedings{jarinlipschitz2020evaluating,
  author = {Laura Jarin-Lipschitz and Rebecca Li and Ty Nguyen and Vijay Kumar and Nikolai Matni},
  title = {{Evaluating Robust, Perception Based Control with Quadrotors}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=tgWcePrHdq},
}
Figure