Abstract
Reinforcement Learning (RL) and other learning-based controllers offer significant potential for complex robotic tasks but often lack the formal safety guarantees required for real-world deployment. Safety filters, specifically Model Predictive Safety Filters (MPSFs), are commonly used to bridge this gap by minimally modifying control inputs to ensure constraint satisfaction. However, standard one-step MPSFs frequently suffer from "chattering"—high-frequency oscillations near safety boundaries. This paper introduces a Multi-Step MPSF that extends the prediction horizon to simulate the controller's future intent. By proactively correcting trajectories over a multi-step window, the proposed filter reduces chattering by over 75% in real-world drone experiments while maintaining recursive feasibility and computational efficiency.
Introduction
Deploying learning-based controllers on physical hardware, such as autonomous drones or self-driving vehicles, requires a robust mechanism to prevent safety violations. Safety filters act as a modular "guardian" layer: they monitor the commands from an uncertified controller (e.g., a neural network) and, if a command would lead to a constraint violation, project it onto the set of safe control inputs.
While effective at preventing catastrophic failure, traditional MPSFs typically operate with a one-step-ahead horizon. This myopic view leads to a phenomenon known as chattering. When a controller pushes the system toward a safety limit, a one-step filter intervenes abruptly at the last possible moment. This creates a cycle of aggressive correction and immediate return to the boundary, resulting in jerky, oscillatory behavior that can damage hardware and excite unmodeled dynamics. This paper proposes a Multi-Step MPSF framework designed to eliminate these oscillations by incorporating long-term foresight into the safety intervention process.
Methodology
Simulating Controller Intent
The core innovation of the Multi-Step MPSF is the simulation of the uncertified controller’s future behavior. Rather than evaluating only the current input, the filter uses a nominal model of the system to predict where the controller intends to go over a horizon of $M$ time steps. This allows the filter to identify potential safety violations much earlier in the trajectory.
Multi-Step Optimization
The filter solves an optimization problem that minimizes the discrepancy between the proposed (uncertified) inputs and the certified (safe) inputs over the entire $M$-step window. By spreading the necessary corrections over a longer period, the filter can apply "gentle steering" rather than "emergency braking." Mathematically, this framework generalizes the standard MPSF; when the horizon $M$ is set to 1, the system reverts to the traditional one-step formulation.
Robustness and Recursive Feasibility
To ensure the system remains safe even under uncertainty, the authors employ techniques from Robust Tube-based Model Predictive Control (MPC). The framework provides a mathematical guarantee of recursive feasibility: if the system begins in a safe state, the filter is guaranteed to find a valid sequence of safe inputs for all future time steps, accounting for bounded model errors.
Experimental Evaluation
Simulation Benchmarks
The approach was first validated using safe-control-gym, a benchmark suite for safe robot control. Testing across cartpole, 2D quadrotor, and 3D quadrotor environments demonstrated that the Multi-Step MPSF consistently smoothed trajectories where one-step filters failed. In simulations, the norm of the rate of change of inputs—a standard metric for jerkiness—was reduced by up to 73%.
Hardware Implementation
Real-world experiments were conducted using a Crazyflie 2.0 nano-quadrotor. The drone was tasked with trajectory tracking near strict position and velocity constraints. The Multi-Step MPSF was implemented to run onboard at 25Hz. Even with a prediction horizon of $M=10$, the computational overhead remained manageable, with solve times increasing by only approximately 60% compared to the one-step version.
Key Results and Insights
- Significant Chattering Reduction: In physical experiments, the multi-step filter reduced the peak-to-peak amplitude of chattering by 78% (from 16.3 cm in the one-step case to 3.6 cm).
- Proactive Intervention: Unlike standard regularization techniques that penalize input changes regardless of safety, the Multi-Step MPSF only intervenes when a boundary is threatened. This results in lower overall "correction effort" while maintaining higher safety margins.
- Black-Box Compatibility: The framework does not require knowledge of the internal structure of the RL controller. It only requires the ability to "query" the controller for its suggested input at a given simulated state.
Limitations and Future Work
While the Multi-Step MPSF significantly improves performance, it relies on the accuracy of the nominal system model. If the model used for simulation deviates significantly from reality, the predicted intent of the controller will be inaccurate. Additionally, the computational complexity scales with the horizon $M$, which may pose challenges for extremely high-dimensional systems.
Future research directions include adapting the filter for human-in-the-loop systems. Currently, the filter requires querying a controller for future states—a task that is straightforward for an algorithm but difficult for a human pilot. Developing predictive models of human intent could allow this proactive safety framework to be applied to teleoperated robotics.
Summary updated