TL;DR
This paper introduces a flexible, data-driven framework for uncertainty quantification (UQ) in safe learning-based control that ditches restrictive assumptions like known Lipschitz constants or RKHS norm bounds. By leveraging the scenario approach to sample data-consistent function realizations, the authors construct robust "uncertainty tubes" and demonstrate zero-violation safety tuning on a real-world Furuta pendulum.
What Problem Does This Paper Address?
When deploying learning-based controllers (like reinforcement learning) in safety-critical, real-world systems, we cannot afford a "trial and error" phase that breaks the hardware. Safe Bayesian Optimization (BO) algorithms like SAFEOPT solve this by constructing probabilistic "uncertainty tubes" around unknown reward and constraint functions, ensuring the system only experiments within a verified safe parameter space.
However, existing UQ methods suffer from a major practicality gap: they typically require users to know strict analytical properties of the system, such as a tight upper bound on the Reproducing Kernel Hilbert Space (RKHS) norm or the exact Lipschitz constant. In practice, these constants are notoriously difficult to estimate. Furthermore, standard Gaussian Process (GP) models struggle to capture sharp discontinuities. This paper addresses these limitations by replacing rigid analytical bounds with a purely data-driven sampling approach.
Methods
The authors propose Function-Based Uncertainty Quantification, which models the unknown system functions using a finite linear expansion of user-chosen basis functions with random coefficients:
$$hi(a) = \sum_{r=1}^N ci,r \phi_{i,r}(a)$$
This formulation is highly flexible; by choosing different bases (e.g., Haar wavelets), it can naturally accommodate discontinuous or heavy-tailed behaviors that frustrate traditional GP methods.
To build the uncertainty tubes without knowing Lipschitz constants, the authors turn to the scenario approach—a data-driven decision-making tool. The process works as follows:
- Scenario Generation: They sample independent and identically distributed (i.i.d.) random function realizations ("scenarios") from the prior distribution and project them to match the observed experimental data.
- Tube Construction: They solve a convex optimization problem to find the tightest pointwise upper and lower bounds that enclose all generated scenarios.
- Wait-and-Judge Optimization: Instead of using the conservative classic scenario approach, they implement a "wait-and-judge" framework (Algorithm 1). This evaluates the number of support scenarios (the essential samples defining the boundaries) a posteriori, drastically reducing the number of samples needed to guarantee safety.
Finally, they integrate these tubes into a safe BO loop (Algorithm 2) to iteratively expand the safe set and optimize controller parameters.
Datasets
The method was evaluated using synthetic benchmarks and real-world hardware rather than standard static datasets:
- Synthetic 1D Optimization Benchmark: Used to compare the efficiency of the classic scenario approach against the wait-and-judge framework.
- Haar Wavelet Discontinuous Benchmark: A synthetic test demonstrating UQ on a highly non-smooth function.
- Physical Hardware: Experimental validation on a real Quanser Furuta Pendulum (QUBE-Servo 2) setup to safely tune a linear state-feedback controller.
Key Insights and Ideas
- Sampling Over Analytical Bounding: Instead of trying to mathematically bound the worst-case behavior of an abstract function space, the method simply asks: "What do valid, data-consistent realizations of this system look like?" Pointwise minimization and maximization over these samples yield valid probabilistic bounds.
- The Power of "Wait-and-Judge": The classic scenario approach suffers from the curse of dimensionality because sample size requirements scale with the discretization of the domain. By switching to a wait-and-judge approach, the algorithm dynamically checks if the collected tubes are tight enough, reducing sample complexity by orders of magnitude.
- Discontinuity-Friendliness: Because the framework accepts arbitrary basis functions, it can seamlessly handle non-smooth systems (e.g., friction thresholds or hard obstacles) by utilizing wavelets instead of smooth squared-exponential kernels.
Key Results
- Massive Sample Reduction: In a synthetic 1000-point discretization test, the classic scenario approach required 21,403 scenarios to guarantee the target confidence level, resulting in overly conservative bounds. The wait-and-judge framework achieved the exact same probabilistic guarantees using only 596 scenarios, providing a much tighter and more useful uncertainty tube.
- Hardware Deployment: The safe BO algorithm successfully optimized the controller parameters of a real Furuta pendulum over 20 iterations with zero safety violations.
The optimized policy significantly stabilized the system, drastically reducing pendulum angle oscillations around the upright position compared to the baseline policy.
Limitations and Future Work
While powerful, the authors highlight a few areas for improvement:
- Per-Iteration vs. Joint Safety: The current mathematical proof guarantees safety per iteration. To achieve full joint safety across an entire multi-step optimization horizon simultaneously, a classical union bound cannot be easily applied because the probability measure changes as data is collected. The authors currently use a heuristic confidence-tightening approach to mitigate this.
- Model Misspecification: The framework assumes the chosen basis functions can adequately represent the true underlying system. Performance under severe model misspecification remains an open question.
- Broader Control Paradigms: Future work aims to extend this function-based UQ method beyond safe BO into Model Predictive Control (MPC) and formal verification.
Summary updated