Averaged-Cost Control with Shared Feedback under Model Uncertainty

Piernicola Bettiol, Daisuke Inoue, Matías Gómez-Aedo, Dante Kalise, Richard B. VinterarXiv 2026 · arXiv

TL;DR

This paper introduces a framework for designing a single "shared" feedback controller that performs optimally across a range of uncertain model parameters and initial states. By developing a nonlinear adjoint method for general systems and a highly efficient linear moment method for linear-quadratic models, the authors demonstrate that designing for the average case significantly outperforms standard "nominal" designs, especially as model uncertainty grows.

What Problem Does This Paper Address?

In real-world engineering, we rarely have a perfect model. Whether it's manufacturing tolerances in a robot or the "sim-to-real" gap when moving from a simulator to hardware, physical parameters often vary.

The challenge is designing a controller for repeated deployment. Ideally, you’d have a custom controller for every possible variation, but that’s impractical. Instead, we need a shared feedback law: one single policy that uses only the current time and state to make decisions, while balancing performance across all possible model realizations.

The specific hurdle the authors tackle is the information structure: the controller cannot see the actual model parameters (like friction or mass) online. It only sees the state. The goal is to minimize the "averaged cost" over a joint probability distribution of model parameters and initial conditions.

Methods

The authors propose two distinct computational paths depending on the system type:

  1. Nonlinear Adjoint Method: For general nonlinear systems (e.g., using neural networks as controllers), they use a trajectory-based approach. It involves:

  2. Forward solves: Simulating the closed-loop system for different realizations.

  3. Backward adjoints: Solving an adjoint differential equation to determine how sensitive the cost is to the controller's parameters.
  4. Policy Gradients: Averaging these sensitivities over the uncertainty distribution to update the controller.
  5. Linear Moment Method: For systems that are linear with quadratic costs (LQ), the authors provide a much more efficient shortcut. Instead of simulating thousands of individual "particles" (realizations), they propagate the conditional second moments of the state. By adapting classical Lyapunov and Gramian identities to the finite-horizon shared-gain problem, they derive exact gradient formulas that are independent of the number of samples, making the optimization significantly faster.

Datasets

The paper does not use static datasets but evaluates the methods on two classic nonlinear dynamical benchmarks:

  • Van der Pol Oscillator: A non-conservative oscillator with non-linear damping.
  • Stuart–Landau Oscillator: A common model for the nonlinear evolution of instabilities.

The authors compare Averaged Design (which considers the full distribution of uncertainty) against Nominal Design (which optimizes only for the mean/expected model parameter).

Key Insights and Ideas

  • The Power of Averaging: The "Averaged Design" doesn't just find a middle ground; it actively suppresses parameter-dependent oscillations. In the Van der Pol experiments, the averaged design produced a much "tighter" bundle of trajectories compared to the nominal design, which struggled when the realized model deviated from the mean.
  • Moment Closure in LQ Systems: One of the "aha" moments is that for linear systems, you don't need to know every possible trajectory to calculate the average cost. You only need the second moment of the state distribution. This allows for "exact" optimization that scales with the system dimension, not the number of uncertainty samples.
  • Joint Uncertainty: Unlike previous work that often treats model parameters and initial states as independent, this framework allows for a joint law. This is crucial for cases where the initial state might be correlated with the system's physical parameters.

Key Results

  • Cost Reduction: In nonlinear tests, the averaged neural feedback reduced costs by 21.9% for the Van der Pol system and 28.0% for the Stuart–Landau system compared to nominal designs.
  • Robustness to Variance: As the variance of the model parameters increases, the gap between averaged and nominal design widens significantly. Averaged design remains stable and low-cost, while nominal design costs spike (see Figure 6 in the paper).
  • Computational Efficiency: The Linear Moment Method proved to be orders of magnitude faster than the adjoint method. Its runtime remains nearly constant regardless of how many initial-state samples are considered, whereas the adjoint method's cost scales linearly with the sample count.

Limitations and Future Work

  • Local Optimality: The methods rely on gradients, meaning they find local minima. There is no guarantee of finding the global "best" controller in highly complex nonlinear landscapes.
  • Finite Horizon: The current work focuses on a fixed time window ($[S, T]$). Extending this to infinite-horizon problems while maintaining the efficiency of the moment method is a logical next step.
  • Sufficient Conditions: While the authors derive necessary conditions for optimality (first-order stationarity), they note that exploring sufficient conditions and formal convergence proofs for these specific algorithms remains an open area for research.

Summary updated

Abstract

This paper studies the shared-feedback design problem for finite-horizon averaged-cost control under model uncertainty represented by a joint law over the model parameter and initial state. One feedback map is shared across all realizations and uses only time and the current state as online inputs. We develop two methods: the nonlinear adjoint method combines forward closed-loop solves, backward adjoints, and joint-law policy gradients, while the linear moment method gives exact cost and gradient formulas for a homogeneous linear-quadratic design model by propagating conditional second moments. Numerical studies compare averaged and nominal designs for time-dependent neural feedback and static linear feedback in nonlinear Van der Pol and Stuart-Landau systems; the static gains are designed from oscillator linearizations and evaluated on the original nonlinear systems. Averaged design has lower cost in most comparisons.
BibTeX
@misc{bettiol2026averaged,
  author = {Piernicola Bettiol and Daisuke Inoue and Matías Gómez-Aedo and Dante Kalise and Richard B. Vinter},
  title = {{Averaged-Cost Control with Shared Feedback under Model Uncertainty}},
  year = {2026},
  eprint = {2611.19019},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19019},
}
Figure