Classical optimal control relies on tracking a system's state step-by-step through time. Whether solving the Riccati differential equation for a Linear Quadratic Regulator (LQR) or using dynamic programming to solve the Bellman equation, traditional methods face severe bottlenecks when dealing with non-convex state constraints, unknown nonlinear dynamics, or infinite-dimensional systems (such as those governed by partial differential equations).
A powerful alternative paradigm bypasses these step-by-step temporal updates entirely. Instead of treating a control trajectory as a sequence of points in time, this approach lifts the entire controlled trajectory, or the system's underlying probability distributions, into an infinite-dimensional Reproducing Kernel Hilbert Space (RKHS). By doing so, complex dynamical constraints and cost functions are embedded directly into the geometry of the Hilbert space, transforming difficult control problems into clean, finite-dimensional convex optimization problems.
The Core Intuition: From Time Steps to Trajectory Spaces
To understand why this framework works, consider the standard LQR problem, which minimizes a quadratic cost subject to linear system dynamics. Typically, we solve this by finding a time-varying matrix via a Riccati equation.
As demonstrated by Pierre-Cyril Aubin-Frankowski in Linearly Constrained Linear Quadratic Regulator from the Viewpoint of Kernel Methods, we can instead view the set of all valid, controlled trajectories as forming an RKHS. In this abstract space, the inner product is explicitly engineered to incorporate both the system dynamics and the quadratic cost function.
By framing the trajectory itself as a function in an RKHS, the control problem undergoes a radical simplification:
- Dynamics as Geometry: The requirement that a trajectory must satisfy the system's differential equations is baked directly into the space's kernel.
- Control as Regression: Finding the optimal control policy becomes mathematically equivalent to performing kernel ridge regression.
This perspective becomes exceptionally powerful when adding affine state constraints (such as avoiding obstacles). While traditional methods struggle when constraints become active, the RKHS formulation allows these constraints to be treated as simple linear inequalities within the optimization space, resulting in an exact, interior solution via standard convex solvers.
Traditional Control:
[State x_0] --(u_0)--> [State x_1] --(u_1)--> [State x_2] --> (Solve step-by-step via Riccati/Bellman)
RKHS Control:
[ Entire Trajectory (x_t, u_t) ] ===> Mapped as a single point in an RKHS
(Dynamics & costs encoded in the kernel; solve via Convex QP/LP)
Operator-Valued Kernels for Infinite Dimensions
The utility of this framework extends far beyond standard finite-dimensional systems. In Operator-valued Kernels and Control of Infinite dimensional Dynamic Systems, Pierre-Cyril Aubin-Frankowski extends this methodology to infinite-dimensional linear systems, such as those described by partial differential equations (PDEs).
To achieve this, the author introduces operator-valued kernels. Instead of outputting a scalar value, these kernels output linear operators that map inputs to outputs across infinite-dimensional spaces. Derived directly from the solutions of the associated differential Riccati equations, these kernels encapsulate both the system's evolution and its control costs into a unified representation.
This functional alternative to traditional control laws is highly versatile. For instance, in Reproducing Kernel Approach to Linear-Quadratic Mean Field Control Problems with Additive Noise, Pierre-Cyril Aubin-Frankowski applies this framework to mean-field control problems (which govern massive populations of interacting agents). By optimizing over push-forward maps in the Hilbert space rather than individual control trajectories, the author bypasses the traditional complexities of Pontryagin's maximum principle, yielding closed-form solutions even in the presence of nonlinear terminal costs and additive noise.
Embedding Distributions for Stochastic Control
When a system is stochastic or its dynamics are entirely unknown, tracking exact trajectories becomes impossible. However, the kernel framework adapts naturally to this setting by shifting focus from trajectory points to probability distributions.
In Stochastic Optimal Control via Hilbert Space Embeddings of Distributions, Adam Thorpe et al. demonstrate that by embedding probability distributions and expectation operators into an RKHS, the stochastic optimal control problem can be completely reformulated.
Instead of dealing with intractable integrals over arbitrary disturbances, the expectation of a cost function can be expressed as a linear combination of kernel evaluations. This transformation linearizes the objective function with respect to the policy coefficients, turning a notoriously difficult stochastic optimization problem into a highly efficient Linear Program (LP). This formulation can be solved via its Lagrangian dual, completely bypassing the need for error-prone gradient-based optimization methods.
This distribution-embedding approach forms the core engine of several data-driven control frameworks:
- SOCKS Toolbox: In SOCKS: A Stochastic Optimal Control and Reachability Toolbox Using Kernel Methods, the authors leverage these expectation embeddings to build a model-free toolbox capable of performing stochastic optimal control and reachability analysis for systems with unknown dynamics and arbitrary cost functions.
- Physics-Informed Control: To mitigate the high sample complexity of purely data-driven methods, Adam Thorpe et al. introduce physics-informed kernel embeddings in Physics-Informed Kernel Embeddings: Integrating Prior System Knowledge with Data-Driven Control. By constraining the kernel embeddings to satisfy known physical laws, they drastically improve sample efficiency and prediction accuracy when data is scarce.
- Nonparametric Adaptation: Similarly, Stephen Tu et al. in Nonparametric Adaptive Control and Prediction: Theory and Randomized Algorithms utilize the expressiveness of RKHS to learn complex, high-dimensional system dynamics online. They employ random feature approximations to maintain computational efficiencies that rival classical parametric methods while capturing a significantly wider range of system behaviors.
Why It Works: The Representer Theorem
The idea of mapping control problems into an infinite-dimensional Hilbert space might initially sound like it would make optimization harder, not easier. The reason this approach is computationally viable is due to the Representer Theorem.
The Representer Theorem guarantees that even though the RKHS is infinite-dimensional, the optimal control policy or trajectory minimizing the regularized cost function can always be expressed as a finite linear combination of the kernel functions evaluated at a discrete set of observed data points or constraint locations:
$$\gamma^*(x) = \sum_{i=1}^N \alpha_i K(x_i, x)$$
This theorem acts as a mathematical bridge, collapsing an infinite-dimensional variational problem down to the task of finding a finite set of coefficients $\alpha_i$. Consequently, whether optimizing a deterministic LQR with state constraints or learning a stochastic policy from data, the problem ultimately reduces to standard, finite-dimensional convex optimization (such as Quadratic Programs or Linear Programs). These can be solved rapidly and reliably using off-the-shelf numerical solvers, providing a mathematically rigorous, computationally efficient path for controlling highly complex systems.
Key Foundational Work
Operator-valued Kernels and Control of Infinite dimensional Dynamic Systems (2022) — contributed 6 concept elements to this direction.
Linearly Constrained Linear Quadratic Regulator from the Viewpoint of Kernel Methods (2021) — contributed 4 concept elements to this direction.
Stochastic Optimal Control via Hilbert Space Embeddings of Distributions (2021) — contributed 2 concept elements to this direction.