Abstract
Model predictive control (MPC) has played a more crucial role in various robotic control tasks, but its high computational requirements are concerning, especially for nonlinear dynamical models. This paper presents a latent linear quadratic regulator (LaLQR) that maps the state space into a latent space, on which the dynamical model is linear and the cost function is quadratic, allowing the efficient application of LQR. We jointly learn this alternative system by imitating the original MPC. Experiments show LaLQR's superior efficiency and generalization compared to other baselines.
Introduction
Controlling complex robotic systems, such as quadrupeds or humanoids, typically necessitates the use of nonlinear Model Predictive Control (MPC). While MPC offers high performance, it is computationally intensive, requiring the solution of complex optimization problems like Sequential Quadratic Programming (SQP) at every time step. This overhead often precludes deployment on low-power embedded hardware.
Existing alternatives often fall short:
- Local Linearization (LoLQR): Approximates the system as linear near a fixed point. While fast, it fails when the system deviates significantly from that point.
- Standard Imitation Learning (IL): Trains a neural network to map states directly to actions. Although efficient, IL is often brittle and lacks the structural robustness of physics-based controllers, failing in out-of-distribution scenarios.
This paper introduces Latent Linear Quadratic Regulator (LaLQR), which bridges the gap by learning a latent space where complex dynamics become linear and costs become quadratic. This allows for MPC-level performance with the computational efficiency of a simple matrix multiplication.
Methodology
The core of LaLQR is the transformation of a nonlinear system into a structured latent representation where the Linear Quadratic Regulator (LQR) framework can be applied.
Latent Embedding and Dynamics
A neural network embedding $\phi$ maps the raw state $x$ into a higher-dimensional latent state $z$. Inspired by Koopman Operator Theory, this "lifting" allows nonlinear dynamics to be represented linearly: $z_{t+1} = Az_t + Bu_t$. To ensure efficiency and inherent controllability, the matrices $A$ and $B$ are parameterized using a controllable companion form.
Quadratic Cost Mapping
To apply LQR, the cost function must also be quadratic. LaLQR learns a monotonic function $F$ that maps the original task-specific cost into a quadratic form in the latent space.
Learning via Imitation
Unlike standard IL, which imitates actions, LaLQR imitates the optimization problem. By training the network to recreate the expert MPC's behavior within the LQR structure, the model gains a physics-based inductive bias that improves generalization.
Inference
During deployment, the controller avoids online optimization. It uses a pre-computed static gain matrix $K$ derived from the discrete-time algebraic Riccati equation. The control law is a simple linear transformation: $u = -K\phi(x)$.
Experiments and Results
The performance of LaLQR was evaluated using the MuJoCo physics engine across several benchmarks, including Cartpole, Swimmer, Walker, and Humanoid.
Computational Efficiency
LaLQR demonstrates significant speedups over nonlinear MPC. In the Humanoid task ($n=55$ states), the MPC expert required 44.36ms per step, whereas LaLQR required only 5.48ms. On simpler tasks like Cartpole, the speedup reached approximately 40x.
Control Performance and Generalization
In complex tasks like the Humanoid, LaLQR significantly outperformed baselines. LaLQR achieved a total cost of 790.85, compared to 22,606.60 for standard IL and 18,456.23 for LoLQR. Furthermore, LaLQR exhibited superior robustness when tested on unseen initial states and noisy expert data, where standard IL typically failed to stabilize the robot.
Conclusion
LaLQR provides a robust framework for efficient robotic control by combining the speed of neural networks with the structural benefits of LQR. While it currently lacks a formal mechanism for hard safety constraints, its ability to generalize from expert data while maintaining high computational efficiency makes it a promising candidate for real-world embedded applications. Future work will explore sim-to-real transfer and integration with specialized embedded solvers.
Summary updated