Classical control theory relies on a elegant premise: if you can write down the differential equations that govern a physical system, you can compute the exact inputs needed to steer it. This approach powers everything from cruise control to rocket launches.
However, when a robot interacts with the real world, this premise breaks down. Writing explicit equations for the physics of cutting food, manipulating thousands of granular rice grains, or navigating a quadrotor through turbulent air is practically impossible.
To bridge this gap, a powerful paradigm has emerged: Model Predictive Control (MPC) with learned neural dynamics. Instead of manually deriving the laws of physics, researchers train deep neural networks to act as forward dynamics simulators. By embedding these learned models directly inside an online optimization loop, systems can safely and flexibly navigate complex, highly non-linear environments.
The Core Architecture: Learning to Predict the Future
At the heart of Model Predictive Control is a simple loop: predict what will happen over a short future horizon, evaluate those outcomes against a goal, execute the first optimal action, and repeat.
[Current State] ---> [ Neural Dynamics Model ] ---> [ Predicted Future States ]
^ |
| (Optimize Actions) v
[ MPC Optimizer ] <----------------- [ Cost Function ]
Traditionally, the "Neural Dynamics Model" block was a rigid physics engine. In this modern data-driven framework, it is a neural network trained to approximate the transition function:
$$s_{t+1} = f(s_t, a_t)$$
where $s_t$ is the current state and $a_t$ is the control input. As demonstrated by Lambert et al. in Low Level Control of a Quadrotor with Deep Model-Based Reinforcement learning, a deep neural network can learn this forward dynamics model directly from raw sensor inputs and motor commands. With just a few minutes of flight data, the learned model captures hardware nuances that traditional system identification misses, enabling stable low-level hovering.
Similarly, Zhao shows in Auto-tuning for cascade PID height position controller of rotorcraft that treating a physical plant as a "black box" and mapping historical input-output data via neural networks allows for the creation of highly accurate virtual simulation environments, bypassing the need for complex aerodynamic modeling.
Taming Non-Linearities and Hysteresis
Why use neural networks instead of simpler linear approximations? Real-world systems are riddled with non-linearities, friction, and memory effects like hysteresis.
For instance, pneumatic cylinders exhibit severe non-linear hysteresis due to air compressibility and seal friction. Wu et al. demonstrate in Data-Driven Feedforward Force Control of a Single-Acting Pneumatic Cylinder with a Nonlinear Hysteresis Characteristic that data-driven neural network modeling maps these intricate input-output relationships far more effectively than traditional analytical approaches, yielding precise force tracking.
When these models are accurate, task-appropriate behaviors emerge naturally from the optimization loop. In Deep Haptic Model Predictive Control for Robot-Assisted Dressing, Erickson et al. trained a deep network to predict the forces applied to a human during dressing. By pairing this predictive model with an MPC cost function that penalizes high forces, the robot inherently learned to navigate complex, multi-contact human interactions safely without explicit behavioral programming.
Overcoming the Markov Assumption: History-Dependent Dynamics
Standard MPC assumes the system is Markovian—meaning the next state depends only on the current state and action. But real-world physics often depends on the system's history due to unobserved variables, fluid turbulence, or time delays.
To capture these effects, researchers utilize Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks within the control loop.
- Time-Delayed Optimization: Theodorou et al. developed Delayed Differential Dynamic Programming (DDP) in Differential Dynamic Programming for time-delayed systems. By treating an RNN's reliance on past states as explicit time-delays, the authors enabled high-performance trajectory optimization over systems with unknown, history-dependent dynamics.
- Industrial Process Control: In RNN-LSTM-Based Model Predictive Control for a Corn-to-Sugar Process, Meng et al. used an RNN-LSTM network as the predictive engine for MPC. The recurrent model tracks long-term chemical reactions and process lags, allowing the controller to maintain target outputs despite external disturbances.
- Contact-Rich Tasks: Cutting food involves highly variable contact forces depending on the material's texture and thickness. In Data-Driven Model Predictive Control for the Contact-Rich Task of Food Cutting, Mitsioni et al. combined an RNN dynamics model with MPC. The network infers the changing material properties from the history of forces, allowing the controller to adapt on the fly to items never seen during training.
This capacity to approximate non-Markovian systems extends beyond robotics. For example, Jiang et al. show in Neural network aided approximation and parameter inference of non-Markovian models of gene expression how neural networks can map complex, history-dependent biological models into simpler, time-inhomogeneous Markovian representations for efficient simulation.
Online Adaptation and Complex State Spaces
A common failure mode of learned models is the distribution shift: a robot encounters a real-world scenario slightly different from its training data, makes a poor prediction, and the controller fails. Modern research addresses this through online adaptation and compact visual representations.
Real-Time Model Tuning
Instead of keeping the neural network static, systems can update their dynamics models online based on immediate feedback. In Self-Adapting Recurrent Models for Object Pushing from Learning in Simulation, Lin et al. introduced a hybrid control strategy (RMPPI) that uses a recurrent neural network to predict object pushing dynamics. By adapting the network's predictions in real time during the optimization process, they successfully bridged the simulation-to-reality gap.
Taking this further, Yu et al. proposed AD-NODE in AD-NODE: Adaptive Dynamics Learning with Neural ODEs for Mobile Robots Control. By leveraging Neural Ordinary Differential Equations (NODEs) to infer environmental states from past observations, their model adapts to changing terrain and environmental conditions dynamically, demonstrating robust path-tracking and generalization in unseen environments.
Controlling directly from Vision and Touch
When the state of a system cannot be reduced to simple coordinates (like positions and velocities), neural networks can learn dynamics directly over high-dimensional sensory spaces.
In Gaussian Splatting Visual MPC for Granular Media Manipulation, Zhang et al. integrated MPC with a learned visual dynamics model based on Gaussian Splatting. The controller optimizes robotic actions by minimizing the difference between the predicted future visual scene and the target arrangement of granular materials.
[Visual Input] ---> [ Spatial/Visual Model ] ---> [ Predicted Visual State ]
(e.g., U-Net /
Gaussian Splatting)
This principle applies to tactile sensing as well. George et al. demonstrate in BeadSight: An Inexpensive Tactile Sensor Using Hydrogel Beads that deep learning architectures like U-Net can map the complex, non-linear deformation of physical hydrogel beads directly to accurate pressure maps, bypassing the need for intractable physical contact equations.
Alternative Representations: Koopman and Physics-Informed Models
While black-box neural networks are highly flexible, embedding structural priors can make learning significantly more data-efficient.
One approach is to lift non-linear dynamics into a higher-dimensional space where the transitions become linear, known as Koopman operator theory. In Learning Bilinear Models of Actuated Koopman Generators from Partially Observed Trajectories, Otto et al. developed a method to learn bilinear Koopman models from noisy, partial observations. When integrated with MPC, these learned models successfully tracked reference trajectories in complex fluid dynamics simulations across entirely unseen operating conditions.
Alternatively, one can bake physical conservation laws directly into the network architecture. Lutter et al. introduced Deep Lagrangian Networks in Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems, enforcing the laws of Lagrangian mechanics within the network structure. This physics-informed constraint allows the model to capture hardware nuances efficiently, leading to high success rates in controlling complex under-actuated systems without requiring manual physics derivations.
Finally, to deploy these learned models safely in critical applications, researchers are developing verification tools. As shown by Hashemi et al. in Data-Driven Reachability Analysis of Stochastic Dynamical Systems with Conformal Inference, pairing learned neural network surrogate models with conformal inference makes it possible to provide rigorous probabilistic bounds on future system states, ensuring safety guarantees even when the underlying mathematical model is entirely unknown.
Key Foundational Work
Differential Dynamic Programming for time-delayed systems (2017) — contributed 1 concept elements to this direction.
Exploiting Chaos to Control the Future (1993) — contributed 1 concept elements to this direction.