TL;DR
This paper introduces an integrated framework for imitating complex, non-convex Time-Optimal Model Predictive Control (TOMPC) using neural networks. By leveraging a novel "Self-Informed Sampling" (SIFS) strategy based on a learned time-to-go metric, the authors achieve real-time control frequencies of 1 kHz on a physical Furuta pendulum—a 700-fold speedup over the expert controller—while using significantly less training data than standard methods.
What Problem Does This Paper Address?
Model Predictive Control (MPC) is a powerful tool for controlling systems with complex constraints, but it is computationally expensive because it solves an optimization problem at every time step. For "Time-Optimal" MPC (TOMPC), which aims to reach a target in the absolute minimum time, the math becomes even more grueling (non-convex and long-horizon), often making it too slow for fast-moving robots.
While researchers have tried using Neural Networks (NNs) to "imitate" MPC to save time, several hurdles remain:
- Data Scarcity: Generating "expert" training data from a slow TOMPC is time-consuming.
- Covariate Shift: NNs often drift away from the expert's path and don't know how to recover.
- Hard Constraints: NNs are smooth functions and struggle to respect strict physical limits (like motor torque or joint angles).
- Sim-to-Real Gap: A network trained in simulation often fails on real hardware due to model inaccuracies.
Methods
The authors propose a complete architecture that handles the entire pipeline from data generation to real-world deployment.
Key Innovations:
- Self-Informed Sampling (SIFS): Instead of randomly sampling states or using complex uncertainty metrics, the NN is trained to predict the Time-to-Go (T2G). In a perfect time-optimal trajectory, T2G should decrease linearly. If the NN's predicted T2G deviates from this trend during a simulation rollout, the system flags that state as "difficult" and queries the expert MPC for the correct action.
- T2G-Based Weighting: Non-convex optimization can produce multiple valid solutions for the same state (multimodality). The authors use a weighting scheme based on T2G to prioritize higher-quality, more optimal expert demonstrations during training.
- Ancillary Controllers: To bridge the sim-to-real gap, the nominal NN output is adjusted by a lightweight secondary controller (either a Dynamic Inversion Compensator or an LQR) that corrects for plant-model mismatch online.
- Safety Filtering: A differentiable safety layer ensures the final control signal strictly respects physical constraints.
Datasets
The researchers evaluated their framework using a simulated and physical Furuta Pendulum (a rotary inverted pendulum).
- Initial Dataset: 15,601 grid-based data points.
- Aggregated Dataset: Through iterative training, they generated ~1.2 million points, which were filtered down to ~700,000 using optimal sampling to reduce correlation and redundancy.
Key Insights and Ideas
- T2G as a Health Check: The most elegant insight is using T2G as a direct indicator of policy performance. Because T2G has a known, predictable behavior (it must drop by exactly the sampling interval each step), any deviation is a clear signal that the controller is struggling.
- Efficiency over DAgger: Standard dataset aggregation (DAgger) queries the expert at every step of a rollout. SIFS only queries the expert when the T2G trend breaks, drastically reducing the computational burden of generating training data.
- Holistic Integration: The paper argues that solving individual problems (like just constraints or just data efficiency) isn't enough for real-world robotics; you need an integrated stack to handle the interdependencies between these challenges.
Key Results
- Speed: The control frequency jumped from 1.4 Hz (expert MPC) to 1 kHz (NNMPC on a GPU), enabling real-time swing-up and stabilization.
- Data Efficiency: SIFS reached comparable performance to DAgger three times faster and required less than half the training data.
- Robustness: The integrated architecture matched the expert's "region of attraction" (the set of states from which it can successfully recover). The ancillary controllers significantly improved success rates when the physical model parameters were intentionally perturbed.
Limitations and Future Work
- Multimodality: While the weighting scheme helps, the authors note that conflicting expert labels (where two different actions are both "optimal") still pose a challenge for standard NNs. They suggest exploring Mixture Density Networks in the future.
- Dataset Growth: As the dataset expands through iterative sampling, training times naturally increase. The authors point toward continual learning as a way to manage this growth without retraining from scratch every time.
- Hardware Oscillations: Real-world tests showed some high-frequency oscillations near equilibrium due to sensor noise, suggesting a need for better output filtering or noise-aware training.
Summary updated