Abstract
Nonlinear model predictive control (NMPC) has proven to be an effective control method, but it is expensive to compute. This work demonstrates the use of hardware FPGA neural network controllers trained to imitate NMPC with supervised learning. We use these Neural Controllers (NCs) implemented on inexpensive embedded FPGA hardware for high frequency control on physical cartpole and F1TENTH race car. Our results show that the NCs match the control performance of the NMPCs in simulation and outperform it in reality, due to the faster control rate that is afforded by the quick FPGA NC inference. We demonstrate kHz control rates for a physical cartpole and offloading control to the FPGA hardware on the F1TENTH car.
Introduction
Nonlinear Model Predictive Control (NMPC) is a gold standard for controlling complex, non-linear systems because it optimizes current actions by predicting future states. However, NMPC is computationally demanding, typically requiring powerful desktop CPUs and often operating at relatively low frequencies (10–50 Hz). In fast-moving robotics, this latency is critical; a robot may move significantly between sensor acquisition and control output. This research explores whether a lightweight neural network can be trained to "copy" NMPC behavior and run on inexpensive, ultra-fast FPGA hardware to overcome these latency bottlenecks.
Methodology: Imitation Learning for Control
The researchers employ a strategy of Imitation Learning combined with Hardware Acceleration. The process begins with a "Teacher" model—a high-end PC running a sophisticated NMPC solver—to generate expert control data in simulation. A "Student" model, specifically a Multilayer Perceptron (MLP), is then trained to map sensor inputs directly to the optimal control actions defined by the teacher.
To deploy these models on resource-constrained hardware, the authors utilize quantization-aware training via QKeras and weight pruning. For the F1TENTH race car model, approximately 80% of the weights were pruned to save space without sacrificing accuracy. The final models are synthesized into hardware logic using the hls4ml framework and deployed on a Xilinx Zynq-7020 SoC. This allows the neural network to run in parallel, achieving inference latencies of less than 4 microseconds.
Hardware Implementation and Efficiency
By offloading inference to FPGA hardware, the neural networks achieve high-frequency control while maintaining a very low power profile. The implementation details highlight the efficiency of this approach for embedded systems.
As shown in the implementation table, the Cartpole model uses a 7-32-32-1 MLP architecture with a core logic power consumption of only 12 mW. The F1TENTH model, despite being larger (64-64-64-2), maintains a low core power draw of 13 mW. The entire SoC setup consumes less than 3W total, making it highly suitable for mobile robotics where power and weight are constrained.
Experimental Results
The models were evaluated on two distinct platforms: a physical Cartpole and a 1/10-scale F1TENTH race car.
CartPole
The hardware Neural Controller (NC) achieved a 1 kHz control rate. In physical testing, the NC reacted significantly faster to target changes than the NMPC running on a host PC. This high frequency allows for near-instantaneous corrections of small errors before they escalate, demonstrating that a simpler controller running at high speed can outperform a complex one running slowly.
F1TENTH Race Car
In simulation, the NC performed within 10% of the "perfect" NMPC teacher and was 30% faster than a standard Pure Pursuit (PP) baseline. On the physical race car, the advantage was even more pronounced: the NMPC could not run in real-time on the onboard CPU, maxing out at 10 Hz. The FPGA-based NC, however, completed laps 20% faster than the optimized Pure Pursuit baseline and tracked the race line with higher precision.
Conclusion and Future Work
This work demonstrates that hardware-accelerated neural controllers can effectively bridge the gap between expensive NMPC theory and real-time robotic application. While successful, the authors note the "Sim2Real" gap remains a challenge; performance can degrade if physical parameters like floor friction differ significantly from simulation. Future research will focus on Domain Randomization to increase robustness and Online Adaptation to allow the neural network to adjust to environmental changes on the fly.
Code and hardware implementation for this paper are available at https://github.com/SensorsINI/Neural-Control-Tools.
Summary updated