Heavy Metal Acrobatics: Learning Dynamic Control for Skid-Steer Loaders

Pierre-Jean Lorentz, Pascal Egli, Edo Jelavic, Cliff Li, Ramitha Sundar, Harikrishnan Suresh, Paul Montgomery, Lee Redden, Matthias Müller, Ozan Sener, Vladlen Koltun, Marco HutterarXiv 2026 · DOI · arXiv

TL;DR

Researchers have successfully applied Reinforcement Learning (RL) to teach a 5-tonne John Deere skid-steer loader to perform "acrobatic" maneuvers, such as swinging up and balancing on two wheels. By combining a learned actuator model with domain randomization, the team achieved seamless sim-to-real transfer, enabling autonomous control at the physical limits of heavy machinery—tasks previously mastered only by elite human operators.

What Problem Does This Paper Address?

Automating heavy machinery like excavators and loaders is notoriously difficult. Unlike lightweight robots, these machines rely on complex hydraulic systems characterized by nonlinear delays, fluid dynamics, and flexible conduits that defy traditional analytical modeling. Furthermore, their sheer size and mass (5 tonnes in this case) make physical trial-and-error experiments extremely dangerous and expensive.

While Model Predictive Control (MPC) works well for steady-state tasks, it often struggles with "highly dynamic" behaviors involving frequent contact changes (e.g., lifting wheels off the ground) and underactuated balancing. This paper aims to bridge the gap between simulation and reality to enable agile, high-performance autonomous control in the heavy equipment sector.

Figure 1. Acrobatic maneuvers trained with RL in simulation and deployed on a 5-tonne SSL. (A) Swing-up from four onto two wheels. (B) Driving over rough terrain on two wheels. (C) Stable balancing while moving the arm. (D) Spinning in place on the front wheels. (E) Soft landing after balancing on two wheels.

Methods

The authors employed a Reinforcement Learning (RL) framework based on Proximal Policy Optimization (PPO). The core of their approach is a robust sim-to-real pipeline consisting of three main pillars:

  1. Learned Actuator Model: Instead of relying on a simplified first-order model for hydraulics, they trained a neural network on 1.5 hours of real-world data to predict wheel speeds. This model captures the "messy" parts of hydraulics—hysteresis, delays, and nonlinearities.
  2. Modular Control Skills: Rather than training one giant policy, they developed four specialized controllers: Swing-Up, Stabilize and Balance, Balance and Track Velocities, and Soft Landing.
  3. Domain Randomization: To account for modeling inaccuracies, they randomized physical parameters (mass, center of mass, inertia, and arm position) during training in the RaiSim simulation engine.

The system was deployed on a modified John Deere 332G SSL equipped with retrofitted IMUs and a standard Linux laptop for real-time inference.

Hardware setup including the wheelie bar (1), bucket wheels (2), and onboard sensors.

Datasets

The study did not use a pre-existing public dataset. Instead, the authors generated a custom dataset for system identification:

  • Actuator Training Data: 1.5 hours of real-world wheel speed data collected using randomized commands (steps, sinusoids, ramps).
  • Validation Data: 20 minutes of data specifically collected during balancing maneuvers to refine the actuator model's accuracy in high-stakes configurations.

Key Insights and Ideas

  • The Actuator Model is the Bridge: The "aha moment" was realizing that standard rigid-body simulations fail because they don't account for hydraulic lag. By learning the actuator dynamics from data, the simulation became "real" enough for the RL policy to work on the first try in the field.
  • RL vs. MPC: While MPC is excellent for precision, RL proved more robust for "non-convex" problems like the swing-up maneuver. RL policies naturally handle the abrupt contact switches and momentum accumulation required to pop a 5-tonne machine onto two wheels.
  • Robustness Through Randomization: Even though the controller was only trained on flat ground, domain randomization allowed it to successfully navigate 75 cm-high bumps while balancing—a task that caused the baseline MPC to fail.

Key Results

The RL policies demonstrated several world-record-level behaviors on the 5-tonne SSL:

  • Dynamic Swing-Up: Successfully transitioned from four wheels to two by rocking the machine to accumulate momentum.
  • High-Speed Tracking: The RL policy tracked velocities up to 4 rad/s while balancing, whereas the MPC baseline failed beyond 3.5 rad/s.
  • Terrain Generalization: Successfully traversed rough terrain with 75 cm undulations while maintaining a two-wheel balance.
  • Soft Landing: Developed a "reverse-thrust" strategy (driving backward just before impact) to minimize landing forces and prevent rebounding.
  • Cross-Platform Success: The same methodology was successfully applied to a smaller 100 kg "Super Mega Bot" (SMB), proving the approach scales across different weight classes.
Comparison of RL and MPC performance during velocity tracking and balancing.

Limitations and Future Work

Despite the impressive "acrobatics," the authors noted a few areas for improvement:

  • Hand-Crafted Switching: The transitions between different skills (e.g., from swing-up to balancing) currently rely on simple, rule-based logic. Future work could use hierarchical RL to learn these transitions autonomously.
  • Lack of Perception: The current controllers are "blind"—they don't use LiDAR or cameras. Adding exteroceptive sensors would allow the machine to navigate even more complex, unstructured environments.
  • Single-Policy Distillation: The authors suggest that the individual skills could eventually be distilled into a single, comprehensive policy for smoother overall operation.

Summary updated

Abstract

Automation of heavy machinery offers the opportunity to improve productivity in mechanized industries such as construction, mining, and material handling. Yet, automation of such machines is challenged by their complex dynamics, in part because of hydraulic actuation that involves fluids, flexible elements, and nonlinear delays. A concomitant challenge is the size and weight of these machines, which can make experiments on the physical systems exceedingly dangerous. Here, we show that \ac{RL} in simulation, followed by transfer of the learned policies from simulation to reality, can successfully automate acrobatic maneuvers that push a heavy construction machine to the limits of its actuation and have only previously been accomplished by a small number of elite human operators in official world-record performances. We trained policies that perform autonomous acrobatic maneuvers on a John Deere 332G, an operational 5-tonne skid-steer loader. The policies can autonomously swing the machine up onto two wheels and, while balancing on two wheels, drive over rough terrain, trace out specified trajectories, spin around, and swing the bucket back and forth before landing it safely back onto four wheels. This work demonstrates that \ac{RL} and sim-to-real transfer enable dynamic autonomous control of heavy machinery, directly contributing to greater automation in heavy equipment sectors. Videos: https://sites.google.com/leggedrobotics.com/heavy-metal-acrobatics This is the author's version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science Robotics on 11 November 2026, DOI: https://doi.org/10.1126/scirobotics.aef0894
BibTeX
@misc{lorentz2026heavy,
  author = {Pierre-Jean Lorentz and Pascal Egli and Edo Jelavic and Cliff Li and Ramitha Sundar and Harikrishnan Suresh and Paul Montgomery and Lee Redden and Matthias Müller and Ozan Sener and Vladlen Koltun and Marco Hutter},
  title = {{Heavy Metal Acrobatics: Learning Dynamic Control for Skid-Steer Loaders}},
  year = {2026},
  eprint = {2611.18365},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18365},
}
Figure