Abstract
Drifting represents the absolute handling limits of a vehicle, where tires lose full traction and operate at high sideslip angles. Mastering these maneuvers is critical for autonomous vehicle safety, particularly for emergency obstacle avoidance and recovery from sudden friction changes. This paper presents a reinforcement learning (RL) framework that enables autonomous 1/10-scale vehicles to perform complex drifting maneuvers with zero-shot transfer from simulation to reality. By utilizing Individual Wheel Drive (IWD) and GPU-accelerated parallel simulation with heavy domain randomization, the proposed method reduces training times to just 11 minutes while effectively bridging the "sim-to-real" gap.
Introduction
In the realm of autonomous driving, operating at the handling limit—often characterized by drifting—is a significant technical challenge. Drifting involves maintaining control while the vehicle's tires are saturated, a state where traditional linear control models typically fail. While often associated with motorsports, the ability to control a vehicle in a high-sideslip state is a vital safety requirement for navigating extreme scenarios, such as avoiding a collision on a low-friction surface.
Two primary hurdles have historically hindered progress in this area: the difficulty of modeling non-linear tire dynamics (the "sim-to-real gap") and the extreme computational cost of training reinforcement learning agents for complex vehicle control. Furthermore, most existing research focuses on Rear-Wheel Drive (RWD) configurations. This work explores the potential of Individual Wheel Drive (IWD), where each wheel's torque is controlled independently, providing superior control authority to stabilize the vehicle during extreme maneuvers.
Methodology
GPU-Accelerated Simulation
To overcome the training efficiency bottleneck, the researchers developed a custom simulator inspired by NVIDIA Isaac Gym. Built in PyTorch, the simulator runs $10^6$ car instances in parallel on a single GPU. This massive parallelism allows the system to generate data at a speedup factor of $3 \times 10^5$ relative to real-time, enabling the agent to experience centuries of driving data in minutes.
Reinforcement Learning Framework
The framework utilizes the Proximal Policy Optimization (PPO) algorithm. To ensure the policy generalizes across different environments and tracks, the authors implemented a "coordinate-free" observation space. Instead of global coordinates, the policy receives relative errors—such as lateral deviation from a path, heading error, and local curvature—alongside vehicle states like yaw rate and sideslip angle.
Bridging the Sim-to-Real Gap
To ensure the policy could transfer to a physical vehicle without fine-tuning, the researchers employed systematic domain randomization across three dimensions:
- Trajectories: Procedural generation of paths with varying curvatures and direction reversals.
- Initial States: Randomizing the starting position and velocity to ensure the agent can recover from off-nominal conditions.
- Physics Parameters: Randomizing Pacejka tire model coefficients and adding auto-regressive noise to tire forces to account for unmodeled vibrations and surface inconsistencies.
Hardware Implementation
The physical platform, dubbed "Xcar," is a 1/10 scale RC car equipped with four independent motors and an onboard NVIDIA Jetson Orin Nano. This setup allows for precise torque vectoring, enabling the IWD strategies developed in simulation to be executed in the real world.
Experimental Results
Training Efficiency and Success
The RL policy converges in approximately 10.7 minutes on a single NVIDIA RTX 3080. Ablation studies highlighted that initial state randomization was the most critical factor for success; removing it resulted in a 29% drop in success rate, as the policy failed to learn recovery behaviors.
Zero-Shot Real-World Performance
The policy demonstrated successful zero-shot transfer to the physical Xcar. In real-world tests, the vehicle maintained a mean sideslip angle of approximately 47° while tracking complex paths. On an eight-shaped maneuver, the vehicle maintained a position Root Mean Square Error (RMSE) of only 0.22m.
The IWD Advantage
Comparative simulations between IWD and RWD configurations revealed that IWD acts as a "cheat code" for drifting. By spinning outside wheels faster than inside wheels, the IWD controller creates a powerful yaw moment. This allowed the IWD system to reach a stable drift state in 1.0s, compared to 1.5s for RWD, while maintaining higher stability at larger sideslip angles.
Discussion and Future Work
While the results demonstrate a robust framework for high-limit maneuvering, several limitations remain. The current model assumes a 2D plane and does not account for motor dynamics or suspension travel. In high-speed maneuvers, lateral load transfer significantly impacts tire grip; incorporating suspension dynamics could further refine the policy's accuracy. Future research will also focus on testing the system on more diverse surfaces, such as wet pavement or gravel, to prove universal robustness across varying friction levels.
Conclusion
By combining GPU-accelerated simulation with Individual Wheel Drive hardware, this research provides a path toward more capable and safer autonomous vehicles. The ability to train a high-performance drifting policy in under 11 minutes and deploy it directly to the real world represents a significant step forward in autonomous control at the handling limits.
Summary updated