TL;DR
ReACT is a framework that enables humanoid robots to maintain precise velocity tracking while carrying heavy, one-sided loads. By combining symmetry-aware feature modulation with a training objective that enforces consistent motion across different loading states, it eliminates the "drift" that typically causes navigation failures in asymmetric transport tasks.
What Problem Does This Paper Address?
Imagine a humanoid robot carrying a heavy suitcase in its right hand. As it walks, the asymmetric weight pulls its torso to the side, creating a persistent lateral and yaw-rate bias. Even if the robot is "stable," it will slowly drift off course. For a navigation system, this drift is a nightmare: a command to "walk straight" results in a curved path, leading to collisions with obstacles or missing the goal entirely.
Existing methods like domain randomization or analytical disturbance feedback help, but often leave residual drift under large loads. The challenge lies in adapting the robot's joint responses to an estimated disturbance while ensuring that the resulting motion remains consistent regardless of whether the robot is carrying 0kg, 2kg, or 4kg in either hand.
Methods
The authors introduce ReACT, which consists of two primary components integrated into an actor-critic reinforcement learning framework:
-
Reflection-Consistent Disturbance Modulation: Humanoids are sagittally symmetric. ReACT exploits this by using a learned wrench estimator to predict external forces and moments. These estimates are then used to modulate internal policy features. Crucially, the architecture (shown in Figure 1) decomposes features into symmetric and asymmetric components. The estimated wrench scales and couples these features in a way that preserves "action correspondence"—meaning if you mirror the robot's observation and the load, the resulting joint actions are also perfectly mirrored.
Overview of the ReACT architecture, showing the wrench estimation and modulation blocks. -
Cross-Load Tracking Consistency Training: Symmetry alone doesn't stop a robot from drifting; it just ensures it drifts the same way on both sides. To fix the drift itself, ReACT trains the policy in "matched triplets": one unloaded branch, one left-loaded, and one right-loaded. A consistency reward ($r^{con}$) penalizes any deviation in the planar displacement of the robot's root between these three branches for the same velocity command. This forces the policy to learn compensatory behaviors that "cancel out" the load's effect on the global trajectory.
Datasets
The system was developed and tested using:
- Simulation: Trained in Isaac Gym using 4096 parallel environments. The robot model used was the Unitree G1 humanoid.
- Real-World Evaluation: A physical Unitree G1 was used for mapless navigation trials. The robot carried a 1.5kg unilateral payload (shown in Figure 5) through various obstacle courses.
Key Insights and Ideas
- Symmetry as a Prior: By baking sagittal reflection into the network architecture, the robot doesn't have to "re-learn" how to carry a load in its left hand if it already knows how to carry it in its right.
- Modulation > Concatenation: Simply "tacking on" the wrench estimate to the input vector (concatenation) is less effective than using it to actively scale and shift internal feature representations (modulation).
- Consistency is the Key to Navigation: The "aha moment" here is that for navigation, consistency across loads is just as important as accuracy. If a navigation policy knows the locomotion controller will always follow the command regardless of the payload, it doesn't need to be re-tuned for every different object the robot picks up.
Key Results
ReACT significantly outperformed the baseline (FALCON) in both simulation and real-world tests:
-
Bias Reduction: In straight-walking tests, ReACT reduced lateral and yaw-rate tracking errors by approximately 32% and 82%, respectively. As shown in Figure 2, ReACT maintains near-zero directional bias across a wide range of payload masses, whereas the baseline drifts significantly as the weight increases.
Comparison of directional bias vs payload mass. ReACT (red) stays near zero, while FALCON (blue) drifts. -
Trajectory Following: Figure 3 highlights the qualitative difference: ReACT follows the commanded dashed lines almost perfectly, while the baseline veers off-course, especially under 4kg loads.
Planar trajectories showing ReACT's superior path following compared to FALCON. -
Real-World Success: In 40 navigation trials with a 1.5kg hand load, ReACT achieved a 100% success rate (40/40), reaching the goal without collisions. In contrast, the baseline failed every single trial (0/40) due to collisions caused by persistent tracking drift.
Limitations and Future Work
- Static vs. Dynamic Loads: The current study focuses on persistent loads (carrying an object throughout a trial). The authors note that future work should address dynamic changes, such as the sudden shift in wrench when a robot picks up or drops an object mid-stride.
- Payload Variety: While 4kg is significant for a robot the size of the G1, testing a broader range of object shapes and centers of mass would further validate the robustness of the modulation blocks.
- Mapless Navigation: The navigation policy used was fixed. Integrating the locomotion consistency directly into the navigation planning could yield even more agile behaviors.
Summary updated