PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation

Kai Lu, Chenyang Ma, Ioannis HavoutisarXiv 2026 · arXiv

TL;DR

PGX (Physically Grounded Execution) is a modular framework that enables humanoid robots to master long-horizon tasks by combining modern Vision-Language-Action (VLA) models with classical, training-free control layers. By offloading task timing and spatial precision to a state machine and a coordinate-locking system, PGX turns "near-failing" end-to-end policies into reliable systems capable of complex sequences like fetching and brewing coffee.

What Problem Does This Paper Address?

Humanoid "loco-manipulation"—tasks requiring both walking and hand-eye coordination—is incredibly difficult for current AI models. While Vision-Language-Action (VLA) models are great at individual skills (like "grasp the mug"), they struggle with long-horizon sequences.

The authors identify three main "coupling" problems in monolithic, end-to-end models:

  1. Temporal Structure: Models often lose track of task progress, leading to "stage confusion" where the robot repeats a step or skips ahead prematurely.
  2. Precision and Drift: Bipedal robots are inherently unstable. Small movements in the base (pelvis) accumulate, making it nearly impossible for a model to maintain the millimeter-level precision needed to press a small button or place an object accurately.
  3. Data Imbalance: In long demonstrations, walking takes up most of the time. This biases models toward "conservative" or static behaviors, making the actual manipulation parts of the task harder to learn.

Methods

Instead of trying to force a single neural network to handle everything, PGX decomposes the problem into three distinct layers:

  1. Timing (Predicate-Gated State Machine): This layer manages when to act. It uses physical "predicates" (sensor-based checks like "is the hand closed and the object lifted?") to gate transitions between stages. It handles navigation and waiting automatically, so the VLA doesn't have to.
  2. Skills (VLA): The VLA model is only active during manipulation phases. Because it starts from a consistent, "prepared" state provided by the other layers, it can focus entirely on the dynamics of the specific skill.
  3. Precision (World-Coordinate Locking - WCL): This layer manages where to act. It uses three sub-components to fight drift:
  4. Object-relative stance: Re-centers the robot's goal position based on the object's current location before every attempt.
  5. Base station keeping: Actively holds the robot's base still during manipulation.
  6. World-anchored contact targets: Locks the target (e.g., a button) in the world frame so the arm stays on target even if the robot's body sways.
Fig. 1: Humanoid long-horizon loco-manipulation. This work studies humanoid loco-manipulation tasks: with the same VLA model, the flat policy fails before the machine, whereas our method PGX completes the task by keeping timing and world-frame precision outside the network, and the VLA executes only the manipulation stages.

Datasets

The researchers evaluated PGX using:

  • Simulation (Isaac Lab Arena): Three tasks—Box Transport (bimanual), Mug Placement (single-hand), and Coffee Fetching (long-horizon with device interaction). They used 100 MimicGen demonstrations for the box task and 30-50 teleoperated demonstrations for the others.
  • Real-World: A coffee fetching task using a Unitree G1 humanoid robot, trained on 44 human demonstrations.
  • Model Backbones: They tested the framework with two state-of-the-art VLA models: $\pi_{0.5}$ and GR00T N1.6.

Key Insights and Ideas

  • The "Funnel" Effect: By using navigation and coordinate locking to re-establish the starting conditions for every skill, PGX creates a "funnel" that brings the robot back into the specific state where the VLA model was trained to succeed.
  • Physical > Semantic: Using hard physical predicates (like distance and contact) for stage transitions is far more robust than relying on a neural network to infer task progress from images alone.
  • Decoupling Responsibility: The core insight is that VLAs are best at "how to act," while classical control is better at "when and where to act." Separating these allows the VLA to be more transferable and less prone to the "drifting" errors of a moving humanoid base.

The Timing layer uses a state machine to manage transitions and retries. The Precision layer (WCL) maintains spatial consistency despite base movement.

Key Results

The results show that PGX dramatically improves the reliability of existing models without changing their weights:

  • Coffee Fetching: On the most complex task, PGX raised the success rate of a fine-tuned $\pi_{0.5}$ model from 0% to 70%. For the GR00T backbone, success jumped from 35% to 80%.
  • Robustness to Noise: In "Random Initialization" tests (where the robot starts in a position it never saw during training), PGX maintained high success rates while "flat" end-to-end policies often failed completely.
  • Real-World Success: The framework successfully transferred to the Unitree G1 robot, proving that the simulation-to-real gap can be bridged by structured execution.

Limitations and Future Work

  • Execution Speed: Because the system waits for physical predicates to be satisfied and performs "station keeping" corrections, it is slower than a purely end-to-end policy.
  • Predicate Engineering: Currently, the physical predicates (like "is the mug grasped?") must be manually defined for each task. Future work could involve learning these predicates or using VLMs to generate them.
  • Skill Variety: The authors plan to extend the framework to handle a much wider variety of more complex, dynamic skills.

Summary updated

Abstract

Robotic vision-language-action (VLA) models can execute diverse individual manipulation skills, yet they still struggle with long-horizon loco-manipulation tasks, where temporal structure, skills, and precision are tightly coupled within a monolithic model. We propose Physically Grounded Execution (PGX), a framework that offloads temporal and precision control to two classical training-free components, allowing the VLA to focus on stage-specific skill dynamics. PGX decomposes long-horizon loco-manipulation into three layers: (i) a predicate-based state machine governs task progression, using physical predicates to gate phase transitions, trigger recovery, and coordinate navigation and waiting. Shared predicate semantics connect offline demonstration segmentation, online execution, and stage-wise evaluation; (ii) VLA skills operate within active manipulation phases, with flat or stage-trained models concentrating on task-relevant actions; and (iii) world-coordinate locking (WCL) maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. We demonstrate the effectiveness of PGX with extensive experiments conducted on three simulated tasks and one real-world task using a Unitree G1 humanoid robot for long-horizon loco-manipulation, showing improvements in stage progression and task success rates. Project page is available at: https://kl-research.github.io/pgx
BibTeX
@misc{lu2026pgx,
  author = {Kai Lu and Chenyang Ma and Ioannis Havoutis},
  title = {{PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation}},
  year = {2026},
  eprint = {2611.26596},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26596},
}
Figure