Toward Uncertainty-Aware, Physics-Grounded Human World Models

Yufei Zhang, Zijun CuiarXiv 2026 · arXiv

This paper presents a blueprint for a robust "human world model" designed for robotics. By coupling two existing state-of-the-art components—KNOWN for perception and PhysMoP for dynamics—the authors propose a system that not only predicts human movement with sub-centimeter accuracy but also quantifies exactly how much those predictions should be trusted.

TL;DR

The authors propose integrating uncertainty-aware 3D body reconstruction (KNOWN) with physics-grounded motion prediction (PhysMoP) to create a reliable human world model for robots. This framework separates observation noise from model ignorance, allowing robots to make safer, physics-informed decisions in human-centric environments.

What Problem Does This Paper Address?

For a robot to safely navigate or collaborate with humans, it needs a "world model" that answers two questions: Where is the person going? and How sure am I?

Current approaches have significant gaps. Most human mesh estimators provide only point estimates, failing to distinguish between aleatoric uncertainty (noise from things like motion blur or occlusion) and epistemic uncertainty (the model's ignorance of unfamiliar poses). Meanwhile, motion predictors are often trained on perfect "ground truth" data and struggle when fed the noisy, uncertain estimates produced by real-world sensors. This paper addresses the challenge of building a unified pipeline that carries perception uncertainty through to a physics-based prediction engine.

Methods

The proposed architecture joins two specialized modules:

  1. KNOWN (Perception): A two-stage probabilistic network that reconstructs the 3D human body in real-time (11.6 ms). It uses the law of total variance to split uncertainty. Aleatoric uncertainty identifies unreliable observations (e.g., a hidden limb), while epistemic uncertainty flags inputs that look nothing like the training data.
  2. PhysMoP (Dynamics): Instead of a pure black-box neural network, PhysMoP embeds the Euler-Lagrange equations of the human body into its architecture. It predicts physical terms like inertia ($M$), Coriolis forces ($C$), and generalized forces ($G$) to simulate future motion.
The proposed coupling: KNOWN's per-frame belief serves as the input history for PhysMoP's physics-grounded prediction.

The authors define three specific mechanisms for this coupling:

  • Initial-condition sampling: Running multiple physics simulations based on the distribution of possible current states.
  • Covariance-weighted smoothing: Using aleatoric uncertainty to "discount" noisy frames before calculating velocity and acceleration.
  • Epistemic gating: Falling back to a stable data-driven model when the perception system encounters a pose it doesn't recognize.

Datasets

The components were validated using standard benchmarks in the field:

  • Human3.6M: A large-scale indoor dataset for 3D human pose estimation.
  • 3DPW (3D Poses in the Wild): Used as the primary benchmark for "in-the-wild" video evaluation.
  • AMASS: A massive database of motion capture data used to train the PhysMoP dynamics model.

Key Insights and Ideas

  • Physics as a Regularizer: The authors argue that physical structure (like joint limits and inertia) is what keeps a prediction plausible when sensor data is missing or noisy.
  • The Value of Disagreement: By separating types of uncertainty, the system can make nuanced decisions. For example, high aleatoric uncertainty might suggest the robot should move slower, while high epistemic uncertainty might trigger a complete halt or a request for more data.
  • Real-Time Feasibility: Both components are highly efficient. KNOWN's 11.6 ms latency is significantly faster than competing probabilistic models like ProHMR (16.4 ms) or multi-hypothesis models (which can be much larger).
Detailed view of the PhysMoP architecture, showing how inverse dynamics and Euler-Lagrange equations are used to solve for forward dynamics.

Key Results

While this specific abstract focuses on the coupling rather than new experiments, it highlights the impressive performance of the underlying modules:

  • PhysMoP reduced the 80 ms prediction error on Human3.6M from 9.6 mm (standard baselines) to just 2.1 mm. On 3DPW, the error dropped from 12.1 mm to 0.7 mm.
  • KNOWN's uncertainty-guided refinement reduced error on difficult "minority" cases (e.g., extreme poses) from 81.3 mm to 70.3 mm.
  • The physics branch of PhysMoP is the most precise for short-term horizons (up to 320 ms), which is the critical window for a robot's control loop.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Root Translation: KNOWN uses a weak-perspective camera and cannot estimate absolute distance (metric depth) on its own; it currently requires an external depth sensor.
  • Action Conditioning: The model currently forecasts human movement but doesn't yet account for how the robot's own actions might influence the human (e.g., a person stepping back as a robot approaches).
  • Integration: The fusion weights in PhysMoP are not yet dynamically informed by KNOWN's real-time uncertainty, which is a primary target for future implementation.

Summary updated

Abstract

A robot that acts around people needs a world model of the human: a belief over the person's body state and a dynamics model that predicts where that state goes next. This abstract argues that two published components form exactly these two halves, and specifies how to couple them. KNOWN [1] reconstructs the 3D human body while estimating both aleatoric and epistemic uncertainty in real time (11.6 ms per frame); its aleatoric term identifies unreliable observations, its epistemic term identifies inputs from low-density regions of the training data, and refinement driven by the epistemic term reduces the error on such inputs from 81.3 to 70.3 mm. PhysMoP [2] predicts future body configuration by solving the Euler-Lagrange equations of the body with learned dynamics terms, reducing the 80 ms prediction error from 9.6 to 2.1 mm on Human3.6M and from 12.1 to 0.7 mm on 3DPW. We define the interface between the two, propose three mechanisms by which KNOWN's uncertainty enters PhysMoP, and specify an evaluation protocol on in-the-wild video. No new experimental results are reported here.
BibTeX
@misc{zhang2026toward,
  author = {Yufei Zhang and Zijun Cui},
  title = {{Toward Uncertainty-Aware, Physics-Grounded Human World Models}},
  year = {2026},
  eprint = {2611.03428},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03428},
}
Figure