Generalized Dynamics Generation towards Scannable Physical World Model

Yichen Li, Zhiyi Li, Brandon Feng, Dinghuai Zhang, Antonio TorralbaarXiv 2025 · DOI · arXiv

Abstract

Building realistic physical world models is essential for creating "digital twins" and generalist embodied agents. However, traditional physics simulation is often fragmented across different mathematical paradigms for rigid, articulated, and soft bodies. GDGen introduces a unified framework that integrates these dynamics into a single, geometry-agnostic system. By leveraging "directional stiffness" within a potential energy minimization framework, GDGen infers complex physical properties from simple motion observations, enabling realistic simulations across diverse 3D representations like meshes, point clouds, and Gaussian splats.

Addressing the Fragmentation of Physics Simulation

In the quest to build robots that can navigate the real world, we need models that can simulate a bouncing ball, a swinging door, and a squishy toy simultaneously. Traditionally, this requires three distinct mathematical paradigms: rigid body, articulated body, and soft body dynamics. Furthermore, these simulations are often tied to specific geometric representations, making the conversion of real-world scans (like point clouds or Gaussian splats) into simulatable formats a labor-intensive manual process.

GDGen unifies various physical behaviors and geometric representations under a single framework, outperforming specialized algorithms in scope.

GDGen addresses this by proposing a "holistic" entity—a single framework capable of handling any geometry and any type of physical behavior by learning directly from observed motion. As shown in the comparison above, GDGen is the first to support geometry-agnostic simulation across soft bodies, articulated motion, and rigid discontinuums.

Methodology: The Potential Energy Principle

The core philosophy of GDGen is a fundamental governing principle from physics: stable physical systems should exist in low potential energy states. To transform this principle into a generative model, the authors introduced several key innovations:

Anisotropic Neohookean Elasticity

Standard elasticity models assume material stiffness is isotropic (the same in all directions). GDGen introduces directional stiffness (anisotropic Young’s modulus). This is the "secret sauce" for unification: by making a material extremely stiff in some directions but loose in others, the model can represent mechanical hinges or rigid bones within a soft body framework without needing explicit kinematic trees.

Neural Deformation Eigenmodes

To remain geometry-agnostic, GDGen utilizes a neural field (an MLP) to represent deformation. Instead of moving individual vertices, it predicts "eigenmode weights"—a sparse set of transformation handles. This allows the model to work interchangeably on meshes, point clouds, or Gaussian splats.

The GDGen pipeline optimizes deformation and material networks to produce simulatable assets from observed trajectories.

Two-Stage Optimization Pipeline

The system first learns to reconstruct the observed motion (the "what") via a Deformation Net and then optimizes the underlying physical parameters (the "why") via a Material Net by minimizing the proposed total strain energy. This process ensures that the learned dynamics are physically grounded rather than just visually plausible.

Key Insights and Results

  • Unification via Anisotropy: Articulation is treated as a special case of extreme directional stiffness. The model learns that energy is minimized only when an object rotates around a specific axis, effectively discovering joints without prior instruction.
  • Geometry Agnosticism: By decoupling physics from representation, the geometry "rides along" the deformation field, solving the heterogeneity problem in traditional simulators.
  • Superior Performance: GDGen consistently achieves lower reconstruction errors (Chamfer distance) compared to baselines. For instance, on the "Duck" dataset, GDGen's error was 0.013, significantly lower than specialized methods like PhysGaussian.
  • Predictive Power: When trained on limited initial frames, GDGen accurately predicts future motion, avoiding the "stickiness" or unrealistic artifacts seen in other methods.

Conclusion and Future Directions

GDGen represents a significant step toward scannable physical world models. While it currently faces challenges with collision detection and extremely thin geometries like cloth, its ability to infer underlying physics from motion provides a robust foundation for future research. The authors aim to extend this framework to more complex materials and integrate it into real-time robotic control loops.

Summary updated

Abstract

Digital twin worlds with realistic interactive dynamics presents a new opportunity to develop generalist embodied agents in scannable environments with complex physical behaviors. To this end, we present GDGen (Generalized Representation for Generalized Dynamics Generation), a framework that takes a potential energy perspective to seamlessly integrate rigid body, articulated body, and soft body dynamics into a unified, geometry-agnostic system. GDGen operates from the governing principle that the potential energy for any stable physical system should be low. This fresh perspective allows us to treat the world as one holistic entity and infer underlying physical properties from simple motion observations. We extend classic elastodynamics by introducing directional stiffness to capture a broad spectrum of physical behaviors, covering soft elastic, articulated, and rigid body systems. We propose a specialized network to model the extended material property and employ a neural field to represent deformation in a geometry-agnostic manner. Extensive experiments demonstrate that GDGen robustly unifies diverse simulation paradigms, offering a versatile foundation for creating interactive virtual environments and training robotic agents in complex, dynamically rich scenarios.
BibTeX
@misc{li2025generalized,
  author = {Yichen Li and Zhiyi Li and Brandon Feng and Dinghuai Zhang and Antonio Torralba},
  title = {{Generalized Dynamics Generation towards Scannable Physical World Model}},
  year = {2025},
  eprint = {2510.15041},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15041},
}
Figure