Abstract
Building realistic physical world models is essential for creating "digital twins" and generalist embodied agents. However, traditional physics simulation is often fragmented across different mathematical paradigms for rigid, articulated, and soft bodies. GDGen introduces a unified framework that integrates these dynamics into a single, geometry-agnostic system. By leveraging "directional stiffness" within a potential energy minimization framework, GDGen infers complex physical properties from simple motion observations, enabling realistic simulations across diverse 3D representations like meshes, point clouds, and Gaussian splats.
Addressing the Fragmentation of Physics Simulation
In the quest to build robots that can navigate the real world, we need models that can simulate a bouncing ball, a swinging door, and a squishy toy simultaneously. Traditionally, this requires three distinct mathematical paradigms: rigid body, articulated body, and soft body dynamics. Furthermore, these simulations are often tied to specific geometric representations, making the conversion of real-world scans (like point clouds or Gaussian splats) into simulatable formats a labor-intensive manual process.
GDGen addresses this by proposing a "holistic" entity—a single framework capable of handling any geometry and any type of physical behavior by learning directly from observed motion. As shown in the comparison above, GDGen is the first to support geometry-agnostic simulation across soft bodies, articulated motion, and rigid discontinuums.
Methodology: The Potential Energy Principle
The core philosophy of GDGen is a fundamental governing principle from physics: stable physical systems should exist in low potential energy states. To transform this principle into a generative model, the authors introduced several key innovations:
Anisotropic Neohookean Elasticity
Standard elasticity models assume material stiffness is isotropic (the same in all directions). GDGen introduces directional stiffness (anisotropic Young’s modulus). This is the "secret sauce" for unification: by making a material extremely stiff in some directions but loose in others, the model can represent mechanical hinges or rigid bones within a soft body framework without needing explicit kinematic trees.
Neural Deformation Eigenmodes
To remain geometry-agnostic, GDGen utilizes a neural field (an MLP) to represent deformation. Instead of moving individual vertices, it predicts "eigenmode weights"—a sparse set of transformation handles. This allows the model to work interchangeably on meshes, point clouds, or Gaussian splats.
Two-Stage Optimization Pipeline
The system first learns to reconstruct the observed motion (the "what") via a Deformation Net and then optimizes the underlying physical parameters (the "why") via a Material Net by minimizing the proposed total strain energy. This process ensures that the learned dynamics are physically grounded rather than just visually plausible.
Key Insights and Results
- Unification via Anisotropy: Articulation is treated as a special case of extreme directional stiffness. The model learns that energy is minimized only when an object rotates around a specific axis, effectively discovering joints without prior instruction.
- Geometry Agnosticism: By decoupling physics from representation, the geometry "rides along" the deformation field, solving the heterogeneity problem in traditional simulators.
- Superior Performance: GDGen consistently achieves lower reconstruction errors (Chamfer distance) compared to baselines. For instance, on the "Duck" dataset, GDGen's error was 0.013, significantly lower than specialized methods like PhysGaussian.
- Predictive Power: When trained on limited initial frames, GDGen accurately predicts future motion, avoiding the "stickiness" or unrealistic artifacts seen in other methods.
Conclusion and Future Directions
GDGen represents a significant step toward scannable physical world models. While it currently faces challenges with collision detection and extremely thin geometries like cloth, its ability to infer underlying physics from motion provides a robust foundation for future research. The authors aim to extend this framework to more complex materials and integrate it into real-time robotic control loops.
Summary updated