Transforming Surface Splats into Simulation Ready Volumetric Particles

This direction is merged into another direction: surface-to-volume representation gap. Its last version stays readable and citable here.

Version 1 of 119 papers28 conceptsfrozen 2026-10-04

Computer vision has become remarkably adept at reconstructing the "skin" of the world. Techniques like 3D Gaussian Splatting (3DGS) can turn a handful of photos into a photorealistic digital twin in minutes. However, these twins are often hollow shells—visual facades that lack the internal mass and structural integrity required for physical interaction. If you try to "push" a standard 3DGS model in a physics engine, it doesn't deform like a solid object; it collapses like an empty paper bag or shatters into a cloud of disconnected points.

A growing body of research is bridging this gap by converting surface-based visual primitives into volumetric particles capable of supporting continuum mechanics. By "filling" the interior of reconstructed surfaces, researchers are enabling static captures to bounce, flow, fracture, and respond to forces with unprecedented realism.

The Problem of the Hollow Shell

The fundamental issue is a mismatch between rendering and simulation. Rendering primitives are optimized to represent what we see: the surface. In contrast, physics engines—specifically the Material Point Method (MPM) frequently used in this field—require a representation of mass and volume to calculate internal stress and strain.

As Tianyi Xie and colleagues point out in PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics, surface-only reconstructions lack the internal structure needed to maintain volume. Without an interior, there is no "internal pressure" to resist external forces. This leads to structural collapse or unrealistic "sticky" artifacts where the front and back surfaces of an object merge during deformation.

Volumetric Filling: From Skin to Body

To solve this, researchers have developed several strategies to algorithmically "stuff" the interior of a surface reconstruction with particles. This process, often called volumetric densification or internal padding, transforms a visual asset into a physical one.

  1. Ray-Casting and Occupancy: In PhysGaussian, the authors use a ray-casting procedure to identify the "inside" of the 3D opacity field generated by Gaussian Splatting, populating it with internal particles that act as the object's mass.
  2. Voxelization and Octrees: To handle complex environments, Yuanhang Lei in DiffWind employs an octree-based voxel filling algorithm. This ensures that even thin or complex geometries have a stable internal density.
  3. Signed Distance Functions (SDF): For destructive simulations like Fracture-GS, Xiaogang Wang uses an SDF to sample particles throughout the volume. This is critical for fracture: when an object breaks, the "new" surfaces exposed by the crack must already have internal particles ready to be rendered, or the object will look like a broken hollow eggshell.

Bridging Vision and Physics via Differentiable Simulation

Once an object has volume, it can be linked to a differentiable physics simulator. This connection allows for System Identification: the ability to look at a video of a falling object and "reverse-engineer" its physical properties, such as its Young's modulus (stiffness) or viscosity.

Junhao Cai and the team behind GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation demonstrate that by treating Gaussians as particles in a Material Point Method (MPM) continuum, one can optimize for physical parameters directly from visual observations. Similarly, Yang Zhao in MASIV shows that this volumetric approach allows for material-agnostic identification, creating "digital twins" that can predict how an unknown object will react to entirely new forces.

Why It Works: Structural Stabilization

The intuition behind these methods traces back to earlier work in shape-from-template. Shaifali Parashar argued in As-Rigid-as-Possible Volumetric Shape-from-Template that the internal volume acts as a structural stabilizer. By enforcing that the interior must deform "as-rigidly-as-possible," the system can infer the movement of hidden or occluded parts of an object just by watching the visible surface.

In modern contexts like robotic surgery, this is vital. Zhenya Yang uses volumetric padding in SimEndoGS to ensure that soft tissues don't "crack" or collapse when poked by surgical tools. The internal particles provide the necessary resistance to maintain topological integrity during manipulation.

Beyond Reality: Generative Dynamics

This volumetric shift isn't just for reconstruction; it's also for creation. By embedding semantic features into these particles, as seen in Feature Splatting by Ri-Zhao Qiu, users can select a part of a static scene with a text prompt (e.g., "the bread") and assign it physical properties. Because the system automatically generates the internal volume, the bread can then be realistically squashed or shattered, even though the original input was just a few static photos.

By treating the interior as a first-class citizen in the reconstruction pipeline, this research direction is turning the "hollow" digital world into a solid, interactive reality.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_transforming_surface_splats_into_simulat_a2c7b827,
  title = {Transforming Surface Splats into Simulation Ready Volumetric Particles},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/transforming-surface-splats-into-simulation-ready-volumetric-particles/rdr_a2c7b8275eac4fa0ae297df71bfbe642},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_transforming_surface_splats_into_simulat_a2c7b827_v1,
  title = {Transforming Surface Splats into Simulation Ready Volumetric Particles},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/transforming-surface-splats-into-simulation-ready-volumetric-particles/rdr_a2c7b8275eac4fa0ae297df71bfbe642/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}