Volumetric Physics-Integrated Neural Representations

This direction is split into other directions: surface-to-volume representation gap. Its last version stays readable and citable here.

Version 1 of 180 papers86 conceptsfrozen 2026-10-04

Modern 3D reconstruction techniques like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have achieved breathtaking visual fidelity. However, to a computer, these models are often "hollow shells"—thin envelopes of color and density with no internal substance. If you try to slice a virtual watermelon reconstructed this way, or drop a neural vase, the illusion breaks: the interior is empty, and the object lacks the mass or structural integrity to react naturally.

A growing body of research is bridging this gap by integrating continuum mechanics and volumetric physical properties directly into neural representations. The goal is to move beyond "looking right" to "behaving right," creating simulation-ready assets that possess mass, friction, and elasticity.

The Problem of Surface-Centric Representations

Traditional 3D reconstruction focuses on the visible exterior. As noted by Fangyu Wu and Yuhao Chen in FruitNinja, most digital assets are "empty envelopes." This creates three primary failures in virtual environments:

  1. Structural Collapse: Without internal volume, objects cannot support their own weight during deformation.
  2. Inaccurate Dynamics: Simulators cannot calculate how an object should bounce or slide without knowing its mass distribution.
  3. Interaction Gaps: Tasks like slicing, fracturing, or crushing reveal the "hollow" nature of the model, as there is no data for the newly exposed surfaces.

To solve this, researchers are reimagining rendering primitives as physical particles.

Primitives as Physical Particles

The most significant shift in this field is the unification of rendering and simulation. In PhysGaussian, Tianyi Xie and colleagues treat 3D Gaussian kernels not just as "blobs" of color, but as physical particles governed by the laws of physics.

By embedding continuum mechanics—specifically the Material Point Method (MPM)—directly into the Gaussian Splatting pipeline, the Gaussians themselves carry physical strain and stress. This allows a model to undergo complex deformations, like a squishy toy being squeezed or bread being shattered, while maintaining visual consistency. Similarly, Simon Le Cleac’h and Jiajun Wu introduced DANO (Differentiable Physics Simulation of Dynamics-Augmented Neural Objects), which treats neural density fields as probabilistic volumes that robots can interact with, estimating mass and friction directly from video.

Inferring the Invisible Interior

If a camera only sees the surface, how can we know what is inside? Researchers are using two primary strategies: learned priors and dynamic observation.

Learned Material Manifolds

Models like VoMP (Predicting Volumetric Mechanical Property Fields), developed by Rishit Dagli and David Levin I.W., use deep learning to predict internal properties like Young’s modulus (stiffness) and density from visual features. By mapping an object's appearance to a "material manifold," these models can estimate the heterogeneous internal composition of an object in seconds, transforming a static mesh or NeRF into a solid, simulation-ready volume.

Structure from Motion and Collision

Another approach is to "see" the interior by watching how the exterior moves. In SfC-NeRF (Structure from Collision), researchers estimate internal density by observing how an object deforms when it hits the ground. Similarly, TopoGaussian (Inferring Internal Topology Structures from Visual Clues) by Xiaoyu Xiong and Tao Du uses a differentiable simulator to "reverse-engineer" hidden internal structures—like the hollow cavity inside a ball—based on its observed physical behavior in video.

Enabling Realistic Interaction

The ultimate goal of this research is to enable "open-world" interaction where objects can be manipulated in ways not seen in the training data.

  • Fracture and Slicing: Fracture-GS by Xiaogang Wang uses 3D Gaussians to "heal" newly exposed surfaces during a collision, allowing objects to break apart realistically.
  • Fluid Dynamics: Vid2Fluid by Alan Zhao extracts physical properties like viscosity from single-view videos, allowing captured fluids to be re-simulated and edited.
  • Aerodynamics: Gaussian Swaying by Hongru Yan evolves Gaussians into "surface patches" that can interact with wind fields, enabling realistic swaying of trees or rippling of cloth.

By moving from hollow shells to volumetric, physics-informed representations, this research direction is turning 3D captures from static pictures into functional, interactive digital matter. As Hsiao-Yu Chen and Etienne Vouga demonstrate with Virtual Elastic Objects (VEOs), the future of 3D vision lies in capturing not just how the world looks, but how it feels and moves.

Papers in this version

Showing 60 of 80 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_volumetric_physics_integrated_neural_rep_515e97dd,
  title = {Volumetric Physics-Integrated Neural Representations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/volumetric-physics-integrated-neural-representations/rdr_515e97dd44344b45aaacac149d596037},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_volumetric_physics_integrated_neural_rep_515e97dd_v1,
  title = {Volumetric Physics-Integrated Neural Representations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/volumetric-physics-integrated-neural-representations/rdr_515e97dd44344b45aaacac149d596037/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}