Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization

Takuhiro KanekoCVPR 2024 · DOI · arXiv

Introduction

System identification (SysID) is the process of estimating the physical parameters of an object—such as elasticity, viscosity, or density—from visual observations. While traditional SysID methods often rely on pre-existing 3D models, "geometry-agnostic" approaches aim to simultaneously reconstruct an object's shape and its physical properties. A prominent recent framework, PAC-NeRF, combines Neural Radiance Fields (NeRF) for geometric representation with the Material Point Method (MPM) for physics simulation. However, PAC-NeRF faces significant challenges in sparse-view scenarios where initial geometric estimates are often flawed, leading to a cascade of errors in physical parameter estimation.

The Geometry-Physics Dependency Problem

PAC-NeRF typically follows a two-step process: it first learns the object's geometry from the initial frame and then freezes this geometry to estimate physical parameters from subsequent motion. In sparse-view settings (e.g., only three camera angles), the initial NeRF reconstruction is frequently inaccurate. Because the geometry is fixed, the physics engine attempts to simulate motion on a non-existent or distorted shape, causing the system identification to fail. This creates a "chicken-and-egg" problem where accurate physics requires accurate geometry, but accurate geometry in sparse views could benefit from physical constraints.

Methodology: Lagrangian Particle Optimization (LPO)

To address this, the authors introduce Lagrangian Particle Optimization (LPO). Unlike standard NeRFs that optimize a fixed spatial (Eulerian) grid, LPO allows for the refinement of geometry by optimizing the positions ($x^P$) and features (density $\sigma^P$ and color $c^P$) of the Lagrangian particles themselves throughout the entire video sequence.

The three-stage optimization pipeline: (1) Initial Eulerian grid optimization, (2) Physical property estimation, and (3) The proposed Lagrangian Particle Optimization (LPO) for geometric refinement.

The LPO framework operates through an iterative loop:

  1. Initial Geometry: A static voxel grid is optimized using the first frame.
  2. Physics Estimation: Physical properties are estimated using differentiable MPM (DiffMPM).
  3. Geometric Refinement: Particle positions and features are optimized in Lagrangian space, using the physics of motion as a regularizer to correct the initial shape.

By shifting optimization to the Lagrangian particles, the model can "push" and "pull" the geometry through time, ensuring that the reconstructed shape is consistent with both visual data and physical laws.

Experimental Results

The proposed method was evaluated on a diverse benchmark of nine scenes, including Newtonian and non-Newtonian fluids, elastic materials, plasticine, and granular media. The focus was on challenging sparse-view settings with only three training viewpoints.

Comparison of geometry and physical parameter estimation on a torus scene. LPO significantly improves both visual fidelity (PSNR) and the accuracy of Young’s modulus ($E$) and Poisson’s ratio ($\nu$).

As shown in the results for the "Torus" scene, LPO dramatically improves the Peak Signal-to-Noise Ratio (PSNR) from 23.99 to 30.07 compared to the PAC-NeRF baseline. Furthermore, the estimated physical parameters—Young’s modulus ($E$) and Poisson’s ratio ($\nu$)—are much closer to the ground truth. Across all material types, LPO demonstrated superior robustness, effectively correcting geometric errors that previously led to physical identification failure.

Conclusion

Lagrangian Particle Optimization represents a significant step forward for geometry-agnostic system identification. By treating physics as a geometric regularizer and optimizing in Lagrangian space, the method successfully breaks the dependency cycle between shape and physical parameters. While the iterative nature of LPO increases computational overhead, its ability to recover accurate physics from sparse visual data makes it a powerful tool for understanding the physical world from limited observations.

Summary updated

BibTeX
@inproceedings{kaneko2024improving,
  author = {Takuhiro Kaneko},
  title = {{Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.00523},
}
Figure