Preconditioned Deformation Grids

Julian Kaltheuner, Alexander Oebel, Hannah Dröge, Patrick Stotko, Reinhard Klein

Introduction

Reconstructing a deforming 3D object from a sequence of unstructured point clouds—such as those captured by LiDAR or depth sensors—remains a fundamental challenge in computer graphics. The primary difficulty stems from the lack of temporal correspondences; raw point clouds do not inherently indicate how a specific point in one frame moves to its new position in the next.

Reconstruction of various deforming objects from unstructured point cloud sequences.

Existing methodologies typically fall into two categories: template-based (or learning-based) approaches and optimization-based approaches. Template-based methods often rely on pre-trained models (e.g., SMPL for humans), which limits their ability to generalize to novel objects or unpredictable motions. Conversely, optimization-based methods frequently struggle with the balance between data fitting and smoothness, often resulting in over-smoothed geometry that sacrifices fine details. Preconditioned Deformation Grids addresses these issues by introducing a training-free, category-agnostic framework that estimates temporally coherent motion without requiring explicit point correspondences.

Methodology

The core of this framework is an optimization process that represents motion through a Multi-Resolution Transformation Grid. This approach allows the system to capture both global movements and localized surface details simultaneously.

Multi-Resolution Voxel Grids

Instead of relying on a single grid resolution, the method utilizes a hierarchy of up to 10 levels. Coarser levels are responsible for capturing broad, global deformations, while finer levels handle high-frequency surface details. The final transformation at any given point is derived from an average across these levels, providing a redundant yet highly stable representation of the deformation field.

Sobolev Preconditioning

A key technical innovation is the use of Sobolev preconditioning. While traditional methods add smoothness terms to the loss function (regularization), this framework incorporates smoothness directly into the optimization update step. By employing a Laplacian-based preconditioner, gradient information is "diffused" across neighboring grid cells. This naturally encourages smooth solutions during optimization, bypassing the typical trade-off between accuracy and jittery artifacts.

Robust Loss Functions and Adaptive Weighting

The system optimizes for a robust Chamfer loss to ensure accurate data fitting and a weak isometry loss on mesh edges to maintain physical plausibility (preventing unnatural stretching). To mitigate error propagation—a common issue in sequential tracking—the authors implement an Adaptive Confidence Weighting scheme. This down-weights contributions from later frames until the deformations in earlier frames have reached a sufficient level of accuracy.

Key Results and Insights

The framework was evaluated on major animation datasets including AMA (human motion), DFAUST (human body benchmarks), and DT4D (animal motions). Key findings include:

  • Superior Accuracy: The method consistently outperformed state-of-the-art baselines like CaDeX and DynoSurf. On the DT4D dataset, it reduced Chamfer Distance (CD) errors by nearly 10x compared to some existing methods.
  • Temporal Coherence: Reconstructed surfaces maintained high normal consistency and F-scores, demonstrating that the surfaces stay smooth and aligned with ground truth over long sequences.
  • Efficiency: Despite being optimization-based, the system is highly efficient, processing sequences in approximately 7 minutes on a consumer-grade GPU (RTX 4090) while utilizing less than 2GB of VRAM.
  • Robustness: Unlike many learning-based methods whose errors accumulate rapidly over time, this approach remained stable even across long-form animations (e.g., 80+ frames).

Conclusion and Future Work

Preconditioned Deformation Grids demonstrate that high-fidelity, correspondence-free reconstruction is viable through spatially coherent deformation fields. While the method is sensitive to the quality of the initial keyframe and can occasionally suffer from sequential error propagation, it represents a significant step forward in category-agnostic 3D reconstruction. Future research may explore adapting the grid structure for complex fluid dynamics or integrating the framework with 3D Gaussian Splatting for accelerated rendering.

Summary updated

BibTeX
@article{kaltheuner2025preconditioned,
  author = {Julian Kaltheuner and Alexander Oebel and Hannah Dröge and Patrick Stotko and Reinhard Klein},
  title = {{Preconditioned Deformation Grids}},
  journal = {Computer Graphics Forum},
  year = {2025},
  doi = {10.1111/cgf.70269},
}
Figure