Dynamic Probabilistic Volumetric Models

Ali Osman Ulusoy, Octavian Biris, Joseph L. MundyICCV 2013 · DOI

Abstract

This paper introduces a novel 4D volumetric representation that adaptively subdivides in both space and time to model dynamic 3D scenes efficiently. By combining a hybrid grid-octree with binary time trees, the framework achieves significant data compression while enabling real-time rendering and robust 3D tracking of complex, non-rigid motions.

Introduction

Modeling dynamic 3D scenes from video is a cornerstone of computer vision, with applications ranging from 3D video production to autonomous navigation. However, the sheer volume of data is a massive bottleneck. Traditional methods often store a complete 3D model for every single frame. This approach is redundant—static objects like buildings or trees don't need to be re-saved thousands of times—and it doesn't scale well for long sequences.

Furthermore, many existing models assume surfaces are "Lambertian" (meaning they look the same from every angle). In reality, real-world objects have specular highlights and reflections that change based on the viewer's position. Capturing these view-dependent effects while keeping the data footprint small is a significant challenge.

Proposed Methodology

The authors propose a Dynamic Probabilistic Volumetric Model that treats space and time as an integrated 4D structure.

1. The 4D Data Structure

The scene is divided into a "time-supplemented hybrid grid-octree." * Spatial Subdivision: The scene is divided into a grid of shallow octrees (4 levels deep). This "shallow" nature is intentional; it minimizes memory accesses, making it highly optimized for GPU processing. * Temporal Subdivision: Each 3D cell contains a binary "time tree." If a region is static, the tree remains a single root node. If motion occurs, the tree subdivides to store new data only for the necessary time intervals.

The hybrid grid-octree structure combined with a shallow time tree for efficient 4D representation.

2. View-Dependent Appearance

To handle shiny surfaces, the model moves beyond simple color averages. It implements a mixture of Gaussian distributions parameterized by canonical viewing directions. When rendering, the model weights these distributions based on the current camera ray, effectively capturing specular highlights.

3. Online Modeling Algorithm

The system processes data in a "conform and compare" fashion. It predicts what the next frame should look like based on its current 4D model. If the incoming data matches the prediction (measured via KL divergence), it treats the area as static. If there’s a mismatch, it adaptively subdivides the time tree and allocates memory to "explain" the new motion.

Datasets and Evaluation

The framework was evaluated using the publicly available INRIA "4-d repository," which consists of high-resolution (1624x1224) imagery from 16 synchronized cameras. The specific sequences used include:

  • STICK: An actor performing Taekwondo with a staff.
  • GUARD PUNCH TWO: Two actors in karate uniforms.
  • ADULT CHILD BALL & BOY PLAYING BALL: Sequences involving fast-moving, deforming objects (bouncing balls) and non-rigid human motion.

Key Results and Performance

The system demonstrates a strong balance between visual fidelity and computational efficiency.

  • Rendering Quality: The view-dependent model significantly outperformed standard Gaussian and Mixture of Gaussian (MoG) models. For the STICK sequence, it achieved an SSIM of 0.70, compared to 0.59 for a standard Gaussian model.
  • Compression vs. Performance: The system achieved compression ratios of at least 3x with negligible loss in visual quality. In static regions, compression was even higher.
  • Speed: The baseline performance for rendering a novel view was approximately 100ms, enabling interactive frame rates.
Quantitative analysis showing the relationship between compression ratio, SSIM, and rendering speedup across different datasets.
  • Tracking: The model successfully tracked a plastic ball undergoing rapid velocity changes and non-rigid deformations, as well as a fast-rotating Taekwondo staff, without requiring a pre-defined shape model.

Discussion and Future Work

The primary limitation identified is motion blur/artifacts at high compression. If the temporal subdivision is too coarse for an extremely fast-moving object, the model produces "ghosting" artifacts.

While the system excels in controlled studio environments, moving to uncontrolled outdoor environments presents new challenges in camera synchronization and foreground segmentation. Future work aims to refine the 3D tracking to be more adaptive and to test the 4D representation on a much larger geographic scale.

Summary updated

BibTeX
@inproceedings{ulusoy2013dynamic,
  author = {Ali Osman Ulusoy and Octavian Biris and Joseph L. Mundy},
  title = {{Dynamic Probabilistic Volumetric Models}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2013},
  doi = {10.1109/iccv.2013.68},
}
Figure