ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning

Marie-Lena Eckert, Kiwon Um, Nils Thuerey

Abstract

In this paper, we present ScalarFlow, a first large-scale data set of reconstructions of real-world smoke plumes. We additionally propose a framework for accurate physics-based reconstructions from a small number of video streams. Central components of our algorithm are a novel estimation of unseen inflow regions and an efficient regularization scheme. Our data set includes a large number of complex and natural buoyancy-driven flows. The flows transition to turbulent flows and contain observable scalar transport processes. As such, the ScalarFlow data set is tailored towards computer graphics, vision, and learning applications. The published data set will contain volumetric reconstructions of velocity and density, input image sequences, together with calibration data, code, and instructions how to recreate the commodity hardware capture setup. We further demonstrate one of the many potential application areas: a first perceptual evaluation study, which reveals that the complexity of the captured flows requires a huge simulation resolution for regular solvers in order to recreate at least parts of the natural complexity contained in the captured data.

Introduction

In the field of Visual Effects (VFX), creating visually appealing simulations of smoke, fire, and water has reached high levels of sophistication. However, a significant gap remains: the lack of ground truth data to verify the physical accuracy of these simulations. Capturing the 3D motion of transparent, volumetric substances like smoke is notoriously difficult. Traditional methods such as Particle Image Velocimetry (PIV) require seeding the air with particles that obscure the smoke itself. Consequently, research has largely relied on synthetic data or small-scale captures. ScalarFlow addresses this by providing the first large-scale, 3D volumetric dataset of real-world smoke plumes, offering a benchmark for computer animation and machine learning.

Methodology

The reconstruction of 3D density and velocity fields from 2D video streams is treated as an inverse problem. The authors propose a framework that combines a low-cost hardware setup with a physics-constrained optimization loop.

Hardware Setup

The authors developed an accessible hardware setup (costing under $1,100) to capture buoyant plumes. The system includes an insulated, heated box to generate buoyancy, a standard fog machine with servo-controlled valves, and five Raspberry Pi cameras to capture multi-view video.

Schematic of the hardware flow setup, including the container, fog machine, and valves.

Physics-Constrained Reconstruction

The core of the software framework is a physics-constrained optimization loop guided by the incompressible Navier-Stokes equations.

  • Inflow Estimation: A novel solver estimates the smoke entering the volume from unseen regions at the bottom. Accurate source estimation is critical for physical consistency over time.
  • Efficient Optimization: A specialized Conjugate Gradient Least Squares (CGLS) solver was implemented, achieving 8x speedup and 6x memory reduction compared to standard methods.
  • Simulation-in-the-Loop: The algorithm steers a fluid simulation to match real-world footage, ensuring the 3D data is both visually accurate and physically plausible.
Diagram of the reconstruction volume and the inflow region used for flux estimation.

The ScalarFlow Dataset

The ScalarFlow dataset consists of 100 unique sequences of real-world buoyant smoke plumes. Each sequence contains at least 150 time steps, providing 3D density and velocity fields at a resolution of 100x177x100. In total, the dataset encompasses approximately 26 billion voxels. The release includes raw multi-view video, camera calibrations, and the code necessary for reproduction.

Key Insights and Results

  • Physics as Regularization: In the under-constrained task of 3D reconstruction from five views, the laws of physics act as a powerful filter against unrealistic artifacts.
  • The Resolution Gap: A perceptual study revealed that standard digital solvers require 8x to 12x the resolution typically used in production to match the natural complexity of a captured 90cm smoke plume.
  • Iterative Refinement: "Advection-aware" reconstruction, which accounts for temporal movement, significantly outperforms traditional static tomography.
  • Performance: The framework achieved high fidelity with Peak Signal-to-Noise Ratios (PSNR) between 33 and 37 dB against synthetic ground truth.

Limitations and Future Work

The current model assumes linear image formation (thin smoke), which may not generalize to opaque smoke with significant light scattering. Additionally, while the setup is insulated, ambient air currents still introduce minor variations. Future work will leverage this dataset for training deep learning models in fluid super-resolution and learning complex boundary conditions.

Summary updated

Abstract

In this paper, we present ScalarFlow, a first large-scale data set of reconstructions of real-world smoke plumes. We additionally propose a framework for accurate physics-based reconstructions from a small number of video streams. Central components of our algorithm are a novel estimation of unseen inflow regions and an efficient regularization scheme. Our data set includes a large number of complex and natural buoyancy-driven flows. The flows transition to turbulent flows and contain observable scalar transport processes. As such, the ScalarFlow data set is tailored towards computer graphics, vision, and learning applications. The published data set will contain volumetric reconstructions of velocity and density, input image sequences, together with calibration data, code, and instructions how to recreate the commodity hardware capture setup. We further demonstrate one of the many potential application areas: a first perceptual evaluation study, which reveals that the complexity of the captured flows requires a huge simulation resolution for regular solvers in order to recreate at least parts of the natural complexity contained in the captured data.
BibTeX
@article{eckert2019scalarflow,
  author = {Marie-Lena Eckert and Kiwon Um and Nils Thuerey},
  title = {{ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning}},
  journal = {ACM Trans. Graphics},
  year = {2019},
  url = {https://openreview.net/forum?id=MsQqwLBqRuv},
}
Figure