Seeing Thin Walls Through the Flow: Differentiable Double-Sided Bounce-Back for Smoke-and-Shell Reconstruction

Methods that reconstruct a moving fluid from video handle open air and thick containers well. They break as soon as the fluid meets a thin object such as a sheet, a flag, a perforated plate or a rod. Two separate failures add up here. First, neural scene representations draw a thin wall as a semi-transparent fog, because photographs alone cannot tell a sharp opaque surface from a smeared cloud that produces the same pixels. Second, the fluid simulators used inside these pipelines let fluid pass straight through anything thinner than a grid cell. The fluid side of the reconstruction is therefore corrupted by phantom leaks, and the solid side stays foggy. Each error hides the other.

We propose to fix both with one shared quantity. Each element of the thin wall gets an opacity, and that same number is used for two jobs. It controls how much the element hides what lies behind it in the rendered image. It also controls how strongly the element reflects fluid in a grid-based fluid solver, using a "double-sided bounce-back" rule borrowed from computational fluid dynamics. Under this coupling, a half-transparent smear of wall is visible to the camera and also lets half the smoke through, so the observed flow contradicts it. If the camera sees smoke piling up on one side of a sheet and curling around its edge, the optimizer gets evidence for where the sheet is and that it is fully opaque. Ordinary photometric fitting cannot supply that evidence. The fluid becomes a probe of the solid geometry.

Core Claim

Coupling rendering opacity and physical impermeability, and making the impermeability enforcement differentiable and double-sided, should give two results. It should give leak-free fluid reconstructions around thin structures, which no current fluid-from-video method supports. It should also give sharper, more complete thin-shell geometry than image-only reconstruction, especially from sparse views. The second result is the claim that would be new. The first is a necessary engineering step toward it.

Background and Positioning

The fluid-solid mechanism comes from forward simulation. Fast and versatile fluid-solid coupling for turbulent flow simulation introduces double-sided bounce-back (DSBB) for lattice Boltzmann solvers. Lattice Boltzmann solvers advance fluid by streaming particle-distribution "populations" along fixed grid links and colliding them at grid nodes. DSBB reflects populations on every grid link a thin surface cuts, whether the cell is labelled fluid or solid. This seals the shell from both sides, removes leakage, and runs up to 4x faster than earlier kinetic coupling. It exists only as a forward tool on explicit meshes, and its authors note that it degrades at sharp corners and has no efficient handling of deforming shells. We reuse its core rule and make it differentiable and opacity-weighted.

Barrier-based methods are the main alternative way to guarantee non-penetration. Penetration-free Solid-Fluid Interaction on Shells and Rods and A Contact Proxy Splitting Method for Lagrangian Solid-Fluid Coupling enforce it inside Newton solves with barrier energies and continuous collision detection. They are exact, but differentiating through them for many inverse iterations would be expensive. For that reason we treat them as a motivating constraint, not as the method to reuse. Both establish that ordinary velocity boundary conditions do not stop fluid tunneling through codimensional, zero-thickness solids.

On the inverse side, three pipelines recover fluids or continua from video:

  • FluidNexus couples video diffusion with Gaussians and differentiable position-based fluids, starting from a single view.
  • PAC-NeRF advects particles with a differentiable material point method.
  • NeuroFluid pairs learned Lagrangian fluids with a particle renderer.

All three support only open domains or thick rigid containers. PAC-NeRF explicitly leaves thin shells out of scope. GIC fixes volumetric density noise for physics by fitting silhouettes instead of colors and filling Gaussians into solid continua. It is also limited to volumetric bodies. These works motivate the gap.

On the density-noise side, NerfingMVS and ClimateNeRF suppress fog and floaters with depth priors and distortion losses on static scenes. These are image-side regularizers we will compare against. Our proposal adds a physics-side signal they lack.

Finally, the static-scene line treats the moving part and the static part as non-interacting layers. Split4D separates them, and RobustSplat discards transients to protect the background. We instead let the dynamic smoke inform the static shell.

Research Program

The scene has two parts. The thin structure is represented by flat, oriented Gaussian surfels: disk-shaped primitives with a position, normal, extent and opacity $\alpha_k \in [0,1]$. No volumetric density is allowed for the shell, which removes fog by construction. The fluid is a passive smoke density advected by a lattice Boltzmann velocity field on a coarse grid. The smoke is rendered volumetrically, as in the ScalarFlow line of work.

For each grid link from node $x$ in direction $i$ that a surfel $k$ crosses at fractional distance $q \in (0,1)$, the incoming population is replaced by a blend of transmitted and reflected populations:

$$f_{\bar i}(x, t+1) = (1-\alpha_k)\, f_{\bar i}^{\text{stream}}(x,t+1) + \alpha_k\, B_q\!\left(f_i^{*}(x,t),\, u_k\right)$$

The symbols are as follows:

  • $f_i^{*}$ is the post-collision population leaving $x$ along direction $i$.
  • $\bar i$ is the opposite direction.
  • $f_{\bar i}^{\text{stream}}$ is what normal streaming would deliver.
  • $B_q$ is an interpolated bounce-back that places the reflection at fraction $q$ along the link and accounts for the wall velocity $u_k$.

The rule is applied symmetrically on both sides of the surfel, which is the double-sided part. Because $B_q$ is smooth in $q$, and $q$ is smooth in the surfel's position and normal, gradients flow from smoke-image errors back to shell geometry. The discrete event of whether a link is crossed changes only when a surfel sweeps past a grid node. We handle that with a straight-through estimator and anneal surfel extents.

The same $\alpha_k$ is used in alpha compositing during rendering. This tie is the central design choice. The training loss combines photometric error on smoke and shell pixels with a light divergence penalty, so no extra fog regularizer is needed on the shell.

Optimization proceeds in three stages:

  1. Initialize the shell from frames or regions with little smoke, using standard surfel splatting.
  2. Fit the fluid's initial state and inflow with the shell frozen.
  3. Optimize both jointly through time with checkpointed backpropagation.

The lattice Boltzmann update is local and explicit, so its adjoint is a local reverse stream-and-collide. Memory, not solver complexity, is the main cost. We also study the regime of a single input view, where extra views are hallucinated by a video diffusion model in the style of FluidNexus. The physics coupling should then reject geometrically inconsistent hallucinations around the shell. Such geometric inconsistency in generated views is what GeoVideo handles only by masking out dynamic regions.

Deforming shells such as a flapping flag are a stretch goal. They would also test the DSBB limitation on moving boundaries.

Baselines

The naive reference is a decoupled pipeline. A static 3D Gaussian or surfel reconstruction of the shell is followed by fluid reconstruction with ordinary single-sided bounce-back on the thresholded shell, run in the same lattice Boltzmann code at the same grid resolution. This is what a practitioner would assemble today. It isolates what is lost when the shell is fixed and the boundary rule is the standard one. Our method must beat it on leakage flux and on shell Chamfer distance.

The strongest existing fluid-from-video method is FluidNexus. It reports novel-view PSNR 32.45 dB and velocity divergence 0.0126 on ScalarFlow, against roughly 22.6 dB and divergence 0.0297 for PINF. We will run the released system on our thin-shell scenes with the same input views. In the multi-view setting we will feed it real views in place of its generated ones. Its obstacle handling will receive the shell mesh from our stage-one initialization, so that it is not handicapped on geometry. Tuning will match ours: the same number of optimization steps and a small grid over its constraint weights. No result exists for thin-shell scenes: no reported result found on Lacuna for any method on smoke past thin shells. Beating FluidNexus therefore means two things. On the shell scenes we need lower leakage, lower velocity error and comparable or better PSNR. On ScalarFlow, which has no shell, we must not fall meaningfully below 32.45 dB, which confirms that the lattice Boltzmann fluid model itself is not the weakness.

For the shell-geometry claim, the strongest image-only competitor is surfel splatting with depth-prior guidance in the spirit of NerfingMVS, plus the distortion-style losses of ClimateNeRF. It will be fitted to the same images with the same view counts.

Four ablations isolate the key ingredients:

  1. Untied. A separate learned impermeability per surfel replaces the shared $\alpha_k$. This tests whether the tie, and not simply the physics, sharpens geometry.
  2. Single-sided. Only cells labelled fluid apply the reflection.
  3. Binarized. $q$ is snapped to the link midpoint, which kills geometry gradients.
  4. Thickened. The shell is inflated to one or two grid cells and handled with ordinary bounce-back, following the multi-layer boundary idea of Kulp et al..

Why not just…?

Why not just thicken the shell and use an existing differentiable solver? This is the most serious threat. Kulp et al. show that multi-layer boundary particles stop tunneling in SPH, and thickening is trivial to add to FluidNexus or PAC-NeRF. The cost is that the coarse grids needed for differentiable rollouts force a thickness of one or more cells. That shifts the effective wall, narrows gaps in perforated plates, and distorts the forces the wall exerts. Lyu et al. note that this staircase effect already corrupts force and torque in forward simulation. The thickened ablation will be run at one and two cell widths. If it matches our method on leakage, velocity error and shell geometry, the double-sided operator is unnecessary and we will report that. We expect it to lose specifically on perforated plates and slots narrower than two cells, and on recovered shell position.

Why not just reconstruct the shell well from images and then simulate? Depth priors fix fog only coarsely: NerfingMVS reaches Abs Rel 0.0614 on ScanNet, still on static scenes. In addition, Sun et al. and Li et al. show that a correct mesh with standard velocity boundary conditions still leaks around codimensional shells. The decoupled baseline and the image-only geometry baseline test both halves of this answer directly. The sparse-view experiment tests whether flow evidence adds geometric information that images alone cannot provide.

Why not just use barrier-based non-penetration, which is exact? IPC-style barriers give strict guarantees. Differentiating through a Newton solve with collision-aware line searches at every step of a long rollout is costly, however, and the contact-proxy paper already reports neighbor search as a bottleneck in the forward direction alone. The bounce-back rule is a local per-link operation with a cheap adjoint. We will report wall-clock time and memory per optimization iteration. The comparison is fair as long as our leakage stays near zero, and the leak-flux metric checks exactly that.

Why not just let a large video model fill in the missing views? VideoPhy finds that CogVideoX-5B satisfies physical commonsense and semantic adherence together in only 39.6% of cases, and that it fails often on solid-fluid interactions. FluidNexus itself needed a physics solver on top of its diffusion model. Our single-view experiment measures this directly. It compares diffusion-only views, diffusion views with a physics solver but no shell, and diffusion views with the coupled shell, scoring each against ground-truth held-out views.

Proposed Experiments

Synthetic benchmark. No existing benchmark contains thin shells in a reconstructable fluid, so we will build one. Ground truth comes from high-resolution forward DSBB lattice Boltzmann simulation following Lyu et al., at four times the inverse grid resolution so that the inverse solver cannot share discretization errors with it. Buoyant smoke flows past a flat sheet, an angled baffle, a perforated plate with sub-cell holes, a thin rod array, and a slowly rotating vane. The scenes are rendered from many cameras. Metrics are:

  • novel-view PSNR and LPIPS
  • velocity error against ground truth
  • divergence
  • a leak flux, defined as the smoke mass crossing the true shell surface per unit time
  • shell Chamfer distance and surface completeness
  • integrated force on the shell against ground truth

Real data. ScalarFlow (ScalarFlow) serves as the no-shell sanity check against FluidNexus's 32.45 dB. A small real capture adds a thin plate inside a ScalarFlow-style rig. Because velocity ground truth is unavailable there, evaluation uses held-out-view PSNR and leak flux measured against a caliper-measured plate position.

Decisive check. Shell Chamfer distance is measured as the number of views drops from many to three to one. Three methods are compared: the image-only geometry baseline, the untied ablation, and the full method. The core claim holds only if the full method's advantage over the image-only baseline grows as views become sparser, and if the untied ablation does not reproduce that advantage. If the tie gives no geometric gain, the project reduces to leak-free fluid reconstruction around thin walls. That result is still useful, but narrower, and we will present it as such.

Second check. On perforated plates with sub-cell holes, the thickened ablation should either seal the holes or misplace them. Our method should recover the hole flow. This is where a reviewer's simplest alternative is most likely to succeed or fail, and the result will decide whether the double-sided operator is worth its complexity.