Noise-Calibrated Symmetry Breaking for Deblurring Dynamic Scenes

When a camera films a moving, bending subject, such as a dancer, a child on a seesaw, or a horse, each frame is blurred by motion that happened while the shutter was open. Recent methods rebuild a sharp, moving 3D scene from such video. They do this by simulating the shutter: they render several sharp "sub-frames" across the exposure and average them so the result matches the blurry frame. These methods carry a hidden trap. The optimizer can also explain the blurry frame by assuming no motion during the exposure and a scene that is itself smeared. This is the classic kernel-image ambiguity of blind deblurring, now appearing in time instead of space. Existing systems hold the trap off with hand-weighted regularizers. Those weights quietly assume a known, uniform noise level, and real low-light video breaks that assumption.

This note proposes to make the balance between "the scene moved" and "the scene is smeared" independent of the unknown noise level. We replace the squared photometric loss used by current 4D deblurring systems with a grouped square-root loss, the pivotal principle from sparse regression. In that principle the right regularization weight does not depend on the noise variance. We pair this loss with an explicit sharpness prior on the rendered sub-frames. The testable claim is that a single fixed set of regularizer weights then works across noise levels, noise types and scenes. Under the same conditions, the exposure-collapse behavior of current methods gets worse as noise grows and must be re-tuned per setting.

Core Claim

The reconstruction objective in dynamic deblurring trades data fit against priors. The priors decide which of two near-equally good explanations wins: true motion with sharp appearance, or zero motion with blurred appearance. With a squared loss, the correct prior weight scales with the noise level. Lasso theory makes this explicit, since $\lambda$ must be set proportional to $\sigma$. A misjudged or spatially varying noise level therefore silently re-weights the priors and tips the balance. Noise that is heteroscedastic, as Poisson-Gaussian sensor noise is in dark versus bright regions, makes the problem worse.

Square-root lasso: pivotal recovery of sparse signals via conic programming showed that dropping the square on the residual norm removes this dependence in linear sparse regression. The underlying reason is a one-line identity. For a residual $r$ of length $n$,

$$\min_{\sigma>0}\; \frac{\|r\|_2^2}{2\sigma} + \frac{n\sigma}{2} \;=\; \sqrt{n}\,\|r\|_2 ,$$

so the square-root loss is the squared loss with the noise scale $\sigma$ estimated jointly and in closed form. The $n\sigma/2$ term ties that estimate to the actual residual. It does not let $\sigma$ drift freely. Noise-Blind Image Deblurring showed that letting $\sigma$ drift freely makes MAP deblurring collapse to "the blurry image is the answer and the noise is zero."

We apply this identity per group of pixels. Each group is an image tile crossed with an intensity bin, so that noise within a group is roughly uniform. The per-group form gives every group its own implicit noise estimate. The full objective over scene parameters $\theta$ (Gaussian positions, appearance, and per-Gaussian motion during exposure) is

$$\mathcal{L}(\theta) = \sum_{g} \sqrt{n_g}\,\big\|Y_g - \bar R_g(\theta)\big\|_2 \;+\; \lambda_s \sum_{k} \big\|\nabla R^{(k)}(\theta)\big\|_0^{\epsilon} \;+\; \lambda_m \sum_{j} \|m_j\|_2 .$$

The symbols are as follows:

  • $Y_g$ is the observed blurry pixels in group $g$, which has $n_g$ pixels.
  • $\bar R_g$ is the average of the rendered sharp sub-frames over the exposure.
  • $R^{(k)}$ is the $k$-th sharp sub-frame.
  • $\|\cdot\|_0^{\epsilon}$ is a smoothed gradient-sparsity penalty in the style of Deblurring Text Images via L0-Regularized Intensity and Gradient Prior. It pushes toward the "sharp scene" explanation.
  • $m_j$ is the within-exposure motion of a cluster $j$ of Gaussians. Its group penalty, in the style of The Group Square-Root Lasso, says that only a few clusters move. It replaces externally supplied dynamic masks.

The two priors pull in opposite directions. Gradient sparsity favors motion and sharpness; motion sparsity favors stillness. The point of the pivotal data term is that the ratio between them, $\lambda_s/\lambda_m$, can be fixed once and left alone across noise conditions.

The group size is a meaningful knob. With one group for the whole image, the loss is the global square-root loss, which handles unknown but uniform noise. With one pixel per group, it reduces to an $\ell_1$ loss. We predict that per-pixel adaptivity is too flexible: it can absorb the systematic residual that moving, blurred regions produce, and so re-open the trap. An intermediate grouping should be best. The ablation section tests this directly.

Relation to Prior Work

Three systems motivate the proposal by exposing the gap.

  • DyBluRF simulates exposure by averaging radiance-field renders at sub-timestamps and models trajectories with a truncated cosine basis.
  • Deblur4DGS moves to Gaussian splatting (a scene made of many small 3D blobs rendered by projection). It learns exposure time and needs an exposure regularizer to stop that time collapsing to zero. It also relies on SAM 2 masks to know what moves.
  • UPGS folds camera pose refinement into the same loop with per-Gaussian rigid transforms.

All three use a squared photometric loss and fixed regularizer weights. None models unknown or heteroscedastic noise. Deblur4DGS's exposure regularizer is the clearest symptom of the temporal kernel-image ambiguity. We do not add another heuristic against it. Instead we change the data term so that one principled sharpness prior suffices across noise levels.

Two lines of work supply the method.

Two papers constrain the design. Jin et al.'s analysis of the noise-blind MAP collapse rules out simply making $\sigma$ a free learnable parameter; the concomitant form above keeps $\sigma$ tied to the residual. Sparse Signal Recovery Using Markov Random Fields shows that clustered support priors help under noise, which supports grouping the motion penalty by spatial cluster. Grouping also carries a known risk: Superpoint Gaussian Splatting and RigidFlow show that rigid clusters fail on elastic motion. Our clusters therefore share a sparsity pattern, meaning whether they move at all. They do not share a rigid transform, and each Gaussian keeps its own motion.

Research Program

We first study a reduced problem where the claim can be checked almost analytically: one-dimensional and two-dimensional blind deconvolution of step edges and natural patches. We will measure the prior weight at which the optimizer switches from the true kernel to the delta kernel, as a function of noise level, under squared and square-root losses. The prediction is that this switching weight moves with $\sigma$ under the squared loss and stays flat under the square-root loss. If the prediction fails here, the 4D program stops.

The second stage implements the objective as a drop-in replacement in the public-style pipelines of Deblur4DGS and UPGS. The renderer, motion model, pose handling and schedule stay unchanged. Only three things change: the photometric loss becomes the grouped square-root loss, the exposure regularizer is replaced by the sub-frame sharpness prior, and the external dynamic masks are replaced by the group-sparse motion penalty. The third stage measures robustness under controlled corruption. Stereo Blur videos receive added Gaussian noise, Poisson-Gaussian noise at several gain levels, and a small fraction of saturated or hot pixels. The sparse corruptions match the mixed-fault setting of Compressed Sensing with Sparse Corruptions, and we will check whether the per-group form tolerates them or needs an explicit sparse outlier term.

Baselines

The naive reference point is Deblur4DGS run as published, with its squared loss and its default exposure, multi-frame and multi-resolution regularizer weights fixed at their released values for every noise condition. It answers the question practitioners actually face: what happens if you run the existing tool on noisier footage without re-tuning?

The strongest existing method is UPGS. On the Stereo Blur Dataset it reports beating Deblur4DGS by 1.12 dB PSNR with a 35% reduction in LPIPS, at about 4.2 hours of training. For reference, Deblur4DGS reports 27.16 dB PSNR on its 720p synthetic data and DyBluRF reports 25.66 dB PSNR and 0.079 LPIPS on Stereo Blur. We will rerun UPGS on our noise-augmented sequences with the same initial poses, the same number of sub-frames, the same iteration count and the same GPU budget. It will receive a per-noise-level grid search over its regularizer weights of the same size as the single grid search our method is allowed on one clean scene. Beating it means two things. On clean Stereo Blur we must match its PSNR within 0.2 dB. Across the noisy conditions we must beat it in PSNR and LPIPS with our weights frozen, while UPGS keeps its per-condition tuned weights. This is deliberately harder for us than a frozen-versus-frozen comparison.

Three ablations isolate the key ingredient. The first keeps our sharpness and motion priors but restores the squared loss; it tests whether the pivotal data term, not the new priors, carries the robustness. The second sweeps the group size from one global group to single pixels, i.e. the $\ell_1$ limit. The third swaps the group-sparse motion penalty back to SAM 2 masks, to separate mask discovery from noise robustness.

For the two-dimensional sanity stage, the references are Pan et al.'s $L_0$ blind deconvolution and GradNet from Jin et al., run with an estimated kernel. Jin et al. report only relative results on the Levin and Sun benchmarks: matching or beating EPLL and CSF at 1–4% noise, and a 0.3–0.7 dB gain over ground-truth noise settings. No absolute PSNR on those benchmarks was found on Lacuna for either method, so we will rerun both.

Why not just…?

Why not just use an $\ell_1$ or Charbonnier photometric loss? This is the most serious threat, because an $\ell_1$ loss is also insensitive to noise scale. In our formulation it is exactly the one-pixel-per-group limit, so the group-size sweep settles the question directly. Our hypothesis is that per-pixel normalization lets the systematic residual of a blurred moving region be downweighted like noise, which raises the collapse rate on fast-moving subjects. If $\ell_1$ matches the intermediate grouping in the experiments, we will report that the simpler loss suffices and recast the contribution as the noise-scaling analysis plus the mask-free motion prior.

Why not just learn a per-pixel uncertainty map and weight the loss by it? A free per-pixel noise scale is the unconstrained version of what Jin et al. showed collapses: the optimizer can declare the blurry region "uncertain" and stop explaining it. We will run this variant with a standard log-variance penalty under the same budget. We expect it to show the highest exposure-collapse rate of all variants, measured as the ratio of recovered within-exposure motion to ground-truth motion on synthetic sequences where that motion is known.

Why not just denoise the frames first, or re-tune the regularizer weights for each video? Denoising removes exactly the fine gradients that distinguish motion blur from smeared appearance. DyBluRF also already shows that independent per-frame preprocessing before dynamic reconstruction loses to joint modeling, and Real-World Blur Dataset shows 2D restoration networks transfer poorly to real sensor data (SRN-DeblurNet drops from 38.65 to 30.72 dB PSNR on RealBlur-R when trained on synthetic GoPro blur). Per-video re-tuning is unavailable in deployment because no sharp ground truth exists to tune against. We still give it to UPGS as an oracle-tuned baseline so that any win is not just a tuning artifact.

Proposed Experiments

The main evaluation uses the six Stereo Blur sequences (Sailor, Seesaw, Street, Children, Skating, Basketball). We report PSNR, SSIM and LPIPS against the sharp stereo views, plus training time. Each sequence is evaluated clean and under the corruption ladder described above.

The decisive check is a tuning-transfer curve. For each method, we find the best regularizer weights on one clean sequence, freeze them, and plot PSNR on every other sequence and noise level. We also plot the location of the best weight against noise level. The proposal succeeds if two things hold together: our best weight stays flat while the squared-loss variants' best weight drifts with noise, and our frozen-weight PSNR stays within a small margin of the per-condition oracle.

The second check is mask discovery. We compare the moving clusters selected by the group penalty with SAM 2 masks on the dynamic subjects using intersection-over-union, and confirm that rendering quality does not drop when the masks are removed.

Real low-light behavior is probed with the RealBlur-R raw images, using a 2D version of the objective. This test is limited because RealBlur provides image pairs, not monocular video. It speaks only to the heteroscedastic noise model, not to the 4D claim.