When a handheld video is both shaky and full of moving people, every frame is blurred for two reasons at once. The camera moved during the exposure, and so did the objects. Current methods that rebuild such scenes in 3D need to know which pixels belong to moving things. They get that answer from an external video segmenter, and segmenters are least reliable on exactly these blurred frames. This proposal starts from a simple physical observation. In a static scene, the blur at every pixel is fixed by the camera's path during exposure and the depth of that pixel. A moving object produces blur that no single camera path can reproduce. The part of the blur the camera cannot explain is therefore a free signal for finding moving objects, with no segmenter required.
That signal is only usable if the reconstruction does not cheat. Here the proposal borrows from blind deconvolution, the problem of recovering both a sharp image and an unknown blur kernel from one blurry photo. That field has a well-known failure called the Dirac trap: the optimizer "explains" the blur by keeping the image blurry and setting the kernel to a single spike. Recent theory shows the trap is often the global minimum, even under learned priors. In 3D Gaussian splatting, which renders a scene as a cloud of small colored ellipsoids, the equivalent failure is to shrink the camera's exposure motion to zero and stretch the ellipsoids into permanently smeared blobs. A smeared scene with no camera motion leaves no unexplained blur, so it hides every moving object. We propose to import the specific countermeasures that blind deconvolution developed for its trap, and to organize the reconstruction as an explicit alternation among three blocks: the sharp scene, the camera's exposure trajectory, and the static/dynamic assignment. Each block is fixed while the others update.
Core Claim
We make two claims, and each can fail on its own. First, dynamic Gaussian deblurring has a measurable "blurry-ellipsoid trap": the smeared, zero-exposure solution reaches equal or lower photometric loss than the sharp one. This is the 3D counterpart of Theorem 1 in MAP, déconvolution aveugle et a priori appris : un subtil mélange. That paper proves blurry images are global minimizers of the posterior under diffusion priors, and shows that a large initial kernel plus many image steps per kernel step reach the sharp local minimum instead. Second, once the trap is avoided, the residual left by a camera-only blur model separates moving from static content. Its quality is competitive with external segmentation masks on blurry input, and the resulting reconstruction matches or beats mask-dependent dynamic deblurring.
Relation to Prior Work
Four papers already combine motion-blur modeling with dynamic reconstruction. All of them treat a blurry pixel as the average of sharp sub-frames rendered along the exposure. DyBluRF does this inside a dynamic neural radiance field, trained jointly. Deblur4DGS does it with 4D Gaussians, adds an exposure regularizer that stops exposure time from collapsing, and relies on SAM 2 masks for dynamic regions. MoBGS predicts continuous camera trajectories with a neural ODE and shares one exposure interval between the static and dynamic branches. UPGS is the closest to our work. It already uses a staged alternation: scene only, then pose only, then joint training.
We differ from all four in three respects. None of them locates dynamic content from the blur model itself. None applies the update asymmetry that the deconvolution theory shows is needed to reach the sharp basin; UPGS's stages are a fixed one-time schedule and do not cycle. And none checks whether the smeared solution is actually preferred by the loss.
BSGS motivates our choice of trajectory solver. It shows that averaging gradients over sub-frames lets opposing gradients from the two ends of a trajectory cancel, and it fixes this by max-pooling gradients. Its scope is limited to static scenes with one rigid motion.
Distractor-free methods motivate our assignment block and also constrain it. RobustNeRF trims the worst half of pixel residuals. DeGauss learns a probabilistic mask that blends a foreground branch and a background branch. Both assume sharp input. RobustNeRF-style trimming would discard blurred pixels rather than explain them, which is fatal when camera shake affects every view. Our assignment reuses their idea of a soft robust split, but applies it to the residual of a physical blur model rather than to raw photometric error.
Research Program
The scene has static Gaussians $G_s$, dynamic Gaussians $G_d$ with per-object motion over the exposure, a camera exposure trajectory $\xi_t$ for frame $t$ (a low-order spline on the rigid-motion group, with a few dozen parameters per frame), and a per-Gaussian static probability $p$. A blurry frame $y_t$ is modeled as the mean of $N$ rendered sub-frames:
$$\hat y_t = \frac{1}{N}\sum_{j=1}^{N} R\big(G_s, G_d(\tau_j), \xi_t(\tau_j)\big),$$
where $R$ is the splatting renderer and $\tau_j$ are sub-exposure times. This keeps the physical sum-to-one exposure average that BSGS's max-pooling gives up.
The alternation has three blocks. The scene block updates $G_s$ and $G_d$ by gradient descent with the trajectory and assignment frozen. It runs many steps for every trajectory update, which is the update-frequency imbalance of Nguyen and Pauwels.
The trajectory block fixes the scene and solves for $\xi_t$ by Gauss-Newton, using only pixels dominated by high-$p$ Gaussians. Because the problem has few parameters per frame, a second-order solve is cheap, and it sidesteps first-order gradient cancellation without dropping the average. This is our 3D counterpart to the cheap, well-conditioned kernel updates in Anger et al. and to the skeleton-proxy idea in Blind Image Deblurring via Reweighted Graph Total Variation: estimate the kernel from the parts of the image that are most trustworthy.
The assignment block renders each frame under a camera-only hypothesis, in which all Gaussians are treated as static and blurred only by $\xi_t$. It then accumulates each Gaussian's share of the remaining residual across views and sets $p$ with a robust, trimmed-style threshold. Gaussians whose blur the camera cannot explain migrate to the dynamic set and receive motion parameters.
Trajectories are initialized wide rather than at zero. This is the counterpart of the large-kernel initialization: if the exposure starts large, the renderer can only match a blurry frame by making the Gaussians compact. Densification is held back until the first full cycle, following the delayed-densification evidence in RobustSplat.
The analysis component is a trap diagnostic, and we treat it as the scientific core. On synthetic scenes with known sharp geometry, we will compare converged loss and mean Gaussian scale for two starting points: the true sharp scene with the true trajectory, and a smeared scene with zero exposure. We will also measure how often ordinary joint optimization falls into the smeared basin.
Baselines
The naive reference is standard 4D Gaussian splatting trained directly on blurry frames with no blur model. It sets the floor and shows how much of the input blur each method removes.
The strongest existing method for rendering quality is UPGS. It is also the most direct competitor, because it already alternates between scene and pose. On Stereo Blur it reports a PSNR 1.12 dB higher than Deblur4DGS and a 35% reduction in LPIPS (UPGS). Deblur4DGS reports 27.16 dB PSNR on its synthetic 720p setting (Deblur4DGS), and DyBluRF reports 25.66 dB PSNR and 0.079 LPIPS on Stereo Blur (DyBluRF). The file does not show that these are on identical splits, so we will rerun all three on one split. Every method gets the same COLMAP initialization, the same sub-frame count $N$, the same GPU-hour budget per scene, and a matched hyperparameter search. Beating UPGS means higher PSNR and lower LPIPS on at least four of the six Stereo Blur scenes at no more than its reported 4.2 hours of training.
The strongest competitor for decomposition is Deblur4DGS with its SAM 2 masks. We will also give UPGS the same masks, so that our segmenter-free assignment is compared against the best available external masks rather than against no masks. Winning here means dynamic-region PSNR within 0.3 dB of the masked variant, or better, with no segmenter.
Four ablations isolate the key ingredients. The first uses zero-initialized exposure with a balanced one-to-one update ratio, which tests the trap countermeasures. The second replaces Gauss-Newton with averaged-gradient and max-pooled-gradient trajectory updates, which tests the solver. The third replaces the camera-residual assignment with a plain trimmed photometric residual, which tests whether the blur model is what makes the assignment work. The fourth runs a single pass with no cycling, which tests the value of cyclic feedback over a staged schedule.
Why not just…?
Why not just penalize large Gaussian scales? This is the most serious threat to the trap argument. A scale penalty might block smeared ellipsoids just as well as asymmetric alternation does. The argument against it is structural: legitimately large, smooth regions such as sky and walls also get penalized, and defocus blur, which Deblurring 3D Gaussian Splatting models precisely by enlarging covariances, would be suppressed. That argument is not evidence, though. We will sweep the penalty weight on Stereo Blur and on the Deblur-NeRF defocus scenes. If some weight matches our method on both, the asymmetry claim fails. We would then keep the scale penalty as the cheaper trap-breaker and rest the contribution on the camera-residual decomposition alone.
Why not just use SAM 2 masks, as Deblur4DGS does? On sharp video, masks from segmenters are competitive. Split4D shows tracking-based decomposition drifting badly even on clean multi-human capture, with mIoU of 0.592 against 0.893 for its tracker-free method. Blur only weakens the edges that segmenters depend on. The masked baseline above settles this question directly. If masks win at every blur level we test, our decomposition is not needed.
Why not just use UPGS's three-stage schedule? UPGS itself notes that a corrupted scene from its first stage misaligns the poses in its second stage, and nothing in the schedule goes back to correct it. The no-cycling ablation and a blur-severity sweep will show whether cyclic alternation matters, or whether a single staged pass is enough in practice.
Why not just deblur each frame in 2D and then reconstruct? DyBluRF reports that its approach beats pipelines of this kind: 25.66 dB against 23.33 dB PSNR for the NSFF baseline. KernelFusion documents the ringing that kernel errors cause when two stages run in sequence. We include the 2D-deblur-then-4DGS pipeline only as a reference point.
Proposed Experiments
The first decisive check is the trap diagnostic. On synthetic blurred renderings of dynamic scenes with known geometry, we test whether the smeared, zero-exposure state has a photometric loss no higher than the sharp state. We also count, over 20 random seeds, how often plain joint optimization ends up there. Suppose the smeared state has clearly higher loss and joint optimization rarely finds it. Then the trap is not real in 3D, and the asymmetry component goes away.
The second check is the main comparison on the six Stereo Blur scenes. We report PSNR, SSIM and LPIPS, split into static and dynamic regions using a mask from the held-out sharp view, together with training time.
The third check is decomposition quality. It needs ground-truth masks under blur, which no benchmark in our file provides; there is no reported result found on Lacuna for blurred-input motion segmentation. We will therefore synthesize blur by averaging consecutive frames of Split4D's multi-view sequences and of KITTI-360. We will report mIoU against their mask annotations at three blur levels, alongside the sharp-input references of 0.893 for Split4D and 64.84% IoU for RoDUS (RoDUS).
The fourth check is a static-scene sanity test. On real ExBluRF scenes, where BSGS reports 29.16 dB PSNR, the assignment block should put nearly all Gaussians in the static set, and rendering quality should not fall below BSGS. A method that invents dynamic objects in a static scene would falsify the residual cue.
References
- MAP, déconvolution aveugle et a priori appris : un subtil mélange
- DyBluRF: Dynamic Neural Radiance Fields from Blurry Monocular Video
- Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video
- MoBGS: Motion Deblurring Dynamic 3D Gaussian Splatting for Blurry Monocular Video
- UPGS: Unified Pose-aware Gaussian Splatting for Dynamic Scene Deblurring
- BSGS: Bi-Stage 3D Gaussian Splatting for Camera Motion Deblurring
- RobustNeRF: Ignoring Distractors with Robust Losses
- DeGauss: Dynamic-Static Decomposition with Gaussian Splatting for Distractor-free 3D Reconstruction
- Blind Image Deblurring using the l0 Gradient Prior
- Blind Image Deblurring via Reweighted Graph Total Variation
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- Deblurring 3D Gaussian Splatting
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
- KernelFusion: Zero-Shot Blind Super-Resolution via Patch Diffusion
- RoDUS: Robust Decomposition of Static and Dynamic Elements in Urban Scenes