DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting

Seungjun Lee, Gim Hee Lee

Introduction

Reconstructing high-fidelity 3D scenes from multi-view images is a cornerstone of applications in virtual reality, robotics, and digital twin creation. However, standard reconstruction techniques, such as 3D Gaussian Splatting (3DGS), typically require perfectly sharp input photographs. In real-world scenarios—particularly those involving high-speed motion or low-light conditions—camera motion often introduces significant blur, leading to degraded reconstructions.

DiET-GS introduces a novel framework that addresses this challenge by combining the high-speed temporal data of event cameras with the generative capabilities of pretrained diffusion models. By leveraging a two-stage training strategy, DiET-GS overcomes the common trade-off between accurate color reproduction and sharp edge definition, producing high-fidelity 3D models from blurry inputs.

The Challenge of Motion Blur in 3D Reconstruction

Motion blur occurs when the camera moves during the exposure time of a frame, effectively averaging the scene's appearance over a short interval. While researchers have previously attempted to use event cameras—sensors that record pixel-level brightness changes at microsecond resolution—to "undo" this blur, existing methods often fall short.

Prior approaches using Neural Radiance Fields (NeRF) or early event-based deblurring frequently struggle with a "color-vs-detail" dilemma: they either produce accurate colors with blurry edges or sharp edges with distorted, jittery colors. Furthermore, the computational overhead of NeRF makes real-time applications difficult. DiET-GS aims to bring the efficiency of 3DGS to the deblurring problem while resolving these visual inconsistencies.

Proposed Methodology

The DiET-GS framework is divided into two distinct stages to decouple the reconstruction of geometry from the refinement of visual details.

Stage 1: DiET-GS (The Foundation)

In the initial stage, the system optimizes 3D Gaussians using a combination of blurry RGB images and event streams. This stage relies on three core components:

  • Event Double Integral (EDI): This mathematical constraint relates the event stream to the blurry frames, providing a temporal bridge that helps the model understand how the scene changed during the exposure.
  • Learnable Camera Response Function (CRF): To address the discrepancy between how standard cameras perceive color and how event cameras perceive brightness, the authors introduce a learnable CRF. This ensures higher color accuracy by mapping the two data sources into a shared space.
  • Renoised Score Distillation (RSD): The framework incorporates a diffusion prior to ensure the reconstructed scene maintains the characteristics of a natural, sharp image.

Stage 2: DiET-GS++ (The Refiner)

The authors observed that raw event data and diffusion priors can occasionally conflict during training, leading to noisy results. Stage 2, dubbed DiET-GS++, addresses this by freezing the 3D geometry and adding extra learnable features to each Gaussian. These features are guided exclusively by a diffusion model to "paint in" ultra-sharp edge details and textures that the event data might have missed, preventing the "geometry collapse" often seen when over-optimizing with generative priors.

Experimental Evaluation

Datasets

The model was rigorously evaluated on two primary benchmarks:

  1. EvDeblur-Blender: A synthetic dataset featuring four scenes (factory, pool, tanabata, trolley) with simulated motion blur and synchronized event streams.
  2. EvDeblur-CDAVIS: A real-world dataset captured with a DAVIS346 event camera, containing five scenes (batteries, figures, drones, lab equipment, powersupplies) characterized by authentic sensor noise and complex motion.

Results

DiET-GS significantly outperforms previous state-of-the-art methods, such as Ev-DeblurNeRF and E2NeRF, in both quantitative and qualitative metrics:

  • Quantitative Performance: On the real-world CDAVIS dataset, DiET-GS achieved a PSNR of 34.22, a notable improvement over the previous best of 32.30. Perceptual quality also saw a massive leap; DiET-GS++ reached a MUSIQ score of 50.44, compared to 41.32 in prior work.
  • Qualitative Quality: Visual results demonstrate a striking difference in clarity. While previous methods often leave "ghosting" artifacts or smudged text, DiET-GS++ successfully restores readable labels and crisp textures.
  • Efficiency: The Stage 2 refinement process is highly efficient, requiring less than 20 minutes of additional training time to achieve superior sharpness.

Discussion and Insights

A key insight of this work is the "Mutual Compensation" strategy. By applying EDI constraints in both the RGB color space and the monochrome brightness domain, the model allows the two data sources to compensate for each other's weaknesses: events provide precise edge locations, while RGB data provides accurate color information. Additionally, the use of a cycle consistency loss—ensuring that a "re-blurred" version of the sharp output matches the original input—acts as a powerful regularizer.

Limitations and Future Work

Despite its success, the framework has limitations. It currently assumes relatively uniform-speed camera motion; handling highly erratic or non-linear motion remains a challenge. Furthermore, extremely noisy event streams in very low-light conditions can still introduce artifacts. Future research may focus on integrating the two stages into a more seamless end-to-end process and exploring more complex real-world lighting environments.

Conclusion

DiET-GS represents a significant step forward in 3D scene reconstruction from degraded inputs. By effectively fusing the temporal precision of event cameras with the generative power of diffusion models, it provides a robust solution for creating sharp, high-fidelity 3D Gaussian Splatting models in the presence of significant motion blur.

Summary updated

BibTeX
@inproceedings{lee2025diet,
  author = {Seungjun Lee and Gim Hee Lee},
  title = {{DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  url = {https://openreview.net/forum?id=8Vs39ZLagS},
}
Figure