DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting

Seungjun Lee, Gim Hee Lee

Abstract

Reconstructing sharp 3D representations from blurry multi-view images is a long-standing problem in computer vision. Recent works attempt to enhance high-quality novel view synthesis from motion blur by leveraging event-based cameras, benefiting from their high dynamic range and microsecond temporal resolution. However, they often reach sub-optimal visual quality, either restoring inaccurate colors or losing fine-grained details. In this paper, we present DiET-GS, a diffusion prior and event stream-assisted motion deblurring 3D Gaussian Splatting (3DGS) framework. Our approach effectively leverages blur-free event streams and diffusion priors in a two-stage training strategy. Specifically, we introduce a novel framework to constrain 3DGS with event double integrals, achieving both accurate color and well-defined details. Additionally, we propose a technique to leverage diffusion priors to further enhance edge details. Qualitative and quantitative results on both synthetic and real-world data demonstrate that DiET-GS produces superior quality novel views compared to existing baselines.

Introduction

High-quality 3D reconstruction techniques like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting typically require perfectly sharp input images. However, in real-world scenarios—especially low-light environments—long exposure times often result in significant motion blur. While event cameras, which measure brightness changes at microsecond resolution, offer a potential solution for deblurring, existing methods often struggle with color accuracy or over-smoothed textures.

High-level overview of the DiET-GS framework showing the integration of blurry images, event streams, and diffusion priors to generate sharp seen and novel views.

DiET-GS addresses these challenges by fusing the high-temporal resolution of events with the rich color information of standard frames, guided by pretrained diffusion priors to fill in gaps where sensor data is noisy or incomplete.

Methodology

The DiET-GS framework operates in two distinct stages to balance structural integrity with fine-grained detail.

Stage 1: DiET-GS (Joint Optimization)

In the first stage, the system optimizes 3D Gaussian primitives using a combination of blurry images and event streams. A key innovation is the Learnable Camera Response Function (CRF). Unlike prior work that uses fixed mathematical models for the Event Double Integral (EDI), DiET-GS learns the mapping between RGB values and event intensities, allowing for better adaptation to real-world sensor behavior. Additionally, Renoised Score Distillation (RSD) leverages a pretrained diffusion model as a prior to guide the reconstruction toward a natural, sharp appearance.

Stage 2: DiET-GS++ (Edge Refinement)

The authors observed that joint optimization can sometimes weaken the effect of the diffusion prior. To address this, Stage 2 introduces an additional refinement step. Extra learnable features ($f_g$) are attached to each 3D Gaussian. During this stage, the base scene is frozen, and only these features are trained using the diffusion prior to "inject" high-frequency edge details and textures.

The detailed two-stage training pipeline: Stage 1 focuses on joint optimization with event streams and learnable CRF, while Stage 2 (DiET-GS++) performs specialized edge refinement.

Key Insights

  • The "Tug-of-War" Solution: By separating training into two stages, the model allows events to establish global structure and color first, while the diffusion prior acts as a specialized detail enhancer later.
  • Events + Diffusion = Sharpness: Events provide motion cues but are noisy; diffusion models provide structural knowledge of sharp edges. Combining them allows the model to maintain structural integrity while ignoring event noise.
  • Learnable CRF: Making the camera response function learnable avoids the color artifacts and "ghosting" common in earlier event-based deblurring methods.

Experimental Results

DiET-GS was evaluated on the synthetic EvDeblur-Blender dataset and the real-world EvDeblur-CDAVIS dataset. It significantly outperformed existing baselines like BAD-NeRF and Ev-DeblurNeRF.

  • Quantitative Performance: On the synthetic Blender dataset, DiET-GS achieved a PSNR of 26.69, a notable improvement over the previous state-of-the-art (24.76).
  • Perceptual Quality: In real-world tests, DiET-GS++ showed a significant jump in no-reference quality metrics like MUSIQ (50.44 vs. 41.32 for baselines), indicating sharper and more visually pleasing results.
  • Qualitative Results: Visual analysis confirms that DiET-GS successfully recovers small text and complex textures (such as brickwork) that are typically obliterated in blurry inputs.

Conclusion

DiET-GS represents a significant step forward in motion-deblurring for 3D reconstruction. By effectively combining the temporal precision of event cameras with the generative power of diffusion priors through a two-stage process, it achieves state-of-the-art results in both synthetic and real-world scenarios. Future work may involve integrating this framework into real-time SLAM systems for robotic applications.

Summary updated

Abstract

Reconstructing sharp 3D representations from blurry multi-view images are long-standing problem in computer vision. Recent works attempt to enhance high-quality novel view synthesis from the motion blur by leveraging event-based cameras, benefiting from high dynamic range and microsecond temporal resolution. However, they often reach sub-optimal visual quality in either restoring inaccurate color or losing fine-grained details. In this paper, we present DiET-GS, a diffusion prior and event stream-assisted motion deblurring 3DGS. Our framework effectively leverages blur-free event streams and diffusion prior in a two-stage training strategy. Specifically, we introduce the novel framework to constraint 3DGS with event double integral, achieving both accurate color and well-defined details. Additionally, we propose a simple technique to leverage diffusion prior to further enhance the edge details. Qualitative and quantitative results on both synthetic and real-world data demonstrate that our DiET-GS is capable of producing better quality of novel views compared to the existing baselines. The project page link is attached in main paper.
BibTeX
@inproceedings{lee2025diet,
  author = {Seungjun Lee and Gim Hee Lee},
  title = {{DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  url = {https://openreview.net/forum?id=8Vs39ZLagS},
}
Figure