TL;DR
ReDiffuse introduces a rotation-equivariant diffusion model tailored for multi-focus image fusion (MFIF). By embedding rotational symmetry directly into the network architecture, it effectively eliminates structural artifacts caused by defocus blur while slashing parameter counts by over 70%.
What Problem Does This Paper Address?
When capturing a scene, cameras with a limited depth of field can only keep objects at a specific distance in sharp focus. Multi-focus image fusion (MFIF) solves this by merging multiple images taken at different focal planes into a single, completely sharp image.
Recently, generative diffusion models have shown great promise in this area. However, they suffer from a major flaw: defocus blur distorts and warps geometric structures like edges and textures. Standard diffusion models lack the geometric awareness to handle these distortions, leading to unnatural artifacts, warped boundaries, and structurally inconsistent fused images.
Methods
The authors propose ReDiffuse, a diffusion framework built from the ground up to maintain rotation equivariance—meaning if you rotate the input image, the network's internal features and final output rotate correspondingly without distortion.
To achieve end-to-end rotation equivariance, the authors systematically redesigned the standard U-Net denoising backbone:
- Rotation-Equivariant Convolutions (Rot-E Conv): They utilize Fourier-series-based filter parameterization (B-Conv) instead of standard convolutions.
- Equivariant Modules: They mathematically adapt downsampling (Maxpooling), upsampling (Bilinear interpolation), and Group Normalization to ensure discrete image grids do not break the rotational symmetry.
The paper provides rigorous theoretical proofs showing that for a standard 90-degree discrete rotation group ($m=4$), ReDiffuse achieves exactly zero equivariance error.
Datasets
- Training & Validation: Real-MFF, a realistic multi-focus dataset containing 600 image pairs for training and 110 for validation.
- Evaluation Benchmarks: Four widely recognized MFIF datasets:
- Lytro (20 image pairs)
- MFFW (13 image pairs with complex scenes)
- MFI-WHU (120 image pairs)
- Road-MF (80 road-scene pairs)
Key Insights and Ideas
- Stability Under Blur: While defocus blur destroys fine pixel-level details, higher-level geometric properties—like local isotropic symmetry and non-local self-similarity—remain stable under rotation. Forcing the network to respect these symmetries stabilizes the entire reconstruction process.
- Massive Parameter Efficiency: Because equivariant convolutions share weights across different orientations, ReDiffuse is incredibly lightweight. It reduces the parameter count from 26.91M (standard backbone) to just 7.55M, all while improving performance.
- Preserving Non-Local Similarity: By enforcing rotation equivariance, the model excels at restoring repeating patterns along curved paths and maintaining perfectly circular structures without introducing directional artifacts.
Key Results
ReDiffuse was tested against 11 state-of-the-art MFIF methods (including CNN, GAN, Transformer, and other diffusion-based networks):
- Quantitative Performance: ReDiffuse consistently achieved the best results across all four datasets, showing an average improvement of 0.28% to 6.64% across six key evaluation metrics (such as $Q_{abf}$, mutual information, and MS-SSIM).
- Qualitative Quality: Visually, the model significantly reduces edge distortion and artifacts, producing much sharper boundaries and cleaner textures around complex contours.
Limitations and Future Work
Despite its strengths, ReDiffuse can fail when encountering extreme defocus spread. If the blur is so severe that local texture structures are completely obliterated, the model's derivative estimates deviate from the true geometric patterns. This increases the equivariance error and can lead to incorrect focus area selection.
To address this, the authors suggest that future iterations should exploit non-local correlations across distant image regions, rather than relying solely on local features, to better constrain structural selection in heavily blurred zones.
Summary updated