Abstract
Multifocus image fusion is an effective way to overcome the limitation of optical lenses. Many existing methods obtain fused results by generating decision maps. However, such methods often assume that the focused areas of the two source images are complementary, making it impossible to achieve simultaneous fusion of multiple images. Additionally, the existing methods ignore the impact of hard pixels on fusion performance, limiting the visual quality improvement of fusion image. To address these issues, a combining generation and recombination model, termed as GRFusion, is proposed. In GRFusion, focus property detection of each source image can be implemented independently, enabling simultaneous fusion of multiple source images and avoiding information loss caused by alternating fusion. This makes GRFusion free from the number of inputs. To distinguish the hard pixels from the source images, we achieve the determination of hard pixels by considering the inconsistency among the detection results of focus areas in source images. Furthermore, a multi-directional gradient embedding method for generating full focus images is proposed. Subsequently, a hard-pixel-guided recombination mechanism for constructing fused result is devised, effectively integrating the complementary advantages of feature reconstruction-based method and focused pixel recombination-based method. Extensive experimental results demonstrate the effectiveness and the superiority of the proposed method.
Introduction
Multifocus image fusion (MFIF) aims to synthesize a single "all-in-focus" image from multiple source images of the same scene captured with different focus settings. Despite significant progress, current MFIF algorithms face two primary challenges: the "two-input" constraint and the "hard pixel" dilemma.
Most existing methods are designed for only two source images. To handle more, they rely on "alternating fusion"—a sequential process where images are fused in pairs. This often leads to accumulated errors and information loss. Furthermore, pixels at the boundaries of focused regions or in textureless areas, known as "hard pixels," are notoriously difficult to classify. Recombination methods (which select pixels) often produce artifacts at these boundaries, while reconstruction methods (which generate pixels) may lose fine details.
The GRFusion Framework
To overcome these limitations, the proposed GRFusion framework introduces a parallel processing strategy that is independent of the number of input images. It combines the strengths of pixel selection for clear regions and deep-learning-based generation for ambiguous areas.
Core Components
The architecture is built around three main modules:
- Hard Pixel Detection (HPD): Instead of assuming complementarity, GRFusion detects the focus properties of each image independently. By identifying inconsistencies across these maps, it pinpoints "hard pixels" where focus is ambiguous across all inputs.
- Full-focus Image Generation (FFIG-MFE): For these difficult regions, the model generates a high-quality full-focus version. It utilizes a Multi-directional Feature Embedding (MFE) module with eight directional edge-extraction kernels to preserve structural integrity and fine details that standard convolutions might miss.
- Fusion Result Construction (FRC-G-HP): This module acts as a smart editor. It preserves original pixels from source images for clearly focused regions to maintain maximum sharpness, while substituting generated pixels from the FFIG-MFE module for identified hard pixels to ensure smooth transitions.
Experimental Results
GRFusion was trained on a synthetic dataset derived from ImageNet and evaluated on three benchmark datasets: Lytro, MFI-WHU, and MFFW.
Quantitative and Qualitative Performance
The model achieved state-of-the-art results, ranking first or second across five major metrics ($Q_{MI}, Q_{AB/F}, Q_{CB}, Q_{NCIE}, Q_{SSIM}$). Qualitatively, visual inspections and difference maps confirmed that GRFusion produces significantly fewer artifacts at object boundaries compared to existing methods like SwinFusion and SESF. Notably, its performance remained robust even as the number of input images increased (tested up to 6 images), avoiding the blurring and chromatic aberrations common in sequential fusion methods.
Conclusion
GRFusion represents a significant step forward in multifocus image fusion by breaking the two-input limitation and specifically addressing the hard pixel problem. By treating input images as independent sources and employing a hybrid generation-recombination strategy, it achieves superior visual clarity. Future research will look into reducing computational complexity and extending this framework to temporal fusion in video sequences.
Summary updated