Multi-focus Image Fusion Based on Similarity Characteristics

Ya-Qiong Zhang, Xiao-Jun Wu, Hui LiarXiv 2019 · arXiv

Abstract

This paper presents a novel spatial-domain algorithm for multi-focus image fusion designed to overcome the limitations of standard Structural Similarity (SSIM) metrics. The proposed method introduces Signed Structural Non-Similarity (SSNSIM), which incorporates local variance to distinguish focus directionality. By integrating watershed segmentation with Fuzzy C-Means (FCM) clustering and a joint-region strategy for multi-source scenarios, the algorithm accurately identifies in-focus regions. Experimental results demonstrate that this approach yields sharper composites with fewer boundary artifacts compared to traditional SSIM-based methods, as evidenced by improvements in average gradient and edge retention metrics.

Introduction

Optical imaging systems frequently encounter the challenge of limited depth-of-field, where objects at varying distances cannot be simultaneously captured in sharp focus. Multi-focus image fusion addresses this by merging several source images of the same scene, each focused at different depths, into a single "all-in-focus" image.

While spatial-domain region-based fusion is a common strategy, its performance hinges on the accuracy of segmenting in-focus from out-of-focus zones. Traditional metrics like the Structural Similarity (SSIM) index are often insufficient because they measure the magnitude of difference without indicating which image is sharper. This lack of directionality frequently leads to unreliable segmentation and visible artifacts at the boundaries of fused regions. This paper proposes a new metric, Signed Structural Non-Similarity (SSNSIM), to provide the necessary discriminative power for robust segmentation.

Methodology

The proposed fusion pipeline, termed NSIWF (Non-Similarity Watershed Fusion), follows a multi-stage process to ensure precise region identification and merging.

Figure 1: Flowchart of the proposed NSIWF fusion method for two images.

Signed Structural Non-Similarity (SSNSIM)

The core innovation is the SSNSIM metric. Instead of a standard similarity measure, the authors calculate non-similarity ($1 - SSIM$) and assign a sign based on local standard deviation, which serves as a proxy for image sharpness. This transformation creates a map where different focus regions are clearly separated by sign, facilitating more accurate downstream segmentation.

Region Segmentation and Merging

To define the focus regions, a morphological H-minima transform and watershed algorithm are applied to the gradient map of the SSNSIM. To mitigate the watershed algorithm's tendency toward over-segmentation, Fuzzy C-Means (FCM) clustering is employed. FCM groups fragmented pieces based on mean SSNSIM values into homogeneous, semantically meaningful focus areas.

Joint Region Segmentation

For scenarios involving three or more source images, the authors implement a joint segmentation strategy. This method mathematically harmonizes pairwise comparisons (e.g., X-Y, X-Z, Y-Z) to resolve overlapping or disjoint boundaries, ensuring a unified label map for the final fusion step.

Results and Discussion

The NSIWF method was evaluated against existing techniques such as RSIM and RSSF using a variety of objective metrics, including Average Gradient (AG), Spatial Frequency (SF), and Edge Retention ($Q^{AB/F}$).

Figure 3: A segmentation map generated by the proposed method, showing clear separation of focus regions.

Quantitative Performance

In two-image fusion tests, the proposed method significantly outperformed RSIM. For instance, on standard test pairs, the Spatial Frequency improved from 23.58 to 27.04, and the Average Gradient increased from 12.03 to 13.87. In three-image fusion, the joint-segmentation approach achieved higher Edge Retention (0.5134) compared to RSSF (0.4929).

Qualitative Insights

Visually, the SSNSIM-based segmentation maps (as seen in Figure 3) provide a clean separation between foreground and background objects. This precision eliminates the "fuzzy" boundary artifacts typical of methods that struggle with focus directionality. The use of FCM clustering effectively groups over-segmented watershed regions into coherent objects, enhancing the overall visual naturalness of the fused image.

Conclusion

The integration of directional similarity (SSNSIM) and robust segmentation (Watershed + FCM) provides a powerful framework for multi-focus image fusion. While the current joint segmentation logic for multiple sources is rule-based and may face scalability challenges, it offers a significant improvement over traditional spatial-domain methods. Future work may involve optimizing these custom metrics for integration into deep learning architectures to further enhance performance in high-noise environments.

Summary updated

BibTeX
@misc{zhang2019multi,
  author = {Ya-Qiong Zhang and Xiao-Jun Wu and Hui Li},
  title = {{Multi-focus Image Fusion Based on Similarity Characteristics}},
  year = {2019},
  eprint = {1912.07959},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.07959},
}
Figure