Multifocus image fusion using phase congruency

Kun Zhan, Qiaoqiao Li, Jicai Teng, Mingying Wang, Jinhui ShiJ. Electronic Imaging 2015 · DOI

TL;DR

This paper introduces a novel multifocus image fusion method that leverages phase congruency (PC) as a robust focus measure. By effectively identifying sharp features across multiple images, their approach generates fused images with enhanced detail and clarity, outperforming existing state-of-the-art methods in both visual quality and quantitative evaluations.

What Problem Does This Paper Address?

The core challenge in multifocus image fusion is to combine information from multiple images, each focused on a different depth of field, to create a single image with an extended depth of field. This requires accurately identifying which parts of each input image are in sharpest focus and then seamlessly merging these focused regions. Traditional methods often struggle with noise sensitivity, computational complexity, or the preservation of fine details. This paper aims to address these limitations by proposing a fusion method that uses a more robust focus measure and an effective fusion rule.

Methods

The proposed method centers around two key components: a novel focus measure and a fusion rule.

  • Focus Measure (FM): The authors employ phase congruency (PC) as their primary focus measure. PC is a feature detection method that identifies points in an image where the phase of different frequency components aligns, indicating significant image features like edges and corners. A key advantage of PC is its robustness against noise, which is a common issue in image processing. The paper details an "improved PC" that utilizes log-Gabor wavelet filters across multiple scales and orientations to capture these phase congruencies more effectively.
  • Fusion Rule (FR): The fusion rule is designed to select the most focused region from the input images based on the PC values. This involves:
  • Calculating the PC for each input image.
  • Creating a binary decision map where each pixel is assigned '1' if the PC value from one image is greater than the other, indicating a more focused region.
  • Applying a majority filter to this decision map. This filter smooths the map by considering a local neighborhood, ensuring that small, isolated decisions are corrected and that larger, consistent focused regions are preserved. This step is crucial for handling the fact that focused objects are represented by a series of neighboring pixels, not just a single one.
  • The decision map is then expanded using a Kronecker product to align with the original image resolution.
  • Finally, the fused image is constructed by selecting pixels from the input images based on the expanded decision map.

Datasets

The authors evaluated their method using four sets of multifocus image pairs:

  • "lab" images
  • "clock" images
  • "disk" images
  • "Pepsi" images

These image sets were used for both qualitative visual comparisons and quantitative performance evaluations. No new benchmark datasets were introduced.

Key Insights and Ideas

  • Phase Congruency as a Superior Focus Measure: The paper strongly advocates for phase congruency, demonstrating its superior robustness to noise compared to other common focus measures (e.g., energy of image gradient, sum-modified Laplacian, spatial frequency). This is a critical insight, as noise can significantly degrade the performance of fusion algorithms.
  • Spatial Domain Fusion with Multiscale Features: The method cleverly combines the advantages of spatial domain fusion (preserving original pixel values) with multiscale analysis (capturing details at different resolutions) by using PC derived from multiscale log-Gabor filters.
  • Majority Filter for Robust Decision Making: The use of a majority filter on the binary decision map is a key innovation for ensuring that the fusion process is consistent and avoids artifacts. It effectively "spreads" the focused regions to their neighbors, leading to more coherent fused outputs.
  • Kronecker Product for Decision Map Expansion: The application of the Kronecker product is an elegant way to expand the block-based decision map to the full image resolution, ensuring accurate pixel-level selection.

Key Results

  • Noise Robustness: Figure 5 clearly shows that the phase congruency (PC) focus measure remains stable even with increasing levels of Gaussian noise, while other tested focus measures degrade significantly.
  • Visual Quality: Figures 6-9 and 11-14 demonstrate that the proposed phase congruency-based fusion (PCF) method produces visually superior fused images compared to methods like LAP, SIW, PCNN, GIF, IMT, and MWG. The PCF results exhibit sharper details and fewer artifacts (e.g., ringing effects, blurred regions) than many of the compared methods.
  • Quantitative Evaluation: Figure 10 and Table 1 show that PCF achieves high scores across various objective performance metrics (Qablf, Qyf, Qp), often outperforming or matching the best-performing methods.
  • Computational Efficiency: Table 2 indicates that while PCF is not the fastest method, it is significantly faster than PCNN, IMT, and MWG, and is comparable to LAP, SIW, and GIF in terms of running time.

Limitations and Future Work

  • Computational Complexity: Although PCF is faster than some methods, it is noted that its computational complexity is higher than LAP, SIW, and GIF. The paper mentions that the MWG algorithm has particularly high computational complexity due to multiresolution analysis and eigenvalue calculations.
  • Parameter Tuning: While not explicitly detailed as a limitation, the effectiveness of the PC and the majority filter relies on appropriate parameter settings (e.g., for the log-Gabor filters, the majority filter window size). Future work could explore adaptive parameter selection.
  • Further Exploration of Phase Structure: The authors suggest that while the phase structure of natural images is known to be perceptually important, further research could delve deeper into its applications beyond focus detection.

Summary updated

BibTeX
@article{zhan2015multifocus,
  author = {Kun Zhan and Qiaoqiao Li and Jicai Teng and Mingying Wang and Jinhui Shi},
  title = {{Multifocus image fusion using phase congruency}},
  journal = {J. Electronic Imaging},
  year = {2015},
  doi = {10.1117/1.jei.24.3.033014},
}
Figure