Region based image fusion using SVM

Yang Liu, Jian Cheng, Hanqing LuProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2007 · DOI

TL;DR

This paper introduces a novel image fusion technique that combines multiscale decomposition (MSD) with region segmentation and Support Vector Machines (SVMs). By segmenting images into homogeneous regions and using SVMs to intelligently select fusion rules for each region, the approach achieves superior fusion results compared to traditional methods, producing high-resolution multispectral images.

What Problem Does This Paper Address?

The core challenge addressed is the creation of high-resolution multispectral images from lower-resolution multispectral (MS) and high-resolution panchromatic (Pan) images. Traditional fusion methods often struggle with spectral distortion and loss of spatial detail. While Principal Component Analysis (PCA) and multiscale decomposition (MSD) have improved fusion, a key limitation remains: applying a single fusion rule across the entire image, regardless of its diverse content (e.g., vegetation, urban areas, water bodies). This can lead to suboptimal results because different regions have distinct characteristics that might benefit from different fusion strategies. The paper aims to overcome this by adapting fusion rules to specific image regions.

Methods

The proposed method builds upon existing techniques but introduces a novel combination:

  • PCA Merger with Multiscale Decomposition (MSD): This is a standard approach to extract spatial details from the Pan image and inject them into the multispectral data. MSD helps preserve both high spatial and spectral information.
  • Region Segmentation: Instead of fusing the entire image at once, the authors segment the MS image into homogeneous regions based on edge information and a labeling algorithm. This is a key innovation, recognizing that different regions (e.g., vegetation, buildings) have distinct features and should be treated separately.
  • Support Vector Machine (SVM) for Fusion Rule Selection: This is the most novel aspect. After MSD, for each region, the SVM is trained to act as a classifier. It takes the difference between eigenvectors from the Pan and PC¹ components as input. Based on this input, the SVM determines which fusion rule (or combination of rules) is most appropriate for that specific region, effectively creating a dynamic, region-aware fusion strategy. This avoids the drawbacks of relying on a single, fixed fusion rule.
  • Wavelet Decomposition and Reconstruction: Wavelets are used for decomposition and reconstruction, a common practice in multiscale analysis for image fusion.

Datasets

The experiment uses IKONOS-2 imagery:

  • A 1-meter resolution panchromatic (Pan) band.
  • A 4-meter resolution multispectral (MS) image (specifically, the red band is mentioned for the MS image, but the context implies multiple bands are used for fusion). The images cover an area of Sherbrooke, QC, Canada, captured on May 20, 2001. The dataset is described as geometrically registered.

Key Insights and Ideas

  • Region-based fusion is crucial for diverse scenes: Treating homogeneous regions differently from the whole image significantly improves fusion quality because it respects the inherent characteristics of different land cover types.
  • SVMs as intelligent fusion rule selectors: Using SVMs to dynamically choose fusion rules based on local image content is a powerful way to overcome the limitations of single-rule fusion methods and adapt to varying conditions.
  • Balancing spectral and spatial information: The method aims to achieve a better balance between preserving the spectral integrity of the MS image and enhancing the spatial resolution from the Pan image, a common trade-off in fusion.
  • SVM classification performance: The SVM classifiers trained for different regions (vegetation, man-made objects, earth) show high precision and recall, indicating their effectiveness in distinguishing features relevant to fusion rule selection.

Key Results

  • Superior Fusion Quality: The proposed region-based SVM fusion approach consistently outperformed several traditional fusion methods (IHS, PCA, HPF, Brovey, Wavelet) across multiple evaluation metrics.
  • Quantitative Improvements:
    • RMSE: Achieved the lowest RMSE (0.1208), indicating less error in the fused image.
  • SAM: Achieved the lowest SAM (1.2°), suggesting better preservation of spectral angles.
  • ERGAS: Achieved a competitive ERGAS value (1.66).
  • Q4: Showed the highest average Q4 score (0.8028), indicating good overall quality.
  • Correlation Coefficient (CC): While the proposed method's CC (0.8788) was not the absolute highest (IHS and PCA were higher), the paper explains this is because those methods directly replace components, leading to higher correlation but also spectral distortion. The proposed method's CC is still strong and achieved without significant spectral distortion.
  • Qualitative Results: Figure 5 shows subscenes of fused images, visually demonstrating that the proposed method produces a fused image with good spatial detail and spectral fidelity.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the multi-step process involving MSD, segmentation, SVM training, and fusion for each region likely incurs higher computational cost compared to simpler methods.
  • Dependence on Segmentation Quality: The performance of the fusion is inherently tied to the accuracy and effectiveness of the region segmentation step.
  • Future Work: The paper suggests that this region-based SVM approach is a significant improvement and can be applied in various conditions, implying potential for further refinement and application to different types of remote sensing data. The authors also note that their approach avoids spectral distortion and loss of spatial information, which are common drawbacks in other methods.

Summary updated

Abstract

This paper presents a novel fusion approach using PCA merger based on multiscale decomposition (MSD), combined with region segmentation and support vector machine (SVM), the result is a high spatial resolution multispectral image from a high resolution panchromatic (Pan) image and low resolution multispectral (Ms) images. Principal components analysis (PCA) fusion technique is one of typical fusion methods, and PCA merger based on MSD had been proposed which can obtain better performance. As we know that, in pixel fusion level, the original images are fused as internal region regardless of the contents of images, but in this paper, we perform region segmentation after MSD, because the homogeneous regions have similar features such as color, texture and intensity. Traditionally, various fusion rules can be applied after MSD according to different conditions, however, the crucial problem is which fusion rule should be adopted under given condition, hence we use the SVM to combine the most fusion rules so that can avoid some drawbacks using single fusion rule. To validate our approach, we compare it with several typical fusion approaches, and the best result is obtained using our approach.
BibTeX
@inproceedings{liu2007region,
  author = {Yang Liu and Jian Cheng and Hanqing Lu},
  title = {{Region based image fusion using SVM}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2007},
  doi = {10.1117/12.749598},
}
Figure