Fusion of Image Segmentation Algorithms using Consensus Clustering

Mete Ozay, Fatos T. Yarman-Vural, Sanjeev R. Kulkarni, H. Vincent Poor

Abstract

In hyperspectral remote sensing, selecting a single optimal segmentation algorithm is challenging due to varying spectral responses across different bands. This paper introduces a robust framework for segmentation fusion, which combines multiple algorithm outputs into a single consensus result. By utilizing a modified Filtered Stochastic BOEM optimization and incorporating automated distance learning (DL and QD), the proposed method effectively merges disparate segmentations. Experimental results on benchmark hyperspectral datasets demonstrate that this consensus approach consistently outperforms individual base algorithms.

Introduction

Hyperspectral imaging captures data across numerous spectral bands, each revealing different physical features. For instance, near-infrared bands are effective for identifying water bodies, while visible bands better distinguish urban or agricultural features. Consequently, no single segmentation algorithm—or even a single set of parameters—is universally superior. Common issues include over-segmentation and under-segmentation. This study addresses the "segmentation fusion problem," seeking a consensus that leverages the strengths of a "committee" of diverse segmentation outputs.

Methodology

The proposed fusion framework is built upon Consensus Clustering, specifically addressing the Median Partition Problem. The goal is to find a consensus segmentation that minimizes the total distance to all candidate segmentations in an ensemble.

Filtered Stochastic BOEM

The authors employ a stochastic optimization technique called Filtered Stochastic BOEM (Best One Element Move). Instead of exhaustive searching, the algorithm iteratively selects a candidate segmentation and performs a "best one-element move"—reassigning a single pixel's label—to improve the global objective function. A learning rate parameter, $\beta$, acts as a memory mechanism, allowing the algorithm to converge on high-quality solutions by gradually "forgetting" poor initial states.

Distance Learning (DL and QD)

To compare segmentations with varying numbers of clusters, the authors introduce two distance learning approaches:

  1. Distance Learning (DL): Utilizes the Adjusted Rand Index (ARI) to learn and normalize distances between different segmentation partitions.
  2. Quasi-distance Learning (QD): A normalization method that accounts for the theoretical minimum and maximum possible distances between specific pairs of segmentations.

Automated Parameter Tuning

The framework includes automated estimation for the optimal number of clusters ($C$) and the learning rate ($\beta$) using a "Segmentation Index" (SI) and "Beta Index" (BI), respectively. This eliminates the need for manual parameter tuning.

Experimental Results

The methodology was evaluated using two benchmark hyperspectral datasets: a 7-band Thematic Mapper image and a 12-band Moderate Dimension image (Flightline C1).

Performance on 7-band Thematic Mapper

On the 7-band dataset, the fusion process showed clear improvements over the base algorithms. The DL-enhanced fusion reached an ARI of 0.184, significantly higher than the average base ARI of 0.159.

Table 1: Performance comparison (RI and ARI) on the 7-band Thematic Mapper dataset.

Performance on 12-band Moderate Dimension

Similar trends were observed in the 12-band dataset. While the average base ARI was 0.009, the DL-enhanced fusion improved this metric to 0.011.

Table 2: Performance comparison (RI and ARI) on the 12-band Moderate Dimension dataset.

Discussion and Conclusion

The results indicate that a "committee" of algorithms generally provides a more reliable segmentation than any single "expert" algorithm. The ARI-based distance learning allows for flexible comparison across segmentations with different cluster counts. Qualitatively, the fusion algorithm successfully finds a "compromise" between over-segmented and under-segmented results, leading to a more accurate representation of ground truth.

Future work will focus on reducing computational complexity for high-resolution imagery and exploring the integration of deep-learning-based segmentors into the consensus ensemble.

Summary updated

Abstract

A new segmentation fusion method is proposed that ensembles the output of several segmentation algorithms applied on a remotely sensed image. The candidate segmentation sets are processed to achieve a consensus segmentation using a stochastic optimization algorithm based on the Filtered Stochastic BOEM (Best One Element Move) method. For this purpose, Filtered Stochastic BOEM is reformulated as a segmentation fusion problem by designing a new distance learning approach. The proposed algorithm also embeds the computation of the optimum number of clusters into the segmentation fusion problem.
BibTeX
@inproceedings{ozay2013fusion,
  author = {Mete Ozay and Fatos T. Yarman-Vural and Sanjeev R. Kulkarni and H. Vincent Poor},
  title = {{Fusion of Image Segmentation Algorithms using Consensus Clustering}},
  booktitle = {ICIP},
  year = {2013},
  doi = {10.1109/icip.2013.6738834},
}
Figure