Triplet-Watershed for Hyperspectral Image Classification

Aditya Challa, Sravan Danda, B. S. Daya Sagar, Laurent Najman

Abstract

Hyperspectral images (HSI) consist of rich spatial and spectral information, which can potentially be used for several applications. However, noise, band correlations and high dimensionality restrict the applicability of such data. This is recently addressed using creative deep learning network architectures such as ResNet, SSRN, and A2S2K. However, the last layer, i.e the classification layer, remains unchanged and is taken to be the softmax classifier. In this article, we propose to use a watershed classifier. Watershed classifier extends the watershed operator from Mathematical Morphology for classification. In its vanilla form, the watershed classifier does not have any trainable parameters. In this article, we propose a novel approach to train deep learning networks to obtain representations suitable for the watershed classifier. The watershed classifier exploits the connectivity patterns, a characteristic of HSI datasets, for better inference. We show that exploiting such characteristics allows the Triplet-Watershed to achieve state-of-art results in supervised and semi-supervised contexts. These results are validated on Indianpines (IP), University of Pavia (UP), Kennedy Space Center (KSC) and University of Houston (UH) datasets, relying on simple convnet architecture using a quarter of parameters compared to previous state-of-the-art networks.

Introduction

Hyperspectral imaging (HSI) captures hundreds of narrow spectral bands, providing rich data for tasks like land-cover classification. However, this high dimensionality comes with significant challenges, including noise, redundant information (band correlation), and the "curse of dimensionality." While modern deep learning architectures like ResNet and SSRN have improved HSI classification, they almost universally rely on the softmax classifier in their final layer. Softmax is a generic tool that doesn't inherently "understand" the spatial connectivity patterns typical of geographic data. To compensate, researchers often build increasingly complex, parameter-heavy networks.

This paper introduces Triplet-Watershed, a novel classification framework that replaces the standard softmax layer with a morphological watershed classifier. By using a classifier that naturally exploits spatial structure, the authors demonstrate that state-of-the-art results can be achieved with significantly simpler networks.

The Triplet-Watershed Framework

The proposed system consists of two primary components: a feature extractor and a watershed-based classifier, unified through a specialized training objective.

The Triplet-Watershed framework pipeline

Feature Extraction and Triplet Loss

A lightweight, 3-layer convolutional neural network (CNN) is used to extract representations (embeddings) from the HSI data. To ensure these embeddings are suitable for the watershed classifier, the network is trained using Triplet Loss. This loss function forces the CNN to map pixels of the same class closer together and different classes further apart in the embedding space. Crucially, it optimizes these distances specifically to suit the watershed's "flooding" logic.

Watershed Classifier

Instead of softmax, the framework uses a non-parametric classifier based on the watershed operator from Mathematical Morphology. Classification is treated as a flooding process on an edge-weighted graph where pixels are nodes and edge weights represent dissimilarity. The graph is constructed using local spatial connections (4-adjacency) and global connections (via Minimum Spanning Trees or K-Nearest Neighbors) to ensure both local and global consistency.

Graph-based watershed labeling process

A key insight is that the watershed operator on an edge-weighted graph provides a "maximum margin" decision boundary, conceptually similar to a Support Vector Machine (SVM) but natively built for graph-structured data. This inductive bias allows the model to "smooth" classification across related spatial regions.

Experimental Results

The model was evaluated on four standard HSI benchmarks: Indian Pines (IP), University of Pavia (UP), Kennedy Space Center (KSC), and University of Houston (UH).

Performance and Efficiency

Triplet-Watershed achieved state-of-the-art Overall Accuracy (OA) across the datasets:

  • Indian Pines (IP): 99.57%
  • University of Pavia (UP): 99.98%
  • Kennedy Space Center (KSC): 99.72%

Notably, the model achieves these results with only 87.6K parameters, roughly 25% of the parameters used by current state-of-the-art models like A2S2K (370.7K) and SSRN (364.1K). This efficiency stems from moving the "intelligence" of spatial grouping into the classifier itself rather than the feature extractor.

Representation Quality

Qualitative analysis using T-SNE plots showed that Triplet-Watershed produces more uniform and better-separated clusters than competing methods. Furthermore, the method excels in semi-supervised settings, outperforming specialized graph convolution networks (GCNs).

Conclusion

Triplet-Watershed demonstrates that incorporating morphological operators into deep learning pipelines can significantly improve performance and efficiency for spatially structured data like HSI. While the model is robust, its performance can dip at extremely low training sizes (e.g., 2%) if the graph becomes fragmented. Future work may explore different dissimilarity measures and graph construction techniques to further refine the "flooding" process.

The source code for reproducing the experiments is available at https://github.com/ac20/TripletWatershed.

Summary updated

Abstract

Hyperspectral images (HSI) consist of rich spatial and spectral information, which can potentially be used for several applications. However, noise, band correlations and high dimensionality restrict the applicability of such data. This is recently addressed using creative deep learning network architectures such as ResNet, SSRN, and A2S2K. However, the last layer, i.e the classification layer, remains unchanged and is taken to be the softmax classifier. In this article, we propose to use a watershed classifier. Watershed classifier extends the watershed operator from Mathematical Morphology for classification. In its vanilla form, the watershed classifier does not have any trainable parameters. In this article, we propose a novel approach to train deep learning networks to obtain representations suitable for the watershed classifier. The watershed classifier exploits the connectivity patterns, a characteristic of HSI datasets, for better inference. We show that exploiting such characteristics allows the Triplet-Watershed to achieve state-of-art results in supervised and semi-supervised contexts. These results are validated on Indianpines (IP), University of Pavia (UP), Kennedy Space Center (KSC) and University of Houston (UH) datasets, relying on simple convnet architecture using a quarter of parameters compared to previous state-of-the-art networks. The source code for reproducing the experiments and supplementary material (high resolution images) is available at https://github.com/ac20/TripletWatershed Code.
BibTeX
@article{challa2022triplet,
  author = {Aditya Challa and Sravan Danda and B. S. Daya Sagar and Laurent Najman},
  title = {{Triplet-Watershed for Hyperspectral Image Classification}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2022},
  url = {https://openreview.net/forum?id=2VOPhn6MCFd},
}
Figure