Triplet-Watershed for Hyperspectral Image Classification

Aditya Challa, Sravan Danda, B. S. Daya Sagar, Laurent Najman

TL;DR

This paper introduces "Triplet-Watershed," a novel classification framework for hyperspectral images that replaces the standard softmax layer with a morphological watershed classifier. By training a lightweight CNN with triplet loss to produce watershed-friendly embeddings, the authors achieve state-of-the-art accuracy while using significantly fewer parameters than existing deep learning models.

What Problem Does This Paper Address?

Hyperspectral imaging (HSI) captures hundreds of narrow spectral bands, providing rich data for tasks like land-cover classification. However, this high dimensionality comes with challenges: noise, redundant information (band correlation), and the "curse of dimensionality."

While modern deep learning architectures like ResNet have improved HSI classification, they almost universally rely on the softmax classifier in their final layer. Softmax is a generic tool that doesn't inherently "understand" the spatial connectivity patterns typical of geographic data. To compensate, researchers often build increasingly complex, parameter-heavy networks. The authors of this paper ask: can we achieve better results with simpler networks by using a classifier that naturally exploits spatial structure?

Methods

The authors propose a two-part system called Triplet-Watershed:

  1. Feature Extraction: A simple, 3-layer convolutional neural network (CNN) extracts representations (embeddings) from the HSI data.
  2. Watershed Classifier: Instead of softmax, they use a non-parametric classifier based on the watershed operator from Mathematical Morphology. In this context, classification is treated like a flooding process on a graph where pixels are nodes and edge weights represent "dissimilarity."

To make these two parts work together, they use Triplet Loss during training. This loss function forces the CNN to map pixels of the same class closer together and different classes further apart in the embedding space. Crucially, it optimizes these distances specifically to suit the watershed's "flooding" logic. The graph used by the classifier is constructed using local spatial connections (4-adjacency) and global connections (via Minimum Spanning Trees or K-Nearest Neighbors) to ensure both local and global consistency.

Datasets

The model was evaluated on four standard HSI benchmarks:

  • Indian Pines (IP): 145x145 pixels, 224 bands, 16 classes.
  • University of Pavia (UP): 610x610 pixels, 103 bands, 9 classes.
  • Kennedy Space Center (KSC): 176 bands, 13 classes.
  • University of Houston (UH): 144 bands, 15 classes.

Key Insights and Ideas

  • Spatial Connectivity as a Constraint: The core "aha moment" is that enforcing spatial connectivity during training is a powerful inductive bias for HSI. Because the watershed classifier relies on paths through a graph, it naturally "smooths" classification across related spatial regions.
  • Maximum Margin Partitions: The authors demonstrate that the watershed operator on an edge-weighted graph provides a "maximum margin" decision boundary. This makes it conceptually similar to a Support Vector Machine (SVM), but one that is natively built for graph-structured data.
  • Parameter Efficiency: By moving the "intelligence" of the spatial grouping into the classifier itself, the feature-extracting CNN can be much smaller. The Triplet-Watershed uses roughly 25% of the parameters of current state-of-the-art models like A2S2K.

Key Results

  • Superior Accuracy: Triplet-Watershed achieved an Overall Accuracy (OA) of 99.57% on Indian Pines, 99.98% on University of Pavia, and 99.72% on KSC, outperforming established models like SSRN and A2S2K.
  • Lightweight Architecture: The model achieves these results with only 87.6K parameters, compared to 370.7K for A2S2K and 364.1K for SSRN.
  • Better Representations: Qualitative analysis using T-SNE plots showed that Triplet-Watershed produces more uniform and better-separated clusters than competing methods, which often result in "clumpy" distributions within classes.
  • Semi-Supervised Success: The method also excels in semi-supervised settings, outperforming specialized graph convolution networks (GCNs).

Limitations and Future Work

  • The Low-Data Regime: While the model is robust, its performance can dip at extremely low training sizes (e.g., only 2% of data). This happens if the graph becomes fragmented or if there isn't enough boundary information to separate classes effectively.
  • Graph Construction: The current model relies on specific graph construction techniques (EMST/K-NN). The authors suggest that exploring different dissimilarity measures beyond the "pass value" used in the watershed operator could further improve performance.
  • Computational Complexity: While the CNN is small, propagating labels through the watershed classifier involves graph operations. Although these are performed in quasi-linear time, they represent a different computational profile than standard matrix-heavy deep learning.

Summary updated

Abstract

Hyperspectral images (HSIs) consist of rich spatial and spectral information, which can potentially be used for several applications. However, noise, band correlations, and high dimensionality restrict the applicability of such data. This is recently addressed using creative deep learning network architectures, such as ResNet, spectral-spatial residual network (SSRN), and attention-based adaptive spectral-spatial kernel residual networks (A2S2K). However, the last layer, i.e., the classification layer, remains unchanged and is taken to be the softmax classifier. In this article, we propose to use a watershed classifier. Watershed classifier extends the watershed operator from Mathematical Morphology for classification. In its vanilla form, the watershed classifier does not have any trainable parameters. In this article, we propose a novel approach to train deep learning networks to obtain representations suitable for the watershed classifier. The watershed classifier exploits the connectivity patterns, a characteristic of HSI datasets, for better inference. We show that exploiting such characteristics allows the Triplet-Watershed to achieve state-of-art results in supervised and semi-supervised contexts. These results are validated on Indian Pines (IP), University of Pavia (UP), Kennedy Space Center (KSC), and University of Houston (UH) datasets, relying on simple convnet architecture using a quarter of parameters compared to previous state-of-the-art networks. The source code for reproducing the experiments and supplementary material (high-resolution images) is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ac20/TripletWatershed_Code</uri> .
BibTeX
@article{challa2022triplet,
  author = {Aditya Challa and Sravan Danda and B. S. Daya Sagar and Laurent Najman},
  title = {{Triplet-Watershed for Hyperspectral Image Classification}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2022},
  url = {https://openreview.net/forum?id=2VOPhn6MCFd},
}

Loading related content...

Figure