Pixel-Wise Shuffling with Collaborative Sparsity for Melanoma Hyperspectral Image Classification

Yongsheng Gao, Favour Ekong, Jun Zhou, Kwabena Sarpong, Yongsheng Gao 0001WACV 2025 · DOI

Introduction

Hyperspectral imaging (HSI) captures a dense "spectral fingerprint" of skin tissue across hundreds of narrow wavelength bands, offering a non-invasive potential for melanoma diagnosis. However, clinical application is hindered by the high dimensionality of HSI data, which often leads to overfitting, and the significant spectral overlap between malignant melanomas and benign dysplastic nevi. Traditional deep learning models frequently struggle to integrate fine-grained spatial context with complex spectral signatures effectively.

This research proposes a novel framework that addresses these challenges by combining physical tissue modeling with advanced transformer architectures. The approach utilizes collaborative sparse unmixing to decompose pixels into biological components followed by a pixel-wise shuffling transformer to capture intricate spatial-spectral relationships.

Methodology

The proposed architecture is divided into two primary stages: collaborative sparse unmixing and a transformer-based classification module featuring pixel-wise shuffling.

Collaborative Sparse Unmixing (CSU)

Rather than processing raw spectral data, the model first "unmixes" each pixel into its constituent "endmembers"—pure spectral signatures representing substances like melanin and hemoglobin. By employing a collaborative sparsity constraint, the model leverages the spatial correlation between neighboring pixels, which likely share similar material compositions. This step reduces noise and provides a physically interpretable representation of the skin's biological makeup through abundance maps.

Pixel-wise Shuffling Transformer

The abundance maps are processed by a transformer module designed to maintain high-resolution spatial information. A core innovation is the Spectral-Spatial Pixel-wise Shuffling Window Block, which shuffles and interpolates features to minimize information loss common in standard sliding-window approaches. Furthermore, an Inter-similarity Hybrid Attention mechanism uses cosine similarity to compare query and key features, mimicking a dermatologist's focus on texture, shape, and contrast variations across different spectral bands.

Figure 3: Detailed architecture of the proposed framework, highlighting the Spectral-Spatial Pixel-wise Shuffling Window Block and the integration of collaborative sparsity.

Experimental Results

The model was evaluated using a histology-verified hyperspectral skin dataset containing 338 dermoscopy images (85 Melanomas, 185 Dysplastic Nevi, and 78 other lesions) across 16 spectral bands (465–630 nm). To test robustness in data-scarce medical environments, the authors limited training to only 20% of the available data.

As shown in the comparative analysis, the proposed method achieved state-of-the-art performance, reaching an Overall Accuracy (OA) of 73.34% and a Sensitivity (SEN) of 66.40%. It outperformed 14 other competitive methods, including specialized Vision Transformers like SpectralFormer and large-scale architectures, while maintaining a relatively efficient parameter count.

Figure 5: Performance comparison of the proposed method against 14 state-of-the-art models across multiple metrics including Overall Accuracy (OA), Average Accuracy (AA), and Kappa coefficient.

Discussion and Conclusion

The success of this framework highlights two critical insights for medical HSI:

  1. Physical Priors: Preceding deep learning with physical modeling (unmixing) provides a "biological head start" that improves classification accuracy and interpretability.
  2. Architectural Efficiency: Domain-specific innovations like pixel-wise shuffling are more effective for specialized medical tasks than simply increasing model scale.

Future work will explore expanding the spectral range into the infrared spectrum and integrating multimodal data, such as clinical metadata, to further enhance diagnostic reliability across diverse clinical settings.

Summary updated

BibTeX
@inproceedings{gao2025pixel,
  author = {Yongsheng Gao and Favour Ekong and Jun Zhou and Kwabena Sarpong and Yongsheng Gao},
  title = {{Pixel-Wise Shuffling with Collaborative Sparsity for Melanoma Hyperspectral Image Classification}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00632},
}
Figure