Introduction
Hyperspectral imaging (HSI) captures a dense "spectral fingerprint" of skin tissue across hundreds of narrow wavelength bands, offering a non-invasive potential for melanoma diagnosis. However, clinical application is hindered by the high dimensionality of HSI data, which often leads to overfitting, and the significant spectral overlap between malignant melanomas and benign dysplastic nevi. Traditional deep learning models frequently struggle to integrate fine-grained spatial context with complex spectral signatures effectively.
This research proposes a novel framework that addresses these challenges by combining physical tissue modeling with advanced transformer architectures. The approach utilizes collaborative sparse unmixing to decompose pixels into biological components followed by a pixel-wise shuffling transformer to capture intricate spatial-spectral relationships.
Methodology
The proposed architecture is divided into two primary stages: collaborative sparse unmixing and a transformer-based classification module featuring pixel-wise shuffling.
Collaborative Sparse Unmixing (CSU)
Rather than processing raw spectral data, the model first "unmixes" each pixel into its constituent "endmembers"—pure spectral signatures representing substances like melanin and hemoglobin. By employing a collaborative sparsity constraint, the model leverages the spatial correlation between neighboring pixels, which likely share similar material compositions. This step reduces noise and provides a physically interpretable representation of the skin's biological makeup through abundance maps.
Pixel-wise Shuffling Transformer
The abundance maps are processed by a transformer module designed to maintain high-resolution spatial information. A core innovation is the Spectral-Spatial Pixel-wise Shuffling Window Block, which shuffles and interpolates features to minimize information loss common in standard sliding-window approaches. Furthermore, an Inter-similarity Hybrid Attention mechanism uses cosine similarity to compare query and key features, mimicking a dermatologist's focus on texture, shape, and contrast variations across different spectral bands.
Experimental Results
The model was evaluated using a histology-verified hyperspectral skin dataset containing 338 dermoscopy images (85 Melanomas, 185 Dysplastic Nevi, and 78 other lesions) across 16 spectral bands (465–630 nm). To test robustness in data-scarce medical environments, the authors limited training to only 20% of the available data.
As shown in the comparative analysis, the proposed method achieved state-of-the-art performance, reaching an Overall Accuracy (OA) of 73.34% and a Sensitivity (SEN) of 66.40%. It outperformed 14 other competitive methods, including specialized Vision Transformers like SpectralFormer and large-scale architectures, while maintaining a relatively efficient parameter count.
Discussion and Conclusion
The success of this framework highlights two critical insights for medical HSI:
- Physical Priors: Preceding deep learning with physical modeling (unmixing) provides a "biological head start" that improves classification accuracy and interpretability.
- Architectural Efficiency: Domain-specific innovations like pixel-wise shuffling are more effective for specialized medical tasks than simply increasing model scale.
Future work will explore expanding the spectral range into the infrared spectrum and integrating multimodal data, such as clinical metadata, to further enhance diagnostic reliability across diverse clinical settings.
Summary updated