Multi-Scale U-Shape MLP for Hyperspectral Image Classification

Moule Lin, Weipeng Jing, Donglin Di, Guangsheng Chen, Houbing Song

Abstract

Hyperspectral images (HSIs) provide rich semantic and spatial information through numerous spectral bands, yet identifying pixels accurately remains challenging due to the need to correlate local and global information while managing model complexity. We propose the Multi-Scale U-shape Multi-Layer Perceptron (MUMLP), a lightweight architecture consisting of a Multi-Scale Channel (MSC) block and a U-shape Multi-Layer Perceptron (UMLP) structure. The MSC block embeds deep-level representations by mixing spectral band features, while the UMLP uses an encoder-decoder structure to compress parameters significantly. Extensive experiments on the Pavia University, Houston 2013, and Houston 2018 datasets demonstrate that MUMLP outperforms state-of-the-art methods in both accuracy and efficiency.

Introduction

Hyperspectral imaging (HSI) captures a wide range of spectral signatures, enabling precise material identification. However, the high dimensionality of HSI data poses significant challenges. Traditional Convolutional Neural Networks (CNNs) often require millions of parameters, leading to high computational costs that hinder real-time deployment. While MLP-based architectures offer a faster alternative, they frequently lack the expressive power to capture deep semantic relationships between neighboring pixels. MUMLP addresses these hurdles by providing a parameter-efficient model that maintains high classification accuracy.

Methodology

The MUMLP architecture replaces heavy convolutional layers with an efficient MLP-based pipeline. It is composed of two primary modules: the MSC block and the UMLP structure.

The MUMLP architecture, featuring the Multi-Scale Channel (MSC) blocks and the U-shape Multi-Layer Perceptron (UMLP) structure.

Multi-Scale Channel (MSC) Block

The MSC block serves as the feature extractor. It utilizes $1 \times 1$ convolutions to transform the channel dimension into a unified $2^n$ space, allowing the model to handle diverse datasets consistently. By mixing spectral features, it creates deep-level representations without the information bottlenecks common in simpler models.

U-shape Multi-Layer Perceptron (UMLP) Structure

Inspired by the U-Net architecture, the UMLP employs an encoder-decoder structure. It progressively compresses features to capture high-level semantics and then expands them, using skip connections to fuse these semantic details with low-level spatial information. This pixel-wise classification approach allows the model to remain slim while effectively understanding spatial context.

Experimental Results

The model was evaluated on three public benchmarks: Pavia University (103 bands), Houston 2013 (144 bands), and Houston 2018 (48 bands).

Efficiency comparison of MUMLP against other models, showing its superior trade-off between model size (parameters) and inference time.

MUMLP demonstrated superior performance across all datasets. On the Houston 2018 dataset, it achieved an Overall Accuracy (OA) of 89.64%, significantly outperforming standard MLP-Mixers (88.94%) and U-Net (61.05%). Most notably, MUMLP achieves these results with only 0.817M parameters, making it 20 to 50 times smaller than many competing CNN-based models. Statistical significance was confirmed via p-value testing ($p < 0.05$).

Conclusion

MUMLP proves that MLPs, when structured with a U-shape and multi-scale channel mixing, can match or exceed CNN performance in HSI classification while being far more efficient. Its extreme parameter efficiency makes it an ideal candidate for edge devices and satellite-based remote sensing. Future work will focus on testing the model's robustness against noisy sensor data and evaluating its performance on embedded hardware like NVIDIA Jetson or FPGAs.

Summary updated

Abstract

Hyperspectral images have significant applications in various domains, since they register numerous semantic and spatial information in the spectral band with spatial variability of spectral signatures. Two critical challenges in identifying pixels of the hyperspectral image are respectively representing the correlated information among the local and global, as well as the abundant parameters of the model. To tackle this challenge, we propose a Multi-Scale U-shape Multi-Layer Perceptron (MUMLP) a model consisting of the designed MSC (Multi-Scale Channel) block and the UMLP (U-shape Multi-Layer Perceptron) structure. MSC transforms the channel dimension and mixes spectral band feature to embed the deep-level representation adequately. UMLP is designed by the encoder-decoder structure with multi-layer perceptron layers, which is capable of compressing the large-scale parameters. Extensive experiments are conducted to demonstrate our model can outperform state-of-the-art methods across-the-board on three wide-adopted public datasets, namely Pavia University, Houston 2013 and Houston 2018
BibTeX
@article{lin2022multi,
  author = {Moule Lin and Weipeng Jing and Donglin Di and Guangsheng Chen and Houbing Song},
  title = {{Multi-Scale U-Shape MLP for Hyperspectral Image Classification}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2022},
  doi = {10.1109/lgrs.2022.3141547},
}
Figure