Exploring an application-oriented land-based hyperspectral target detection framework based on 3D–2D CNN and transfer learning

Jiale Zhao, Guanglong Wang, Bing Zhou, Jiaju Ying, Jie LiuEURASIP Journal on Advances in Signal Processing 2024 · DOI

Abstract

Hyperspectral target detection in land-based scenarios faces significant challenges due to spectral uncertainty caused by varying environmental conditions. This paper proposes a robust framework combining a hybrid 3D–2D Convolutional Neural Network (CNN) with transfer learning and multi-scale spectral similarity measures. By transforming raw spectral data into similarity feature maps, the model achieves high stability and accuracy, reaching an Area Under the Curve (AUC) of over 0.99 on both remote sensing and custom land-based datasets.

Introduction

Hyperspectral imaging (HSI) provides detailed spectral "fingerprints" for pixel-level identification. However, land-based imaging is plagued by "spectral uncertainty"—where an object's signature shifts due to solar angles, shadows, and atmospheric changes. Furthermore, traditional deep learning models often lack universality, failing when applied to new environments. This research introduces a framework designed to overcome these hurdles through invariant feature learning and transferability.

Methodology

The framework employs a three-stage approach to ensure robustness:

  1. Multi-scale Preprocessing: Raw data is converted into similarity feature maps using three metrics: Spectral Angular Similarity (SAM), Normalized Euclidean Distance (NED), and Spectral Correlation Coefficient (CC). This provides the network with a structured representation of target-to-background similarity.
  2. Hybrid 3D–2D CNN: Drawing inspiration from HybridSN, the architecture utilizes 3D convolutional layers to capture joint spatial-spectral features, followed by 2D layers for spatial refinement. This hybrid approach balances computational efficiency with the need for deep spectral analysis.
  3. Transfer Learning: To ensure universality, the model is trained on a source dataset and migrated to a target dataset with different environmental conditions. This forces the network to learn fundamental target characteristics rather than image-specific noise.

Experimental Results

The framework was validated using the public San Diego remote sensing dataset and a custom land-based dataset captured in Shijiazhuang, China. The land-based data, featuring aircraft models in complex near-ground settings, proved particularly challenging for traditional algorithms.

Detection visualization in the land-based dataset (d)

The proposed method achieved an AUC of 0.9958 on the San Diego data and 0.9912 on the land-based data. In contrast, traditional methods like Orthogonal Subspace Projection (OSP) saw performance drop significantly in land-based scenarios (AUC ~0.76), highlighting the necessity of the proposed deep learning approach.

Detection visualization in the land-based dataset (f)

Discussion and Conclusion

The results demonstrate that using similarity measures as a "metamodel" input allows the CNN to effectively distinguish targets despite environmental variability. While the framework offers superior stability and accuracy, it requires higher computational resources, with processing times ranging from 2.6 to 7.9 seconds per image. Future work will focus on optimizing the architecture for real-time deployment and reducing false alarm rates in high-shadow environments.

In conclusion, the integration of 3D–2D CNNs with transfer learning provides a powerful solution for practical, land-based hyperspectral target detection, offering a significant leap over traditional linear detection methods.

Summary updated

BibTeX
@article{zhao2024exploring,
  author = {Jiale Zhao and Guanglong Wang and Bing Zhou and Jiaju Ying and Jie Liu},
  title = {{Exploring an application-oriented land-based hyperspectral target detection framework based on 3D–2D CNN and transfer learning}},
  journal = {EURASIP Journal on Advances in Signal Processing},
  year = {2024},
  doi = {10.1186/s13634-024-01136-0},
}
Figure