Introduction
In remote sensing, the integration of Hyperspectral Images (HSI) and Light Detection and Ranging (LiDAR) data is a pivotal strategy for land-cover classification. HSI provides dense spectral information for material identification, while LiDAR offers critical 3D structural and elevation data. The primary challenge is the "heterogeneous gap"—the difficulty in effectively fusing data types representing fundamentally different physical properties. This paper introduces the Linear Self-Attention Fusion (LSAF) model, a novel framework designed to bridge this gap through adaptive cross-modal feature interaction.
Methodology
The LSAF model architecture is structured into three primary stages: feature extraction, linear self-attention-based interaction, and adaptive decision fusion.
1. Feature Extraction Module
To ensure compatibility between modalities, the model employs a retrained HybridSN (a 3D-2D CNN) for HSI data and a series of three ConvBlocks for LiDAR data. This stage produces feature maps of consistent dimensions, facilitating subsequent fusion.
2. Linear Self-Attention (LiST) Module
The LiST module is the core innovation, acting as a "plug-and-play" component for cross-modal interaction. It integrates channel-wise attention, inspired by SENet, which recalibrates feature channels through a summation of HSI and LiDAR features. This is complemented by spatial attention using a softmax operator to prioritize significant spatial regions across the combined feature set.
3. Self-adaptive Decision Fusion
Rather than simple concatenation, the final classification output is a weighted summation of the HSI-specific result, the LiDAR-specific result, and the fused result. The weights are learned dynamically, allowing the model to prioritize the most reliable data source for each specific pixel.
Experimental Results
The LSAF model was evaluated using the Houston dataset, a standard benchmark in remote sensing consisting of 15 land-cover categories.
As shown in the results table, LSAF achieved a state-of-the-art Overall Accuracy (OA) of 95.40%, outperforming existing methods such as $A^3CLNN$ (90.55%) and PToP CNN (92.48%). Notably, the model demonstrated significant improvements in challenging categories like "Healthy grass" (98.22%) and "Road" (95.50%), where spectral similarities often lead to misclassification in single-modality systems. Qualitatively, the classification maps produced by LSAF exhibit high visual fidelity to ground truth labels, with reduced noise in complex urban areas.
Conclusion
The LSAF model demonstrates that linear self-attention is a highly effective mechanism for bridging the heterogeneous gap between HSI and LiDAR data. By forcing cross-modal interaction and employing adaptive decision fusion, the model achieves superior classification accuracy. Future research will focus on evaluating the model's generalizability across diverse datasets and optimizing its computational efficiency for real-time applications.
Summary updated