Accounting for Spatial Variability with the Histogram of Oriented Gradients Based Masking Improves Performance of Masked Autoencoder over Hyperspectral Satellite Imagery (Student Abstract)

Tanjim Bin Faruk, Abdul Matin, Shrideep Pallickara, Sangmi Lee PallickaraAAAI 2025 · DOI

This research, presented at AAAI-25, introduces a refined self-supervised learning approach for hyperspectral satellite imagery. By replacing the standard "random masking" in Masked Autoencoders (MAE) with a strategy driven by the Histogram of Oriented Gradients (HOG), the authors demonstrate significant improvements in image reconstruction and feature learning, particularly in geographically unseen regions.

The Problem: Sub-optimal Random Masking

Masked Autoencoders (MAE) typically rely on random masking, where a large portion of an image (often 75%) is hidden, and the model is tasked with reconstructing the missing patches. While effective for standard datasets like ImageNet, random masking is often sub-optimal for hyperspectral satellite imagery. Satellite data is dense and spatially diverse, containing complex topographical features across hundreds of spectral bands. Randomly masking these images can result in the loss of critical structural information, such as edges and textures, which are vital for learning meaningful geospatial representations.

The HOGMAE Approach

To address this, the authors propose HOGMAE, a "zero-shot" masking strategy that identifies important spatial features before training begins. Instead of expensive pre-trained attention mechanisms, HOGMAE utilizes the Histogram of Oriented Gradients (HOG)—a classic computer vision descriptor—to guide the masking process.

Figure 1: Comparison of Random Masking versus the proposed HOG-based masking strategy. HOG features are extracted to identify high-information patches (edges/textures) which are then preferentially retained.

The HOGMAE workflow involves:

  1. Dimensionality Reduction: Using Principal Component Analysis (PCA) to identify the single channel with the highest variance from the hundreds of hyperspectral bands.
  2. Feature Extraction: Computing HOG features for every image patch. Patches with high HOG values represent high visual information (e.g., sharp edges or complex textures).
  3. Biased Masking: Patches are categorized by variability. The strategy is biased to retain high-variability patches for the encoder while masking more of the low-variability (flatter) patches.
  4. Reconstruction: The model is trained to reconstruct the masked patches using a standard MAE architecture with a 75% total masking ratio.

Experimental Results

The study utilized data from the Environmental Mapping and Analysis Program (EnMAP), consisting of over 22,000 tiles with 218 spectral bands. The model was evaluated on two test sets, with Test Dataset 2 representing "unseen" geospatial areas to test generalization.

The HOG-based masking strategy consistently outperformed random masking across all metrics. On the unseen test set, HOGMAE achieved a 17.6% reduction in Mean Absolute Error (MAE), a 5.4% increase in PSNR, and a 4.8% increase in SSIM.

Figure 3: Quantitative performance comparison between Random and HOG-based masking strategies across two test datasets. HOG-based masking shows superior performance in all metrics.

Key Insights and Future Work

The results suggest that for satellite imagery, the spatial location of the mask is as critical as the masking ratio. By ensuring the encoder sees the "skeleton" of the landscape (the edges and shapes), the decoder becomes significantly more adept at filling in the gaps. This approach is particularly effective for generalizing to new terrains, as it helps the model learn universal topographical rules.

Future work will explore the computational overhead of the HOG/PCA preprocessing and evaluate the performance of HOGMAE-trained models on downstream tasks such as land-cover classification and change detection.

Summary updated

BibTeX
@inproceedings{faruk2025accounting,
  author = {Tanjim Bin Faruk and Abdul Matin and Shrideep Pallickara and Sangmi Lee Pallickara},
  title = {{Accounting for Spatial Variability with the Histogram of Oriented Gradients Based Masking Improves Performance of Masked Autoencoder over Hyperspectral Satellite Imagery (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35253},
}
Figure