Abstract
The Harmonized LandSat-Sentinel (HLS) dataset has significantly advanced Earth Observation by integrating data from Landsat and Sentinel satellites. However, challenges persist in achieving spectral band parity between LandSat and Sentinel-derived HLS products. This paper presents an extended investigation aimed at enhancing spatial reconstruction accuracy to enable spectral band parity within HLS products. Building upon our previous work, which utilized generative neural networks to address partial feature mismatches between S30 and L30 products, we introduce a refined approach that fully integrates a Self-Supervised Learning (SSL)-pretrained encoder into a U-Net architecture. This method aims to access multi-scale features and improve spatial reconstruction accuracy, addressing the limitations in spatial resolution observed in our earlier study. Our methodology incorporates a comprehensive ablation study to assess various SSL-pretrained backbone architectures. Preliminary results demonstrate significant improvements in spatial reconstruction accuracy compared to our previous work. The adapted U-Net architecture, leveraging SSL-pretrained encoders, shows enhanced capability in capturing intricate spatial features within the HLS dataset. Our experiments demonstrate a substantial improvement in spatial resolution and feature reconstruction for L30 products, particularly in bands not natively present in Landsat data, paving the way for more accurate multi-sensor analyses.
Introduction
In the field of Earth Observation (EO), the Harmonized Landsat-Sentinel (HLS) dataset is a cornerstone for multi-sensor analysis, combining data from NASA’s Landsat (L30) and ESA’s Sentinel-2 (S30) satellites. However, these sensors are not perfectly aligned spectrally. Sentinel-2 captures several "Red Edge" bands—wavelengths critical for monitoring vegetation health—that are absent in Landsat imagery.
To make these datasets truly interchangeable, researchers attempt to synthesize these missing bands for Landsat. Previous attempts using generative neural networks, specifically Fully Convolutional Networks (FCNs), were radiometrically accurate but spatially degraded. The resulting images often appeared "blurry," losing the sharp boundaries and fine details necessary for precision agriculture. This study addresses this spatial reconstruction challenge by integrating advanced neural architectures with self-supervised pretraining.
Methodology
The proposed solution transitions from a standard FCN to a U-Net architecture integrated with self-supervised learning (SSL) backbones to preserve spatial fidelity.
The U-Net Framework and SSL Integration
The U-Net architecture is characterized by its symmetric encoder-decoder structure and, crucially, its skip connections. These connections pass high-resolution feature maps directly from the encoder to the decoder, bypassing the central bottleneck. This ensures that fine spatial structures are not lost during deep processing.
The encoder utilizes a ResNet50 backbone pretrained on the massive SSL4EO-L dataset. By leveraging weights from models trained via algorithms like MoCoV2 and SimCLR, the network starts with a sophisticated "understanding" of satellite imagery features. The model extracts features at five different downsampling levels (from 1:2 to 1:32 resolution), allowing it to capture both broad spectral context and pixel-level textures.
Data and Training
The model was trained using HLS S30 and L30 products from geographic tiles covering central and northern Greece in July 2023. This region provides a diverse mix of mountainous terrain and agricultural plains. During training, the SSL-pretrained encoder weights were frozen, allowing the model to retain robust feature representations while the decoder was optimized for band synthesis.
Results and Discussion
The integration of skip connections and SSL-pretraining yielded substantial improvements in both quantitative metrics and qualitative visual fidelity.
Quantitative Performance
The current approach outperformed previous benchmarks across all synthesized bands (RedEdge-1, RedEdge-2, RedEdge-3, and NIR-Broad). The model achieved a Peak Signal-to-Noise Ratio (PSNR) of approximately 34 dB and a Structural Similarity Index (SSIM) exceeding 90%. Furthermore, the Mean Absolute Error (MAE) was reduced to roughly 5.8% - 6.3%, confirming high radiometric accuracy alongside spatial improvements.
Qualitative Analysis
Visual comparisons demonstrate a significant leap in spatial clarity. As shown in the figure below, where previous FCN-based results appeared blurry, the current U-Net approach recovers distinct field boundaries and intricate spatial features that closely mimic actual Sentinel-2 captures.
Ablation studies further revealed that MoCoV2-based pretraining consistently produced higher-quality results with fewer visual artifacts compared to SimCLR.
Conclusion
By combining a U-Net architecture with SSL-pretrained encoders, this work successfully mitigates the spatial blurriness previously associated with synthetic band generation in the HLS dataset. While a minor scale loss persists compared to native high-resolution data, these results represent a significant step toward full spectral parity between Landsat and Sentinel-2. Future research will explore the integration of temporal context and the application of this methodology to other multi-sensor fusion tasks.
Summary updated