TL;DR
This paper tackles the challenge of filling gaps in satellite-derived Land Surface Temperature (LST) data, often caused by clouds. They introduce a novel deep learning approach using partial convolutions and ground-station air temperature data to achieve gapless, high-resolution LST maps, outperforming traditional statistical methods.
What Problem Does This Paper Address?
Land Surface Temperature (LST) is a crucial variable for various applications, from climate monitoring to agricultural yield prediction. Satellite remote sensing, like the MODIS sensor, provides LST data with high spatial and temporal resolution, often for free. However, clouds and cloud shadows frequently obscure the Earth's surface, creating data gaps that limit the utility of these datasets. While statistical interpolation methods exist, they struggle to incorporate external knowledge and learn complex local dependencies, leading to incomplete or less accurate reconstructions. This paper aims to overcome these limitations by developing a deep learning method that effectively interpolates missing LST data, even in challenging scenarios.
Methods
The authors propose a two-step deep learning approach:
- Air Temperature to LST Homogenization: They first convert local ground-site air temperature measurements into surface temperature approximations. This is achieved by training individual linear regression models for each weather station, aligning the air temperature data with the available LST data from nearby pixels. This step ensures a minimum amount of valid data for each day, even when satellite coverage is entirely obscured.
- Deep Interpolation with Partial Convolutions: The core of their method is a U-Net architecture enhanced with partial convolutions. Partial convolutions, introduced by Liu et al. [18], are designed to handle irregular holes in images by masking out invalid pixels during the convolution process. This prevents the network from learning from incorrect data. The U-Net architecture allows for effective feature extraction and reconstruction of images at the same resolution as the input. To train this network, they employ a novel strategy using partially occluded ground-truth data. This involves creating training pairs where both the input and the ground truth have missing information, with the network learning to infer the missing values based on the available data and the learned spatial dependencies. A partial loss function is used, calculated only on pixels where ground truth is available.
Datasets
- MODIS LST Data (MOD11 product): Used as the primary source for Land Surface Temperature (LST) data, characterized by 1 km spatial resolution. This dataset inherently contains gaps due to cloud cover.
- Local Ground-Site Weather Station Data: Air temperature measurements from 20 weather stations in the Rhineland-Palatinate region of Germany. This data is used to supplement the satellite data and provide a baseline for interpolation.
The paper does not introduce a new benchmark dataset but rather uses existing data within a specific research area for training and evaluation.
Key Insights and Ideas
- Leveraging Ground Truth for Deep Learning: The paper demonstrates that even partially occluded ground-truth data can be effectively used to train deep learning models for image inpainting tasks, a significant advantage given the scarcity of fully gapless LST data.
- Partial Convolutions for Irregular Gaps: The use of partial convolutions is key to handling the irregular shapes of cloud-induced gaps in LST data, allowing the network to learn from valid neighboring pixels without being misled by masked areas.
- Air Temperature as a Proxy: Converting ground-site air temperature to LST provides a crucial initial step to ensure some valid data points exist for every location, even under complete cloud cover, which is essential for the subsequent deep learning interpolation.
- Learning Spatial Context: The U-Net architecture, combined with the training strategy, allows the model to learn the inherent environmental constraints and spatial patterns of the research area, leading to more accurate interpolations than methods that treat each missing pixel in isolation.
Key Results
- Improved Accuracy: The proposed deep interpolation approach achieved a Root Mean Square Error (RMSE) of 1.67 °C, which is a 44% improvement over the state-of-the-art statistical approach (2.52 °C).
- Gapless Coverage: The method successfully provides 100% coverage of the research area, even in cases where satellite data is entirely missing and only inferred data from weather stations is available.
- High Coefficient of Determination: The model achieved a coefficient of determination (R²) of 0.95, indicating that it explains a high degree of variability in the target variable.
- Outperforms Statistical Methods: The deep learning approach consistently outperformed the statistical method in reconstructing LST, particularly in challenging areas with steep elevation gradients and extensive cloud cover.
Limitations and Future Work
- Research Area Specificity: The authors acknowledge that while their approach is highly specialized for their chosen research area, applying it to a new area would require retraining the model with new training examples specific to that region.
- Ground Station Availability: Achieving 100% coverage relies on the availability of local ground-site weather stations. While the approach can function without them, the coverage might be compromised.
- Cloud-Free Assumption: The evaluation of LST interpolation methods often relies on a "cloud-free assumption" for ground truth, as obtaining actual ground-truth LST for occluded pixels is challenging. This inherent bias in evaluation might affect the absolute accuracy assessment.
Future work could explore adapting the model to new research areas with less effort, potentially through transfer learning or more generalized architectures. Further investigation into the impact of different types of ground-truth data and the integration of other environmental covariables could also enhance the model's performance.
Summary updated