Estimating Physical Information Consistency of Channel Data Augmentation for Remote Sensing Images

Tom Burgert, Begüm Demir

Introduction

In computer vision, data augmentation is often considered a "free lunch" that improves model generalization by creating synthetic variations of training data. However, in Remote Sensing (RS), images are not merely pictures; they are precise measurements of physical reflectance, known as spectral signatures. A long-standing debate in the RS community centers on whether standard "channel" augmentations (e.g., solarize, grayscale, or brightness adjustments) destroy these underlying physical properties. This paper addresses this by quantifying "Physical Information Consistency"—the degree to which artificial augmentations mimic natural variations.

Methodology

The authors propose a framework to estimate physical consistency by comparing artificial augmentations against the "natural noise floor" of a time series. The process involves three key steps:

  1. Establishing a Baseline ($S_{noaug}$): Researchers analyze time-series data of specific geographical locations to measure the natural distance between pixel signatures at different timestamps (excluding clouds and land-use changes).
  2. Measuring Augmentation Impact ($S_{aug}$): A specific augmentation is applied to a pixel, and the distance it moves away from its closest "natural" neighbor in the time series is measured.
  3. The Consistency Test: An augmentation is deemed physically consistent if its score ($S_{aug}$) remains within the standard deviation ($\sigma$) of the natural baseline ($S_{noaug}$).
The three-step process for establishing a natural spectral baseline from time-series satellite imagery.

Experimental Setup

The study utilized the BigEarthNet-S2 dataset, a large-scale benchmark for remote sensing. Training data was sourced from the Lithuania split, while validation and testing were performed on the Ireland split. For the consistency metric, the authors extracted time series (2017–2020) for 70 randomly selected homogeneous locations to ensure the "natural" baseline was accurate.

Key Results and Discussion

The findings reveal that physical consistency is a necessary, though not always sufficient, condition for an augmentation to be beneficial.

Comparison of augmentation impact ($S_{aug}$) vs. natural deviation ($S_{noaug}$) across various transformations and magnitudes.
  • Physically Consistent & Helpful: Contrast and Sharpness consistently stayed within natural deviation bounds and improved the mean Average Precision (mAP) of the ResNet18 baseline.
  • Physically Inconsistent & Unhelpful: Grayscale was found to be physically catastrophic, yielding a score 40 times higher than natural variation. Brightness adjustments were consistent at low magnitudes but failed the test once changes exceeded roughly 12%, at which point model performance began to degrade.
  • The "Consistent but Useless" Category: Posterize and Solarize were physically consistent (they did not move pixels outside natural ranges) but failed to improve performance. This suggests that while staying "physical" is required, the augmentation must also provide a useful learning signal.

Conclusion

This research establishes a clear rule of thumb: if an augmentation distorts a pixel's spectral signature beyond the range of natural variation (such as seasonal changes or atmospheric conditions), it fails to improve deep learning model performance. While this study focused on supervised multi-label classification, the authors suggest this consistency metric is highly relevant for self-supervised learning, where selecting appropriate "views" is critical for success. Future work will focus on automating the selection of homogeneous areas to scale this metric across larger datasets.

Summary updated

Abstract

The application of data augmentation for deep learning (DL) methods plays an important role in achieving state-of-the-art results in supervised, semi-supervised, and self-supervised image classification. In particular, channel transformations (e.g., solarize, grayscale, brightness adjustments) are integrated into data augmentation pipelines for remote sensing (RS) image classification tasks. However, contradicting beliefs exist about their proper applications to RS images. A common point of critique is that the application of channel augmentation techniques may lead to physically inconsistent spectral data (i.e., pixel signatures). To shed light on the open debate, we propose an approach to estimate whether a channel augmentation technique affects the physical information of RS images. To this end, the proposed approach estimates a score that measures the alignment of a pixel signature within a time series that can be naturally subject to deviations caused by factors such as acquisition conditions or phenological states of vegetation. We compare the scores associated with original and augmented pixel signatures to evaluate the physical consistency. Experimental results on a multi-label image classification task show that channel augmentations yielding a score that exceeds the expected deviation of original pixel signatures can not improve the performance of a baseline model trained without augmentation.
BibTeX
@inproceedings{burgert2024estimating,
  author = {Tom Burgert and Begüm Demir},
  title = {{Estimating Physical Information Consistency of Channel Data Augmentation for Remote Sensing Images}},
  booktitle = {IGARSS},
  year = {2024},
  url = {https://openreview.net/forum?id=BBL8OHEqLb},
}
Figure