Source-Free Online Domain Adaptive Semantic Segmentation of Satellite Images Under Image Degradation

Fahim Faisal Niloy, Kishor Kumar Bhaumik, Simon S. Woo

Abstract

Semantic segmentation of satellite imagery is critical for real-time environmental monitoring and disaster response. However, deep learning models often suffer from significant performance degradation when deployed in the presence of unforeseen environmental conditions such as fog, snow, or sensor noise. This paper presents a novel approach for Source-Free Online Test-Time Adaptation (TTA) that enables models to adapt to degraded data streams in real-time without requiring access to the original training data. By utilizing a backpropagation-free strategy involving dynamic Batch Normalization (BN) statistics and global class centers, the proposed method achieves state-of-the-art performance across various degradation types while maintaining the computational efficiency necessary for high-resolution satellite data processing.

Introduction

Satellite-based semantic segmentation—the process of assigning categorical labels to every pixel in an image—is a foundational technology for tracking urban development, forest cover, and disaster impact. While modern convolutional neural networks (CNNs) perform exceptionally well on "clean" datasets, their accuracy plummets when faced with a "domain shift." In the context of remote sensing, this shift is often caused by atmospheric conditions (rain, snow, fog) or hardware limitations (sensor noise, motion blur).

Traditional Unsupervised Domain Adaptation (UDA) techniques attempt to mitigate this by training on both source and target data. However, in many practical scenarios, the original source data is unavailable due to privacy concerns or massive storage requirements. Furthermore, satellite data often arrives as a continuous stream, requiring "online" adaptation rather than "offline" batch processing. This paper addresses the challenge of Source-Free Online Test-Time Adaptation (TTA), proposing a lightweight, backpropagation-free framework designed for immediate deployment.

Methodology

The core innovation of this research is a two-pronged strategy that avoids the high computational cost of gradient-based optimization (backpropagation), making it suitable for resource-constrained or high-throughput environments.

1. Distribution Matching via Dynamic BN Statistics

Standard CNNs utilize Batch Normalization (BN) layers to stabilize training by tracking the mean and variance of activations. During deployment, these statistics are typically frozen. However, image degradations cause a shift in these internal distributions.

The proposed method replaces frozen statistics with a running average that progressively estimates the target domain's global statistics from the incoming data stream. To balance the need for rapid adaptation with long-term stability, the authors introduce Dynamic Momentum. This mechanism uses a high momentum value initially to quickly learn from the first few frames of a new environment and gradually decreases the momentum to stabilize the model as it gains a representative sample of the target domain.

2. Instance Matching via Global Class Centers

To refine pixel-level predictions that may still be erroneous after BN alignment, the method maintains "global class centers" (GCC). These centers act as prototypical feature signatures for each semantic category (e.g., urban, water, vegetation) within the degraded domain.

As new images are processed, high-confidence pixel features are used to update these centers. The final segmentation mask is then refined by comparing the features of every pixel to these prototypes, ensuring that the local predictions remain consistent with the global understanding of the class appearance under current conditions.

Experimental Setup

The researchers evaluated their approach using the DeepGlobe satellite dataset as the clean source domain. To simulate real-world challenges, they applied five types of synthetic corruption to the test set using the RobustBench framework:

  • Noise: Gaussian and Impulse noise.
  • Blur: Gaussian blur.
  • Weather: Snow and Fog.

The performance was measured using mean Intersection over Union (mIoU), comparing the proposed method against established TTA baselines such as TENT, DUA, and DIGA.

Results and Discussion

The experimental results demonstrate that the proposed method consistently outperforms existing state-of-the-art techniques across all degradation categories.

Performance in Extreme Weather

The most dramatic improvements were observed in the "Snow" degradation task. The original model, trained only on clean data, saw its performance drop to a negligible 3.5% mIoU. While other adaptation methods struggled to recover, the proposed method achieved 45.7% mIoU, outperforming the next best baseline (TENT) by 5.5%.

Robustness to Noise and Blur

Across Gaussian Noise, Impulse Noise, and Gaussian Blur, the method showed a consistent improvement of 3-6% mIoU over competing TTA methods. Ablation studies confirmed that both BN matching and Class Center matching are vital; for instance, on Impulse Noise, the unadapted model scored 37.3% mIoU, while the combined proposed strategy reached 56.1%.

Computational Efficiency

Because the method is backpropagation-free, it avoids the memory-intensive process of storing gradients. This allows the model to adapt to high-resolution satellite imagery in real-time, a feat that is often impossible for gradient-based TTA methods on standard hardware.

Conclusion

This research provides a robust and efficient framework for adapting satellite segmentation models to harsh environmental conditions on the fly. By focusing on distribution and instance matching without the need for backpropagation, the method bridges the gap between theoretical domain adaptation and practical, real-time deployment. Future work will focus on testing these methods against real-world multi-sensor data and addressing class imbalance in streaming data to ensure that rare but critical features are not lost during the adaptation process.

Summary updated

BibTeX
@inproceedings{niloy2024source,
  author = {Fahim Faisal Niloy and Kishor Kumar Bhaumik and Simon S. Woo},
  title = {{Source-Free Online Domain Adaptive Semantic Segmentation of Satellite Images Under Image Degradation}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10447965},
}
Figure