Abstract: In the field of remote sensing and more specifically in Earth Observation, new data are available every day, coming from different sensors. Leveraging on those data in classification tasks comes at the price of intense labelling tasks that are not realistic in operational settings. While domain adaptation could be useful to counterbalance this problem, most of the usual methods assume that the data to adapt are comparable (they belong to the same metric space), which is not the case when multiple sensors are at stake. Heterogeneous domain adaptation methods are a particular solution to this problem. We present a novel method to deal with such cases, based on a modified cycleGAN version that incorporates classification losses and a metric space alignment term. We demonstrate its power on a land use classification tasks, with images from both Google Earth and Sentinel-2.
Introduction
The proliferation of remote sensing sensors provides a wealth of Earth Observation data. However, utilizing these data for classification tasks is often hindered by the prohibitive cost of manual labeling. While domain adaptation (DA) techniques offer a potential solution by transferring knowledge from labeled source domains to unlabeled target domains, standard methods typically assume that both domains share the same metric space. In practice, data from different sensors—such as high-resolution RGB imagery versus lower-resolution multispectral data—reside in fundamentally different domains. This challenge, known as Heterogeneous Domain Adaptation (HDA), requires specialized deep learning approaches to bridge the gap between disparate data distributions.
Proposed Methodology
This work introduces a novel HDA framework based on a modified CycleGAN architecture. Unlike standard CycleGANs designed for style transfer, this approach incorporates specific constraints to ensure that the semantic content and topological structure of the data are preserved during translation.
The architecture integrates two primary loss functions into the standard CycleGAN framework:
- Classification Consistency Loss ($L_{classif}$): Pre-trained classifiers on both source and target domains guide the generators to produce images that maintain their class identity, ensuring that translated images are still recognizable by domain-specific models.
- Metric Alignment Loss ($L_{metric}$): This loss ensures that the relative distances between samples are preserved across domains. If two images are close in the source domain, their translated versions remain close in the target domain, maintaining the data's topological structure.
Following training, the authors evaluated several strategies for final target classification, most notably HDAtarget, which involves translating unlabeled target images into the source domain to obtain high-quality pseudo-labels from the accurate source classifier.
Experimental Results
The method was evaluated on a cross-sensor land-use classification task using two distinct datasets: NWPU-RESISC45 (high-resolution RGB from Google Earth) as the source, and EuroSAT (lower-resolution multispectral from Sentinel-2) as the target. Nine common land-use classes were selected to test the adaptation.
As shown in the visual results, the proposed translation mechanism effectively bridges the gap between the high-resolution source and the multispectral target imagery. Quantitatively, the HDAtarget strategy proved highly effective. In a fully unsupervised setting with zero target labels, it achieved a classification accuracy of 65.75%. This significantly outperforms standard baselines and approaches the 77.00% accuracy of a fully supervised model trained on 650 labels per class. Interestingly, combining both translation directions (HDAfull) led to a performance drop, suggesting that source-to-target translations might introduce noise that degrades classifier training.
Conclusion and Future Work
By augmenting CycleGAN with classification and metric consistency losses, this research demonstrates a robust method for Heterogeneous Domain Adaptation in remote sensing. The approach significantly reduces the reliance on manual annotations when deploying models across different satellite sensors. Future work will focus on detailed ablation studies to isolate the impact of individual loss components and exploring more sophisticated distance metrics beyond simple Euclidean distance for metric alignment.
Summary updated