Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion

Yonghao Xu, Pedram Ghamisi, Qihao WengarXiv 2026 · arXiv

TL;DR

Dataset distillation condenses massive training sets into a handful of synthetic images while preserving model performance, but it struggles with the complex spatial layouts of satellite imagery. This paper introduces Discriminative Prototype-guided Diffusion (DPD), the first dataset distillation framework tailored for remote sensing. By guiding a diffusion model with latent category prototypes and filtering candidates with a specialized classifier, DPD creates tiny, hyper-effective synthetic datasets that drastically cut down training costs.

What Problem Does This Paper Address?

Deep learning has revolutionized remote sensing (RS) image interpretation, but it comes at a price: massive datasets that demand significant storage and computational power. Dataset distillation (DD) offers an elegant solution by synthesizing a tiny, ultra-representative fraction of data (e.g., 10–20 images per class) that can train a model to nearly the same accuracy as the full dataset.

Fig. 1. Overview of the motivation for this work. Through dataset distillation, large-scale remote sensing datasets can be condensed into compact synthetic datasets that enable models to achieve performance comparable to training on the original full dataset.

While DD has shown promise for natural images (like cats and dogs), directly applying it to remote sensing is incredibly difficult. Satellite and aerial images feature highly complex spatial structures, diverse visual styles, and varying illumination conditions. Existing generative methods focus heavily on making images look visually realistic, but they often fail to capture the precise, discriminative features that downstream classification models actually need to learn.

Methods

The authors propose Discriminative Prototype-guided Diffusion (DPD), which operates in three main phases:

  1. Latent Diffusion Pretraining: The authors fine-tune a Stable Diffusion backbone in a compressed latent space using Low-Rank Adaptation (LoRA) on the target remote sensing dataset.
  2. Latent Prototype Extraction: To capture the diverse sub-types within a single class (e.g., different layouts of "stadiums"), real images are encoded into a VAE latent space and clustered using K-Means. For each cluster, a representative "prototype" is selected based on a custom margin metric that maximizes intra-cluster compactness and inter-cluster separability.
  3. Prototype-guided Latent Denoising: During the reverse diffusion process, the model samples "semantic anchors" on a hypersphere around these prototypes. This guidance prevents the diffusion model from collapsing into generating redundant, identical images, ensuring the synthetic set remains diverse.
Fig. 2. Overview of the proposed prototype-guided latent denoising process. Given a prototype sample, DPD constructs a semantic anchor via hyperspherical sampling in the latent space. During reverse diffusion, the clean latent estimate at each guided timestep is adjusted toward the semantic anchor through a prototype guidance direction, and the resulting window-masked latent is used for the reverse update. The final denoised latent is decoded by the VAE decoder to obtain the distilled sample.

To further polish the dataset, DPD generates multiple candidates for each prototype slot. A lightweight latent classifier ranks these candidates using a logit-margin criterion, selecting only the most highly discriminative samples to form the final distilled dataset.

Datasets

The framework was evaluated on three widely used high-resolution remote sensing scene classification benchmarks:

  • UC Merced (UCM): 2,100 aerial images across 21 land-use categories.
  • Aerial Image Dataset (AID): 10,000 images across 30 categories with varying spatial resolutions.
  • NWPU-RESISC45: A large-scale dataset containing 31,500 images across 45 scene categories.

Key Insights and Ideas

  • Visual Realism $\neq$ Classification Utility: Standard generative metrics like Frechet Inception Distance (FID) measure how "real" an image looks, but don't guarantee that a neural network can learn from it. DPD explicitly bridges this gap by optimizing for discriminative quality.
  • Hyperspherical Guidance for Diversity: Instead of unguided generation, which naturally gravitates toward the most generic "average" image of a class, sampling anchors on a hypersphere around category prototypes forces the model to explore diverse visual variations.
  • Latent-Space Filtering is Cheap: Rather than decoding hundreds of synthetic images back into pixel space to evaluate their quality, DPD filters candidates entirely within the VAE latent space using a lightweight classifier, keeping computational overhead remarkably low.

Key Results

The authors tested their distilled datasets across multiple architectures, including VGG16, Inception-v3, ResNet18, and DenseNet121, comparing DPD against state-of-the-art diffusion-based distillation methods like ManifoldGD and DiT-IGD.

  • Superior Accuracy at Low Budgets: Under a strict budget of just 5 images per class (IPC = 5), DPD outperformed the strongest baseline (ManifoldGD) by over 8 percentage points on the AID dataset across multiple architectures.
  • Scalability to Large Datasets: On the massive NWPU dataset, where the distilled data accounted for a mere 2.5% of the original training set, DPD outperformed baselines by roughly 8 percentage points on DenseNet121.
  • Favorable Efficiency: DPD achieves a highly competitive accuracy-efficiency trade-off. It can be trained in under an hour on a single NVIDIA A100 GPU, and generating the distilled dataset takes less than 50 minutes.
Accuracy-efficiency trade-off comparison

Limitations and Future Work

While DPD excels at image-level scene classification, the authors note that extending dataset distillation to more complex, fine-grained remote sensing tasks remains an open challenge. Future work will focus on adapting this framework for object detection (e.g., finding ships or airplanes) and pixel-level land-cover mapping (semantic segmentation).

Summary updated

Abstract

Recent years have witnessed the remarkable success of deep learning in remote sensing image interpretation, driven by the availability of large-scale benchmark datasets. However, this reliance on massive training data also brings substantial storage and computational costs. To address this challenge, this study introduces the concept of dataset distillation into the field of remote sensing image interpretation for the first time. Specifically, we propose discriminative prototype-guided diffusion (DPD), a diffusion-based generative distillation framework that condenses a large-scale remote sensing dataset into a compact and representative distilled dataset. To improve the semantic fidelity and diversity of the synthesized samples, we extract representative prototypes for each category in the latent space. We then construct hyperspherical semantic anchors around the prototypes to guide the reverse denoising trajectory. Furthermore, to enhance the discriminative quality of the generated samples, multiple candidates are generated for each prototype and ranked by a latent classifier using a logit-margin criterion, with the most discriminative candidates selected to form the final distilled dataset. Experiments on three high-resolution remote sensing scene classification benchmarks show that the proposed method can distill realistic, diverse, and discriminative samples for downstream model training. Code and pre-trained models are available online (https://github.com/YonghaoXu/DPD).
BibTeX
@misc{xu2026towards,
  author = {Yonghao Xu and Pedram Ghamisi and Qihao Weng},
  title = {{Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion}},
  year = {2026},
  eprint = {2601.15829},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.15829},
}
Figure