Introduction
Remote Sensing Image Change Captioning (RSICC) is a specialized task that involves generating natural language descriptions of semantic changes between bi-temporal satellite images. This task is critical for environmental monitoring, urban planning, and disaster assessment. However, RSICC faces significant challenges: the "pixel problem," where non-semantic changes (lighting, shadows) confuse models; data scarcity in annotated remote sensing datasets; and the limitations of traditional auto-regressive decoders that often produce rigid, repetitive sentence structures.
To address these issues, we present Diffusion-RSCC, a generative framework that treats change captioning as a probabilistic diffusion process. By iteratively denoising Gaussian noise into descriptive text, the model achieves greater linguistic flexibility and better alignment with visual features compared to standard LSTM or Transformer-based approaches.
Methodology
The Diffusion Framework
Diffusion-RSCC operates through a two-stage process. In the Forward Process, ground-truth captions are converted into continuous word embeddings, and Gaussian noise is gradually added until the signal is destroyed. The Reverse Process involves a learned Condition Denoiser that starts from pure noise and iteratively recovers the caption, guided by the bi-temporal image features.
Condition Denoiser Architecture
The core of the model is the Condition Denoiser, which maps visual changes to textual sequences. It consists of three primary components:
- Feature Extraction: A ResNet101 backbone extracts high-level features from the "before" and "after" images, which are then used to compute a residual map representing the semantic changes ($I_{di}$).
- Cross-Mode Fusion (CMF): This module employs multi-head cross-attention to fuse visual change features directly into the textual diffusion sequence, ensuring that the generated words are grounded in the observed pixels.
- Stacking Self-Attention (SSA): A series of self-attention blocks refine the internal coherence of the sentence. Ablation studies indicate that a configuration of three SSA blocks provides the optimal balance between grammatical structure and semantic accuracy.
Experimental Evaluation
Dataset
The model was evaluated on the LEVIR-CC dataset, the current benchmark for RSICC. It contains 10,077 image pairs with over 50,000 descriptive sentences covering five change types: buildings, roads, parking lots, vegetation, and water. The images feature a high resolution of 0.5m/pixel and span time intervals of 5 to 15 years.
Quantitative Results
Diffusion-RSCC demonstrates state-of-the-art performance across all standard metrics on LEVIR-CC, outperforming established baselines like DUDA and RSICCformer:
- BLEU-4: 60.9% (vs. 57.6% for DUDA)
- METEOR: 37.8%
- CIDEr: 125.6
- BARTScore: -417.7
- MoverScore: 32.3
The improvement in BARTScore and MoverScore highlights the model's superior ability to generate linguistically diverse and semantically accurate captions that go beyond simple n-gram matching.
Discussion and Conclusion
Diffusion-RSCC represents a significant shift from deterministic to generative modeling in remote sensing. By leveraging the diffusion process, the model effectively mitigates overfitting on small datasets and produces more descriptive outputs, such as identifying "neatly arranged villas" instead of generic "buildings."
While the iterative nature of diffusion models introduces higher computational costs compared to single-pass decoders, the gains in factual correctness and descriptive richness are substantial. Future work will focus on optimizing the computational efficiency of the reverse process and extending the model to handle more complex, multi-object change scenarios.
Summary updated