Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning

Hongruixuan Chen, Naoto Yokoya, Chen Wu, Bo Du

Introduction

Change detection (CD) is a cornerstone of remote sensing applications, ranging from disaster assessment to urban development monitoring. Traditionally, CD methods rely on homogeneous data—images captured by the same sensor type at different times. However, real-world scenarios often necessitate the comparison of multimodal data, such as pre-event optical imagery and post-event Synthetic Aperture Radar (SAR) data.

The primary challenge in multimodal CD is "modal heterogeneity": different sensors record different physical properties, making direct pixel-level comparisons mathematically invalid. While supervised deep learning offers a solution, the scarcity of labeled multimodal datasets remains a significant bottleneck. This paper presents an unsupervised framework that leverages structural relationships to bridge the gap between disparate modalities.

Methodology: The SR-GCAE Framework

The proposed method centers on the Structural Relationship Graph Convolutional Autoencoder (SR-GCAE), which shifts the focus from raw intensity values to the underlying structural patterns of the scene.

1. Object-Based Graph Construction

To reduce computational complexity and capture semantic context, the framework employs the Fractal Net Evolution Approach (FNEA) to segment images into superpixel objects. From these objects, two distinct graph types are constructed:

  • Local Structural Relationships: These capture the internal texture and spatial arrangement within individual objects.
  • Nonlocal Structural Relationships: These model the relationships between an object and other similar objects across the entire image, capturing broader spatial context.

2. Structural Relationship Graph Convolutional Autoencoder (SR-GCAE)

The core of the representation learning phase involves two specialized GCAEs designed to extract modality-independent features:

  • Edge Information Reconstruction: This branch focuses on capturing local textures, shapes, and boundaries.
  • Vertex Information Reconstruction: This branch targets high-level semantic features associated with the nodes (objects) themselves.

A key design choice is the use of "weak decoders." By limiting the decoder's complexity, the model forces the encoder to compress only the most robust, structural features into the latent space, effectively filtering out modality-specific noise.

3. Adaptive Fusion and Refinement

The system generates two separate difference maps based on local and nonlocal features. These maps are combined using an adaptive fusion strategy that weights each map based on its variance, ensuring the most informative features dominate the final output. Finally, simple morphological filtering (opening and closing operations) is applied to refine the change map and remove salt-and-pepper noise.

Experimental Evaluation

The SR-GCAE was evaluated against several state-of-the-art (SOTA) unsupervised methods, including M3CD and IRGMcS, across five diverse datasets: Shuguang (SAR/Optical), River (SAR/Panchromatic), Farmland (SAR/SAR), Texas (Multispectral/Multispectral), and Hanyang (Homogeneous).

The results demonstrate significant improvements in both quantitative metrics and visual quality. On the Shuguang dataset, the SR-GCAE achieved a Kappa Coefficient (KC) of 0.8214, outperforming the next best method (0.7698). In the Texas dataset, which involved identifying forest fire damage across different multispectral sensors, the model reached a KC of 0.9306. Qualitatively, the generated change maps were notably cleaner, showing fewer false positives in heterogeneous regions compared to traditional graph-based or transformation-based approaches.

Discussion and Key Insights

The success of the SR-GCAE highlights two critical insights for multimodal remote sensing:

  • Structure over Intensity: While pixel intensities vary wildly between SAR and optical sensors, the structural relationships between land-cover types remain relatively constant. By modeling these relationships, the framework achieves a high degree of modality invariance.
  • Dual-Scale Modeling: By integrating both local (intra-object) and nonlocal (inter-object) information, the model effectively detects changes at multiple scales, from fine-grained urban developments to large-scale environmental shifts like flooding.

Conclusion

The SR-GCAE framework provides a robust, unsupervised solution for the challenging task of multimodal change detection. By prioritizing structural relationship learning over intensity-based comparison, it eliminates the need for labeled data while maintaining high accuracy across diverse sensor pairings. Future work will focus on improving the computational efficiency of the graph learning stage and reducing the dependency on initial segmentation parameters.

Summary updated

BibTeX
@article{chen2022unsupervised,
  author = {Hongruixuan Chen and Naoto Yokoya and Chen Wu and Bo Du},
  title = {{Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2022},
  doi = {10.1109/tgrs.2022.3229027},
}
Figure