Change Detection between Multimodal Remote Sensing Data Using Siamese CNN

Zhenchao Zhang, George Vosselman, Markus Gerke, Devis Tuia, Michael Ying YangarXiv 2018 · arXiv

Abstract

Urban change detection often involves comparing disparate data sources, such as 3D laser scans and 2D aerial imagery. This paper presents a deep learning framework using a Siamese Convolutional Neural Network (S-CNN) to bridge these modalities. By converting data into a common 2.5D representation and employing a patch-based comparison, the system achieves 86.4% accuracy in identifying topographic changes while effectively filtering sensor noise and registration errors.

Introduction

Maintaining accurate topographic maps is essential for urban planning and environmental monitoring. A common hurdle in this field is the temporal and modal discrepancy between datasets; for instance, comparing an older 3D laser survey with newer 2D aerial images. Direct comparison is difficult because 3D point clouds are precise but sparse, whereas image-derived 3D data is often noisy and sensitive to shadows. Simple subtraction of height maps typically yields excessive false positives due to sensor artifacts rather than actual physical changes.

Methodology

The proposed pipeline bridges the gap between 3D laser scans and 2D imagery through a four-step process:

  1. Data Harmonization: Both modalities are converted into a common 2.5D Digital Surface Model (DSM). Photogrammetric techniques are used to generate a point cloud from the 2D images to facilitate this alignment.
  2. Non-linear Height Mapping: To preserve detail, height values are converted into grayscale image patches using a non-linear function. This ensures that smaller objects, like trees, retain fine-grained features while the range for tall buildings is compressed.
  3. Siamese CNN Architecture: The core of the system is a Siamese network with two identical branches sharing weights. It processes patches from two different times simultaneously, extracting high-level features and calculating the Euclidean distance between them.
Patch-based 2.5D representation and height mapping.
  1. Object-Level Verification: Candidate changes are grouped into objects and verified using spectral indices, such as the normalized Excessive Green Index (nEGI), to distinguish between vegetation and man-made structures.

Experimental Setup

The framework was evaluated using a real-world dataset from Enschede, the Netherlands. The data included laser scanning from 2007 and oblique aerial images from 2011. The 1.6 km² study area was divided into nine tiles for training, validation, and testing. Data augmentation techniques, including rotations and flips, were applied to create a balanced training set of over 12,000 patch pairs.

Results and Discussion

The S-CNN model achieved a patch-classification accuracy of 86.4%, with precision and recall reaching 79.9% and 80.6%, respectively. A key strength of the model is its ability to learn the difference between actual topographic changes and "false changes" caused by sensor noise or registration shifts.

Multimodal input data and change detection results.

The use of shared weights in the Siamese architecture ensures consistent feature extraction across both time steps. Furthermore, the hybrid approach—combining deep learning with traditional spectral indices—allows the system to resolve ambiguities that height data alone cannot, such as distinguishing a building roof from a dense tree canopy.

Conclusion

This research demonstrates that a Siamese CNN can effectively manage the complexities of multimodal change detection. While the patch-based approach is robust, future work aims to transition toward pixel-level "dense prediction" to provide higher-resolution change maps. Addressing omission errors in small vegetation changes also remains a priority for refining urban monitoring systems.

Summary updated

BibTeX
@misc{zhang2018change,
  author = {Zhenchao Zhang and George Vosselman and Markus Gerke and Devis Tuia and Michael Ying Yang},
  title = {{Change Detection between Multimodal Remote Sensing Data Using Siamese CNN}},
  year = {2018},
  eprint = {1807.09562},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.09562},
}
Figure