Siamese Networks for Bi-Temporal Change Detection

92 concepts84 papers2 proposals

Detecting structural or environmental changes across images taken at different times—known as bi-temporal or longitudinal change detection—is a foundational challenge in computer vision. Whether tracking urban sprawl from satellite imagery, assessing building damage after a natural disaster, or monitoring disease progression in medical scans, the core problem remains the same: how do we isolate meaningful semantic changes while ignoring "nuisance" variations like illumination shifts, seasonal changes, and sensor noise?

The standard solution has converged on the Siamese network architecture. By processing temporal image pairs through parallel, weight-sharing streams, these networks map disparate observations into a unified feature space where changes can be computed directly and reliably.


The Failure of Early Fusion

To understand why Siamese networks work, it is helpful to first look at the alternative: Early Fusion. In an early fusion setup, a pre-event image ($T_1$) and a post-event image ($T_2$) are stacked together along their channel dimension and fed into a single standard convolutional neural network (CNN) or Transformer.

While straightforward, early fusion forces the network to learn the relationship between the two time points at the lowest pixel level immediately. If the sun has set, if a cloud casts a shadow, or if the satellite angle varies slightly between $T_1$ and $T_2$, the pixel values change drastically. Early fusion networks struggle to disentangle these superficial "pseudochanges" from actual structural modifications, often leading to noisy and inaccurate change masks.


The Intuition of Shared-Weight Feature Extraction

Siamese architectures overcome this limitation by adopting a Middle Fusion strategy. Instead of mixing the images immediately, the network processes each image independently through two parallel branches (or "twins") that share the exact same parameters and weights.

Image T1 ───► [ Shared Encoder ] ───► Feature Map F1 ───┐
                                                         ├───► Fusion & Comparison ───► Change Mask
Image T2 ───► [ Shared Encoder ] ───► Feature Map F2 ───┘

Why is weight sharing so critical? It guarantees that both images are projected into the exact same latent representational space. Because the encoder's weights are identical for both branches, a specific physical object (such as a building corner or a blood vessel) will be mapped to the same coordinate vector in the feature space, regardless of whether it appears in the pre-event or post-event image.

As explored by Chen Wu et al. and reviewed comprehensively by Guangliang Cheng et al., this weight-sharing mechanism forces the network to focus on underlying feature distributions rather than transient pixel-level variations. In self-supervised frameworks like the one proposed by Meiqi Hu et al., this consistency is leveraged to actively suppress prevalent background changes (such as atmospheric or seasonal shifts), allowing the model to isolate rare anomalies or targeted structural alterations.


Isolating the Delta via Feature Fusion

Once the twin encoders have mapped both images into a consistent latent space, the next step is to compare the resulting feature maps ($F_1$ and $F_2$). The two primary mathematical operations used to achieve this are subtraction and concatenation.

Subtraction (Difference-Based Fusion)

Feature subtraction relies on a simple but powerful intuition: if the background remains static, its feature representations in both streams will be identical and will cancel each other out.

$$\text{Delta} = |F_1 - F_2|$$

For instance, Heejong Kim et al. demonstrated that subtracting Siamese feature vectors effectively disentangles meaningful biological progression from nuisance variations like image orientation or contrast in longitudinal medical imaging. The static anatomy disappears, leaving only the structural delta.

Concatenation-Based Fusion

Alternatively, concatenating the feature maps ($[F_1, F_2]$) preserves the full context of both time points, allowing subsequent decoder layers to learn complex, non-linear relationships between them. In frameworks like the one designed by Tianyuan Huang et al. for urban change tracking, models combine both approaches—concatenating the raw hidden vectors alongside their element-wise difference—to ensure the network captures both absolute structural states and their explicit discrepancies.


Overcoming Scale and Attention Bottlenecks

While the basic Siamese structure provides a clean comparative framework, real-world changes occur at multiple scales and across complex geometries. Recent innovations have focused on enhancing the Siamese architecture with multi-scale processing and attention mechanisms.

Multi-Scale Feature Learning

Changes can range from massive urban developments to tiny building extensions. To capture this spectrum, architectures like the one proposed by Yukun Wang et al. utilize multi-scale gated fusion modules to adaptively combine features across different network depths. Similarly, Yuhang Gan et al. employ nested Siamese structures with dense skip connections to preserve fine-grained spatial details that are typically lost during standard downsampling.

Attention and Transformers

To further combat pseudochanges, researchers integrate attention mechanisms directly into the Siamese streams. Jie Chen et al. introduced dual-attentive Siamese networks (DASNet) that apply spatial and channel attention to capture long-range dependencies and emphasize change-sensitive regions.

With the advent of vision transformers, the architecture has evolved from convolutional layers to self-attention blocks. For example, Liang Chen et al. developed a pure Transformer-based Siamese network (Siam-Swin-UNet) that replaces traditional convolutions with Swin Transformer blocks, enabling the model to capture global context more effectively and better handle challenging imaging conditions like off-nadir (side-looking) viewing angles.


Domain Agnosticism: From Earth Observation to Medicine

The core intuition of the Siamese change-detection paradigm—mapping two temporal states into a shared space for direct comparison—is entirely domain-agnostic.

In Earth Observation, it powers rapid disaster response workflows, such as the building damage assessment models deployed by Marie Urban et al. to identify structural failures in conflict zones.

In Healthcare, the exact same architectural principle is used to track patient health over time. Heejun Shin et al. utilized a shared-weight Siamese encoder to compare historical and contemporary chest X-rays, allowing a vision-language model to accurately describe disease progression. For chronic conditions like glaucoma, Vahid Mohammadzadeh et al. demonstrated that a twin-structured network processing serial optic disc photographs can forecast functional vision loss years before it becomes clinically detectable.

By separating feature extraction from feature comparison, Siamese networks provide a robust, mathematically intuitive framework for understanding how our world—and our bodies—change over time.

Key Foundational Work

Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images (2019) — contributed 1 concept elements to this direction.
Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images (2019) — contributed 1 concept elements to this direction.
Procedural Synthesis of Remote Sensing Images for Robust Change Detection with Neural Networks (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_siamese_network_architecture_d068552b,
  title = {siamese network architecture},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/siamese-network-architecture/rdr_d068552b7e8b43efa105d8c35caec304},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_siamese_network_architecture_d068552b_v1,
  title = {siamese network architecture},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/siamese-network-architecture/rdr_d068552b7e8b43efa105d8c35caec304/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake