CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Eunki Cho, Hyeon Bae Kim, Seong Tae KimarXiv 2026 · arXiv

TL;DR

CoDeRNet is a new unified framework for remote sensing that jointly performs Change Detection (localizing changes) and Change Captioning (describing them). By treating the interaction between these tasks as a selective routing problem—transferring information only when one task needs support and the other is reliable—it achieves state-of-the-art performance, particularly in generating semantically accurate descriptions.

What Problem Does This Paper Address?

In remote sensing, understanding changes between two images taken at different times (bi-temporal) usually involves two distinct tasks: Change Detection (CD), which produces a pixel-level map of where things changed, and Change Captioning (CC), which generates a natural language sentence describing what happened.

While these tasks are complementary, they suffer from a "supervision mismatch." Detection is densely supervised at the pixel level, making it great at spatial precision but poor at global semantic reasoning. Captioning is weakly supervised at the sentence level, making it good at context but often spatially imprecise.

Existing unified models often just share features or use simple joint optimization. The authors argue this is suboptimal because not all information is equally reliable across different tasks and spatial locations. They ask: How can we selectively route information between these heterogeneous branches so they actually help each other?

Methods

The authors introduce CoDeRNet, which features a novel Confidence-Decomposed Routing (CoDeR) mechanism. Instead of a simple "gate" that lets information through, CoDeR decomposes the routing decision into two specific cues:

  1. Receiver-side Need ($1-C_t$): Does the target task (e.g., captioning) lack sufficient information at this specific location?
  2. Sender-side Reliability ($C_s$): Does the source task (e.g., detection) have high-confidence evidence to offer?

The routing mask $M$ is the product of these two: $M = (1-C_t) \cdot C_s$. Information is only transferred via a residual connection when the receiver needs help and the sender is reliable.

Overview of the CoDeRNet architecture and the selective routing mechanism.

The architecture uses a shared backbone to extract features, followed by task-specific "interfaces." The CoDeR module operates bidirectionally between these interfaces before sending the refined features to task-specific decoders: an FPN-style decoder for detection and a Transformer-based decoder for captioning.

Datasets

The model was evaluated on two primary benchmarks for unified Change Detection & Captioning (CDC):

  • LEVIR-MCI: A large-scale dataset containing over 10,000 image pairs with multi-class masks (buildings and roads) and five human-written captions per pair.
  • WHU-CDC: A dataset focused on post-earthquake urban reconstruction, containing over 7,000 image pairs with binary building-change masks and descriptive captions.

Key Insights and Ideas

  • Selective > Uniform: The core "aha moment" is that unconditional feature sharing can introduce noise. By explicitly modeling "need" and "reliability," the model preserves the unique strengths of each branch while filling in their gaps.
  • Asymmetric Scale Requirements: The authors found that routing works best when applied at different resolutions for each task. Detection (which needs detail) benefits from captioning cues at higher resolutions, while captioning (which needs semantics) benefits from detection cues at lower, more abstract resolutions.
  • Decomposition is Crucial: Ablation studies showed that the two-part routing mask (need + reliability) outperformed a single-gate approach. This proves that the model isn't just "gating" information, but actually reasoning about the relationship between the two tasks.
  • Factual Grounding: Beyond standard metrics, the authors used human evaluation to show that CoDeRNet produces captions that are more "factually grounded"—meaning the words in the sentence actually match the changes visible in the predicted mask.

Key Results

CoDeRNet achieved state-of-the-art results across both datasets, showing significant gains in captioning quality while maintaining or slightly improving detection accuracy.

  • LEVIR-MCI: Reached a CIDEr-D score of 141.79 and an mIoU of 87.79, outperforming the previous best unified model, MCINet.
  • WHU-CDC: Achieved an IoU of 90.25 and a CIDEr-D of 163.37, showing a substantial lead in consensus-based caption quality.
Table 1: Comparison on LEVIR-MCI [25]. “Change Detection” and “Change Captioning” denote single-task models trained for only one task, whereas “Change Detection & Captioning” denotes unified models that perform both tasks jointly within a single framework. Bold and underlined values indicate the best and second-best results, respectively.

Qualitatively, the model is much better at identifying specific counts and spatial relationships (e.g., "on both sides of the road") compared to baseline models that often miss small details or hallucinate objects.

Figure 2: Qualitative comparison on LEVIR-MCI. In the captions, building-related words (e.g., house, building) and road-related words (e.g., road) are highlighted using colors consistent with the corresponding mask classes.

Limitations and Future Work

  • Asymmetric Benefits: While captioning benefits significantly from detection cues, the reverse effect (captioning helping detection) is more modest. The "semantic gap" remains harder to bridge in that direction.
  • Ambiguity: The model still struggles with very subtle visual changes or linguistically ambiguous scenes where human annotators might disagree.
  • Future Directions: The authors suggest exploring explicit region-language alignment (e.g., using CLIP-like losses) to further strengthen the link between specific pixels and the words used to describe them.

Summary updated

Abstract

Change Detection & Captioning (CDC) aims to jointly localize changed regions and describe their semantic content from bi-temporal remote sensing images. While recent approaches have demonstrated the effectiveness of unified modeling, they primarily rely on shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored. However, detection and captioning differ fundamentally in their supervision granularity, leading to heterogeneous representations with uneven reliability across tasks and spatial locations. In this paper, we reformulate CDC as a selective information routing problem between heterogeneous task branches, where the key challenge is to determine when and where information should be transferred. To this end, we propose CoDeRNet, a unified framework that preserves task-specific representations while enabling bidirectional exchange of complementary information. CoDeRNet introduces a Confidence-Decomposed Routing (CoDeR) mechanism, which decomposes the routing condition into receiver-side need for support and sender-side reliability cues to guide selective transfer. Experiments on LEVIR-MCI and WHU-CDC show that CoDeRNet achieves strong overall CDC performance, particularly improving caption generation while preserving competitive change localization. Extensive analyses further show that the benefits arise not merely from joint learning or generic cross-task communication, but from explicitly modeling selective cross-task information routing. These results highlight the importance of conditional information transfer for unified CDC under heterogeneous supervision.
BibTeX
@misc{cho2026codernet,
  author = {Eunki Cho and Hyeon Bae Kim and Seong Tae Kim},
  title = {{CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning}},
  year = {2026},
  eprint = {2610.34876},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34876},
}
Figure