GenRF: Generative Registration-free Image Fusion

Linfeng Tang, Zhijun Fan, Yang Chen, Zizhuo Li, Hao Zhang, Jiayi Ma

TL;DR

GenRF is a generative, registration-free framework that fuses multi-modal images (like infrared and visible) even when they are severely misaligned or degraded. By adapting the FLUX foundation model, it uses cross-attention to "soft-align" features and generative priors to hallucinate missing content, outperforming traditional cascaded pipelines.

What Problem Does This Paper Address?

In the real world, "perfect" image fusion is a myth. When you try to combine a visible-light photo with an infrared (thermal) image, you run into three major roadblocks:

  1. Geometric Misalignment: Because the sensors are physically separated, they see the world from slightly different angles (parallax).
  2. Field-of-View (FOV) Mismatch: One camera might see more of the scene than the other, leaving "holes" where data from one modality is missing.
  3. Image Degradation: Real-world captures are often plagued by low light, noise, or haze, which makes traditional feature matching fail.

Existing solutions usually try a "register-then-fuse" approach. If the registration (alignment) step fails—which it often does with large parallax—the final fused image ends up with "ghosting" artifacts or weird distortions.

Figure 1. Real-world multi-modal acquisition challenges and rep- resentative solutions. TG-ECNet and DuS-DiFuse results are ob- tained using image pairs registered by AnyMatch and CPMFusion.

Methods

The authors propose GenRF, which skips explicit registration entirely. Instead of trying to warp one image to match the other perfectly, it treats fusion as a conditional generation task.

Key Components:

  • Foundation Model Backbone: It builds on FLUX 3, a state-of-the-art Diffusion Transformer (DiT). To make it efficient, they use LoRA (Low-Rank Adaptation) to fine-tune only a small fraction of the weights.
  • Cross-Attention as "Soft Alignment": Instead of requiring pixel-to-pixel correspondence, the model uses cross-attention. This allows a pixel in the visible image to "query" relevant thermal information from anywhere in the infrared image, effectively performing a position-agnostic search for complementary data.
  • Online Geometric Perturbation: During training, the authors intentionally warp the input images with random affine and elastic deformations. This forces the model to learn how to find correspondences even when things are messy.
  • Dynamic Degradation Construction: The model is trained on a wide variety of synthetic corruptions (noise, blur, haze, stripes) so it can perform restoration and fusion simultaneously.
Figure 2. Overall framework of our generative registration-free image fusion.

Datasets

  • Training: 2,071 image pairs from the DSPFusion dataset, augmented with synthetic warps and degradations.
  • Evaluation:
    • Real Scene: A newly collected benchmark of 250 pairs from various dual-sensor systems (drones, phones, vehicle-mounted cameras).
  • Public Benchmarks: DVTOD (object detection), MSRS, LLVIP, and M3FD.
  • Generalization Tasks: Multi-exposure (MEF) and multi-focus (MFF) datasets to test zero-shot capabilities.

Key Insights and Ideas

  • Attention is all you need (for alignment): The "aha moment" here is that cross-attention naturally handles misalignment. As shown in Figure 3, the model's attention peaks correctly track objects in the infrared domain even when they are shifted relative to the visible reference.
  • Priors fill the gaps: Because it's a generative model, GenRF doesn't just "blend" pixels; it understands scene structure. If the infrared sensor doesn't cover a certain area, the model uses its learned priors to infer what the thermal signature should look like, avoiding the ugly black borders or padding seen in other methods.
  • Joint Optimization > Cascaded Steps: By handling restoration, alignment, and fusion in one pass, the model avoids the error accumulation that kills traditional pipelines.
Figure 3. Attention responses in infrared images for numbered visible queries. \protect \bm {\circ } marks each averaged attention map’s peak.

Key Results

GenRF sets a new state-of-the-art across multiple metrics:

  • Visual Quality: It achieves the highest scores in perceptual metrics like DeQA (3.213) and QualiCLIP (0.708) on real-world scenes, significantly beating methods that rely on explicit registration.
  • Downstream Performance: On the DVTOD object detection task, using GenRF-fused images improved the mAP50:95 of a YOLOv11 detector to 0.524, compared to 0.494 for the next best method.
  • Zero-Shot Generalization: Without any task-specific fine-tuning, GenRF successfully fused multi-exposure and multi-focus images, proving that its learned fusion logic is remarkably general.
Figure 4. Qualitative comparison with registration-based methods on Real Scene and DVTOD. The first Real Scene example was captured with a rigidly mounted dual-sensor camera with a narrow baseline yet exhibits large scene-depth variation. The second uses a separate infrared camera and phone with a larger field-of-view mismatch. The first DVTOD example additionally exhibits temporal misalignment.

Limitations and Future Work

The primary drawback is computational speed. Because GenRF relies on an iterative diffusion process (30 denoising steps), it takes about 24 seconds to fuse a single 640x480 image pair on an RTX 4090. This is far from real-time.

The authors suggest that future work could explore one-step generative fusion (using techniques like Consistency Models or adversarial distillation) to bring the latency down for mobile or robotics applications. Additionally, while the model is robust, it is still a "heavy" foundation model; distilling this knowledge into a smaller, more efficient architecture would be a logical next step.

Summary updated

Abstract

Multi-modal image fusion combines complementary observations to improve scene visibility and perception. However, in real-world acquisition, differences in sensing principles and configurations introduce geometric discrepancies, while degradations further compromise source quality. Existing methods typically rely on explicit registration, yet large parallax complicates correspondence estimation, while mismatched fields of view leave some regions without cross-modal counterparts. Meanwhile, independent restoration underuses cross-modal information, whereas restoration-fusion methods assuming alignment remain vulnerable to misregistration. This study revisits misaligned and degraded image fusion in the era of generative foundation models, in which cross-attention enables position-agnostic complementary information aggregation, while generative priors support degradation recovery and inference in missing regions. Building on this perspective, we propose GenRF, which adapts pretrained FLUX through LoRA to jointly enhance degraded observations, infer missing cross-modal context, and fuse complementary information without explicitly estimating deformation fields. Extensive experiments on real-world infrared and visible captures show that GenRF delivers high-quality fusion directly from unregistered, degraded images without preprocessing, while improving downstream task performance under misalignment. Additional experiments demonstrate that GenRF can generalize to misaligned multi-exposure and multi-focus image fusion without task-specific fine-tuning. Project: https://linfeng-tang.github.io/GenRF.
BibTeX
@misc{tang2026genrf,
  author = {Linfeng Tang and Zhijun Fan and Yang Chen and Zizhuo Li and Hao Zhang and Jiayi Ma},
  title = {{GenRF: Generative Registration-free Image Fusion}},
  year = {2026},
  eprint = {2611.24157},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24157},
}
Figure