SMR-Net: Semantic-Guided Mutually Reinforcing Network for Cross-Modal Image Fusion and Salient Object Detection

Guobao Xiao, Xinyu Liu, Zebin Lin, Rui MingAAAI 2025 · DOI

Abstract

SMR-Net is a lightweight, dual-task network designed to bridge the gap between image fusion and salient object detection (SOD). By employing a "mutually reinforcing" architecture, the model produces high-quality fused images while simultaneously identifying key objects with high accuracy. SMR-Net achieves superior performance compared to specialized models while maintaining a significantly smaller parameter footprint, making it ideal for resource-constrained applications.

Introduction

Visible (Vis) and Infrared (IR) imaging offer complementary strengths: Vis provides rich textures but fails in low-light conditions, while IR highlights thermal targets but lacks environmental context. Traditionally, image fusion and salient object detection (SOD) are treated as independent tasks. However, this paper argues that these tasks are intrinsically linked. High-quality fusion facilitates better detection, and semantic information from detection can guide the fusion process to preserve critical object details.

Methodology

The authors introduce SMR-Net, which consists of two interconnected subnetworks designed to work in a mutually reinforcing loop.

Architecture of SMR-Net showing the PCI and BPS subnetworks.

Progressive Cross-modal Interaction (PCI) Subnetwork

The PCI subnetwork handles image fusion through a two-stage process. A local stage utilizes standard convolutions for fine detail extraction, followed by a global stage that employs spatial and channel attention mechanisms to capture broader context. This ensures that both fine textures and overall scene structure are preserved.

Bit-Plane Slicing-based SOD (BPS) Subnetwork

The BPS subnetwork treats the fused output as a "third modality." It utilizes Bit-Plane Slicing to isolate high-contrast semantic information, which is often obscured in raw pixel data. Additionally, it employs Deformable Convolutions to better adapt to the irregular shapes of real-world objects.

Mutual Reinforcement

A key innovation is the feedback loop where semantic information from the SOD task is used to "reweight" features in the fusion task. By prioritizing pixels belonging to important objects, the model ensures they are prominently featured and clearly defined in the final fused image.

Experimental Setup

The model was evaluated on several industry-standard benchmarks. For Salient Object Detection, the primary training set was VT5000, with additional testing on VT1000 and VT821. For Image Fusion, the TNO, RoadScene, and MSRS datasets were used, covering scenarios such as nighttime, camouflage, and complex urban environments.

Key Results

SMR-Net demonstrates significant improvements in both accuracy and efficiency across both tasks.

Performance comparison of SMR-Net against state-of-the-art methods in terms of mAP, FLOPs, and parameters.
  • Superior Accuracy: On the VT5000 dataset, SMR-Net achieved a 44.4% improvement in F-measure and a 73.7% reduction in Mean Absolute Error (MAE) compared to the MTMR model.
  • Visual Quality: In fusion tasks, SMR-Net consistently outperformed 9 other state-of-the-art methods, producing images with higher contrast and better texture preservation.
  • Efficiency: As shown in Figure 1, SMR-Net maintains a competitive mAP while requiring significantly fewer FLOPs and parameters. It uses 26% fewer parameters than its nearest competitors, making it highly viable for real-time applications on edge devices.

Conclusion

SMR-Net validates the hypothesis that image fusion and salient object detection can mutually benefit from a unified architecture. By leveraging semantic guidance and efficient bit-plane slicing, the network achieves state-of-the-art results with a lightweight design. Future work may involve extending this mutual reinforcement framework to other high-level vision tasks like object tracking or semantic segmentation.

Summary updated

BibTeX
@inproceedings{xiao2025smr,
  author = {Guobao Xiao and Xinyu Liu and Zebin Lin and Rui Ming},
  title = {{SMR-Net: Semantic-Guided Mutually Reinforcing Network for Cross-Modal Image Fusion and Salient Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i8.32933},
}
Figure