BIMII-Net: Brain-Inspired Multi-Iterative Interactive Network for RGB-T Road Scene Semantic Segmentation

Hanshuo Qiu, Jie Jiang, Ruoli Yang, Lixin Zhan, Jizhao LiuarXiv 2025 · DOI · arXiv

Abstract

Semantic segmentation in road scenes is critical for autonomous driving, yet standard RGB-based models often fail under adverse lighting and weather conditions. RGB-Thermal (RGB-T) fusion offers a robust alternative, but effective cross-modal feature integration remains a challenge. This paper presents BIMII-Net, a novel framework that leverages brain-inspired "Continuous-Coupled Neural Networks" (CCNN) to iteratively refine multi-modal features. By mimicking the feedback loops of the human visual cortex, BIMII-Net achieves state-of-the-art performance on major benchmarks, demonstrating superior robustness in complex environmental scenarios.

Introduction

Autonomous navigation requires precise environmental perception. While RGB cameras provide rich texture information, their performance degrades significantly in low-light, fog, or high-glare situations. Thermal sensors complement RGB data by capturing heat signatures, which are invariant to illumination. However, traditional "one-shot" fusion methods—such as simple concatenation or addition—often fail to capture the intricate relationships between shallow textures and deep semantic structures. Furthermore, standard feed-forward architectures lack the iterative refinement processes characteristic of biological vision systems.

Methodology

BIMII-Net addresses these limitations through a multi-iterative interactive architecture designed to optimize feature fusion and refinement.

Deep Continuous-Coupled Neural Network (DCCNN)

The core innovation of BIMII-Net is the DCCNN, which replaces standard static layers with iterative modules inspired by the primary visual cortex. Unlike traditional CNNs that process data in a single forward pass, the DCCNN employs a feedback mechanism where neuron states are updated over multiple time steps ($T$). This allows the network to iteratively "stare" at and refine features, capturing fine-grained details often lost in single-pass processing. The internal state of the CCNN layers is modulated by both current input and previous activity, mimicking the dynamic response of biological neurons.

Detailed structure of the DCCNN and CCNN layers illustrating the iterative feedback mechanism.

Cross Explicit Attention-Enhanced Fusion (CEAEF)

To manage the interaction between RGB and thermal branches, the model utilizes the CEAEF module. This component is designed to solve the "black box" problem of multi-modal fusion by using dual complementary activation branches. This ensures robust feature extraction even when one sensor's input is severely degraded—for instance, a washed-out RGB image in high-glare conditions. By explicitly modeling cross-modal attention, the network avoids over-reliance on a single modality and effectively integrates thermal heat signatures with visual textures.

Multi-Layer Decoder and Phased Training

The decoder architecture is specifically designed to handle the multi-scale nature of road scene features. It is partitioned into three specialized modules:

  • SFI-Module (Shallow Feature Interaction): Focuses on extracting and refining shallow-level textures and object contours.
  • DFI-Module (Deep Feature Interaction): Targets deep-level global "skeletons" and high-level semantic categories.
  • MFE-Module (Multi-scale Feature Enhancement): Dynamically integrates the outputs from the SFI and DFI modules using learnable weights to produce the final high-resolution segmentation map.

To manage the computational cost associated with iterative processing, the authors propose a phased training strategy. The model is initially trained with a single iteration ($T=1$) to establish a baseline for speed and basic feature extraction. It is subsequently fine-tuned with four iterations ($T=4$) to maximize segmentation accuracy through iterative refinement.

Overall architecture of BIMII-Net, showing the encoder-decoder structure and the integration of CEAEF and DCCNN components.

Experimental Results

BIMII-Net was evaluated on two benchmark datasets: MFNet and PST900.

  • MFNet Dataset: The model achieved a mean Intersection over Union (mIoU) of 58.4%, outperforming existing brain-inspired and traditional RGB-T models. It showed particular strength in identifying challenging small-scale objects such as guardrails and color cones.
  • PST900 Dataset: BIMII-Net reached an mIoU of 82.65%, setting a new performance standard for this multi-modal segmentation benchmark.

Crucially, the model maintained balanced performance across both day and night scenes, demonstrating significant robustness compared to competitors that typically experience accuracy drops in nocturnal conditions.

Conclusion

BIMII-Net demonstrates that biological plausibility can directly translate into enhanced computational performance for complex computer vision tasks. By integrating iterative feedback loops and sophisticated cross-modal attention, the framework provides a robust solution for RGB-T road scene segmentation. Future work will focus on optimizing the architecture for real-time deployment on edge devices without compromising its iterative refinement capabilities.

Summary updated

BibTeX
@misc{qiu2025bimii,
  author = {Hanshuo Qiu and Jie Jiang and Ruoli Yang and Lixin Zhan and Jizhao Liu},
  title = {{BIMII-Net: Brain-Inspired Multi-Iterative Interactive Network for RGB-T Road Scene Semantic Segmentation}},
  year = {2025},
  eprint = {2503.19303},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.19303},
}
Figure