Abstract
Real-time semantic segmentation is a critical component for autonomous driving and scene understanding, requiring a delicate balance between computational efficiency and predictive accuracy. This paper introduces the Boundary Corrected Multi-scale Fusion Network (BCMFNet), an architecture designed to mitigate the trade-off between high-resolution detail and inference speed. By integrating a Low-resolution Multi-scale Fusion Module (LMFM) and a novel Boundary Corrected Loss (BCL), BCMFNet achieves state-of-the-art results, including 78.2% mIoU at over 100 FPS on the Cityscapes dataset.
Introduction
Semantic segmentation involves assigning a categorical label to every pixel in an image, a task essential for the spatial awareness of autonomous systems. However, existing solutions typically fall into two polarized categories: high-accuracy models that are too computationally intensive for real-time use, and lightweight models that sacrifice context and boundary precision for speed.
The primary challenge in lightweight models is "boundary smoothing," where object edges become blurred due to the loss of spatial resolution in downsampled feature maps. BCMFNet addresses this by extracting rich contextual information from low-resolution maps and explicitly correcting errors at object boundaries through a specialized loss function.
Methodology
Architecture Overview
BCMFNet utilizes the DDRNet-s backbone for initial feature extraction, organized into a four-stage architecture. The network's efficiency is derived from two primary innovations: the Low-resolution Multi-scale Fusion Module (LMFM) and the Boundary Corrected Loss (BCL).
Low-resolution Multi-scale Fusion Module (LMFM)
Traditional methods often perform expensive operations on high-resolution maps to preserve detail. In contrast, the LMFM operates on low-resolution maps using multiple average pooling kernels at 1/2, 1/4, and 1/8 scales. By employing 3x3 and 1x1 convolutions with "interval connections" to fuse these scales, the module captures global context without the massive computational overhead associated with high-resolution processing. This approach is based on the insight that global context can be effectively understood even at lower resolutions.
Boundary Corrected Loss (BCL)
To resolve the issue of blurry object edges, the authors introduced the Boundary Corrected Loss. During the training phase, BCL identifies "hard samples"—pixels located at boundaries where the model is most prone to error. The process involves:
- Flipping: Rows and columns of the ground truth are flipped to capture long-distance relationships.
- Non-Maximum Suppression (NMS): This technique is used to isolate and focus the model's learning on difficult boundary areas.
By treating boundaries as hard samples, the model is forced to prioritize fine structural details during optimization, leading to sharper transitions between semantic categories.
Experimental Results
Benchmarks
The model was evaluated on two industry-standard datasets:
- Cityscapes: 5,000 high-resolution images of urban street scenes with 19 categories.
- CamVid: 701 images from automotive broadcast video with 11 categories.
Performance Metrics
BCMFNet demonstrated superior performance in both speed and precision:
- Cityscapes: Achieved 78.2% Mean Intersection over Union (mIoU) at an inference speed of 103.4 FPS, outperforming established models like BiSeNet and ICNet.
- CamVid: Achieved 76.2% mIoU at 225 FPS.
Ablation Studies
Ablation experiments confirmed the efficacy of the individual components. The LMFM contributed an 0.8% increase in mIoU, while the BCL added an additional 0.6%. Together, these modules significantly enhanced the baseline DDRNet-s performance with negligible impact on inference speed.
Conclusion
BCMFNet successfully bridges the gap between speed and accuracy in semantic segmentation. By demonstrating that context can be efficiently extracted from low-resolution features and that boundary errors can be corrected through targeted loss functions, the research provides a viable path for deploying high-precision segmentation in real-time autonomous systems. Future work may focus on adapting these modules for even more resource-constrained edge hardware and further streamlining the training complexity of the Boundary Corrected Loss.
Summary updated