Introduction
Standard semantic segmentation (SS) models typically treat every pixel with equal weight or use frequency-based weighting to handle class imbalance. However, in the context of autonomous driving or navigational aids for the visually impaired, a "miss" on a pedestrian is catastrophic, while a "miss" on a patch of sky is irrelevant. Existing loss functions like Cross-Entropy or Focal Loss don't account for this hierarchical importance. Furthermore, real-time models often sacrifice spatial detail to maintain high frame rates, which can lead to poor detection of slender or distant objects like telegraph poles or far-off pedestrians.
Methods
The proposed approach addresses these challenges through two primary innovations: a novel loss function and a bilateral network architecture.
Importance-Aware Loss (IAL)
The authors categorize classes into three groups based on safety relevance:
- G3 (Most Important): Cars, Pedestrians, Bicyclists, Signs.
- G2 (Intermediate): Roads, Sidewalks, Poles, Fences.
- G1 (Least Important): Sky, Buildings, Trees.
IAL applies dynamic weights using an "importance matrix" ($f_t$) that pushes the model to prioritize the recall of G3 objects. The goal is to ensure that if the model is uncertain, it errs on the side of detecting a critical object rather than ignoring it.
BiERF-PSPNet Architecture
Building on the ERF-PSPNet, the authors propose a bilateral structure to balance speed and accuracy:
- Context Path: A deep network (ERFNet + Pyramid Pooling Module) that extracts high-level semantic information at low resolution.
- Spatial Path: A shallower branch processing higher-resolution images to preserve fine textures and spatial details.
- Attention Fusion: An attention module fuses feature maps from both paths to ensure the final output is both semantically rich and spatially accurate.
Key Results and Insights
The models were evaluated on the CamVid and Cityscapes benchmarks, as well as a specialized Water Puddle dataset for preliminary loss function comparison.
- Recall Boost for Safety: On CamVid, IAL increased the recall of "Signs" from 34.0% to 47.0% and "Pedestrians" from 63.7% to 69.1% compared to standard Cross-Entropy. The authors emphasize that in autonomous navigation, a "false alarm" (low precision) is preferable to a "missed hazard" (low recall).
- Precision-Recall Trade-off: While IAL improves recall for critical objects, it can decrease their precision, occasionally causing the model to misclassify background pixels as important objects. Interestingly, IAL improved the precision of G1 (least important) classes; by becoming more certain about what constitutes a car, the model became better at identifying what is definitely background.
- Real-Time Performance: While the original ERF-PSPNet achieved 74.1 FPS, the new BiERF-PSPNet maintains a robust 42.1 FPS on a GTX 1080Ti—well above the 30 FPS real-time threshold—while providing significantly sharper edges and better detection of distant, small objects.
Conclusion
The Importance-Aware Loss and BiERF-PSPNet architecture provide a safety-first approach to semantic segmentation. By prioritizing the recall of critical hazards and maintaining real-time performance through a bilateral structure, this system offers a more reliable foundation for navigational assistants. Future work will focus on optimizing IAL parameters to mitigate the precision-recall trade-off and testing the system in diverse real-world scenarios for the visually impaired.
Summary updated