Abstract
Semi-supervised semantic segmentation (SSSS) leverages a small amount of labeled data alongside a large pool of unlabeled data to reduce the high cost of pixel-level annotation. However, existing methods often suffer from "confirmation bias" due to noisy pseudo-labels and a reliance on pixel-wise confidence thresholds that fail near object boundaries. We propose CFCG, a framework that employs Cross-Fusion of multiple learners and Contour Guidance to refine pseudo-labels. By treating semantic contours as proxies for uncertainty, CFCG effectively mitigates error accumulation. Our method achieves state-of-the-art performance on PASCAL VOC 2012 and Cityscapes benchmarks.
Introduction
Semantic segmentation requires meticulous manual labeling, making it one of the most expensive computer vision tasks. SSSS aims to alleviate this by training models on limited labeled images and a larger set of unlabeled data, typically using consistency regularization. In this setup, a model's own high-confidence predictions serve as "pseudo-labels" for unlabeled images.
Despite its success, SSSS faces two major hurdles:
- Noisy Supervision: A single model's predictions are often biased, leading to error reinforcement.
- The Confidence Trap: Standard methods trust pseudo-labels based on pixel-wise confidence scores. However, models are often "confidently wrong," particularly at object boundaries.
As shown in the visualization above, a significant portion of high-confidence predictions can be incorrect, especially near contours. This demonstrates that simple thresholding is insufficient for robust semi-supervised learning.
Methodology
The CFCG framework addresses these challenges through three core components:
Coherent Perturbation (CP)
To enhance robustness, CFCG applies "heavy" perturbations at both the image level (e.g., noise, blur, erasure) and the feature level within the neural network's layers. This forces the model to learn more invariant and discriminative features.
Cross-Fusion Supervision (CFS)
Instead of a single teacher-student setup, CFCG utilizes multiple learners. It employs a channel-wise attention mechanism to fuse predictions from these learners. This "wisdom of the crowd" approach generates higher-quality pseudo-labels by tapping into the diverse knowledge captured by different model branches.
Adaptive Contour Guidance Module (ACGM)
The ACGM is the framework's most novel contribution. Recognizing that errors cluster at object boundaries, ACGM extracts semantic contours from pseudo-labels using a Laplacian operator. It then generates a weight map that directs the model's attention toward these error-prone regions. This guidance is introduced progressively: as training advances and pseudo-labels become more reliable, the model relies more heavily on these refined contour cues.
Experiments and Results
We evaluated CFCG on two industry-standard benchmarks: PASCAL VOC 2012 (20 categories) and Cityscapes (19 categories for urban scenes). Experiments were conducted under various partition protocols (e.g., 1/8 or 1/16 labels) to simulate data scarcity.
Key Results
CFCG established new state-of-the-art (SOTA) benchmarks:
- PASCAL VOC 2012: Under the 1/8 labeling protocol, it achieved 77.10% mIoU with a single learner and 77.55% with fusion inference, outperforming previous leaders like CPS and UCC by over 2%.
- Cityscapes: Under the 1/8 protocol, it reached 78.49% mIoU, a gain of nearly 1% over previous methods.
Crucially, even the single-learner version of CFCG (without the extra computation of fusion inference at test time) outperforms previous SOTA methods, making it highly practical for real-world deployment.
Discussion and Conclusion
The success of CFCG highlights that confidence is not always a reliable indicator of accuracy. By using semantic contours as proxies for uncertainty, the model can better navigate the "confidence trap." Furthermore, the cross-fusion of multiple learners provides a more robust supervision signal than single-model approaches.
While fusion inference provides the best results, it does increase computational costs during deployment. Future work could explore extending contour guidance to other dense prediction tasks, such as depth estimation or instance segmentation, where boundary accuracy is equally critical.
Summary updated