Abstract
Semantic segmentation is a critical task in computer vision, yet the high cost of pixel-level annotations remains a significant barrier. Semi-supervised learning (SSL) offers a solution by leveraging both labeled and unlabeled data. In this work, we propose the Reliable Weighted Multi-Phase (RWMS) learning method. RWMS utilizes a self-training framework with two distinct teacher models to evaluate the reliability of pseudo-labels. By selecting reliable data at the image level and reweighting pseudo-labels at the pixel level, we guide multi-phase training toward more trustworthy knowledge. Combined with strong data augmentation, RWMS achieves remarkable performance, outperforming baseline methods by over 3% on the Pascal VOC and Cityscapes benchmarks.
The Challenge of Semi-Supervised Segmentation
Assigning a category to every pixel in an image is labor-intensive. While self-training—where a model generates "pseudo-labels" for unlabeled data to train itself—is a popular SSL strategy, it suffers from a fundamental flaw: noise. If a model generates an incorrect pseudo-label, it reinforces that error during training, leading to confirmation bias and degraded performance.
Existing solutions like co-training (CT) or dual-model training (DMT) attempt to mitigate this by using two models to cross-check each other. However, these often come with high computational costs or fail to fully exploit the consensus between different architectures. RWMS was designed to integrate the wisdom of dual models more efficiently while focusing the learning process on the most reliable data.
The RWMS Framework
The RWMS framework introduces three core innovations to stabilize the self-training process:
1. Dual-Teacher Reliability Assessment
RWMS employs two different teacher architectures (e.g., DeepLabv3+ and PSPNet). The core hypothesis is that simultaneous errors across different architectures are rare. Consensus is measured at two levels:
- Image Level: The Mean Intersection-over-Union (MIoU) between the two teachers' predictions determines the overall "reliability" of an image.
- Pixel Level: A reweighting formula, $w_i = ae^{-b(x-1)}$, is applied to each pixel. Here, the weight depends on how highly the second teacher ranks the first teacher's top prediction, ensuring the student model focuses on pixels where both teachers agree.
2. Multi-Phase Training
Instead of training on all unlabeled data simultaneously, RWMS uses a curriculum-based approach:
- Stage 1: The model trains on labeled data and the top 50% most reliable unlabeled images (as determined by image-level MIoU).
- Stage 2: Having established a solid foundation, the model generates higher-quality pseudo-labels for the remaining "unreliable" images and trains on the full dataset.
3. Strong Data Augmentation
To prevent the student from simply memorizing the teacher's output, aggressive augmentations—such as color jitter, blur, and grayscale transformations—are applied to unlabeled images. This forces the model to learn robust, invariant features rather than superficial patterns.
Experimental Results
The researchers evaluated RWMS on two standard benchmarks: Pascal VOC 2012 and Cityscapes.
- Pascal VOC: With only 1/16th of the labels (~660 images), RWMS achieved 72.2% MIoU, a 3.4% improvement over the baseline. It consistently outperformed other state-of-the-art methods like DCC and ELN across various data splits.
- Cityscapes: On the 1/30th labeling split, RWMS outperformed the DMT method by a substantial 6%, demonstrating strong generalization in complex urban environments.
- Qualitative Success: Visual analysis indicates that RWMS produces significantly cleaner object boundaries and reduces "speckle" noise in complex regions compared to standard supervised or baseline semi-supervised approaches.
Conclusion and Future Work
RWMS effectively addresses the noise problem in self-training by using dual-teacher consensus as a proxy for truth. While the method requires an initial overhead to train the teacher models and involves hyperparameters for pixel weighting, its performance gains are significant. Future research may explore automated ways to determine the reliability split dynamically, further optimizing the curriculum as the student model evolves.
Summary updated