Abstract
The current golden standard of semi-supervised semantic segmentation is to generate and exploit pseudo-supervision on unlabeled images. This approach is however susceptible to the quality of pseudo-supervision—training often becomes unstable particularly at early stages and biased to incorrect supervision. To address these issues, we propose a new semi-supervised learning framework, dubbed Guided Pseudo Supervision (GPS). GPS comprises three networks, i.e., a teacher and two separate students. The teacher is first trained with a small set of labeled data and provides stable initial pseudo-supervision on the unlabeled data to the students. The students interactively train each other under the supervision of the teacher, and once they are sufficiently trained, they offer feedback supervision to the teacher so that the teacher improves in subsequent iterations. This strategy enables more stable and faster convergence than previous works, and consequently, GPS achieved state-of-the-art performance on Pascal VOC 2012 and Cityscapes datasets in various experiment settings.
Introduction
Semantic segmentation is a critical yet resource-intensive task in computer vision, requiring dense, pixel-level annotations that are costly to produce. Semi-supervised learning (SSL) offers a solution by leveraging a small amount of labeled data alongside a large volume of unlabeled data. The prevailing approach in SSL is pseudo-supervision, where a model generates labels for unlabeled data to serve as training targets. However, this method is often plagued by instability; early-stage errors can lead to a "garbage in, garbage out" cycle where the model reinforces its own mistakes. This paper introduces Guided Pseudo Supervision (GPS), a framework designed to stabilize this process through an interactive feedback loop between a teacher and multiple student networks.
The Guided Pseudo Supervision (GPS) Framework
The GPS architecture consists of one Teacher Network (TN) and two Student Networks (SNs). Unlike traditional teacher-student models where knowledge flows in one direction, GPS establishes a bidirectional relationship.
The training process is divided into several key phases:
- Teacher Initialization: The TN is pre-trained on the available labeled data to ensure it can provide stable initial guidance.
- Guided Pseudo Supervision: The TN generates pseudo-labels for unlabeled data, which serve as the primary supervision for the two SNs.
- Student Co-training: The two SNs, which are initialized differently, supervise each other by exchanging predictions. This cross-pseudo supervision ensures the students learn consistency beyond simply mimicking the teacher.
- Feedback Pseudo Supervision (FPS): Once the students reach a certain level of maturity, they provide feedback to the teacher. The teacher then updates its parameters based on the consensus of the students on unlabeled data.
- Adaptive Ramp-Up: To ensure the teacher only learns from reliable student feedback, an adaptive mechanism compares student and teacher performance on labeled data. The teacher only begins to "listen" to the students once their loss on the labeled set drops below its own.
Experimental Results
The GPS framework was evaluated on the Pascal VOC 2012 and Cityscapes benchmarks using various labeling ratios (from 1/16 to 1/2).
Faster Convergence and Efficiency
A standout feature of GPS is its training efficiency. While competing methods like PS-MT often require up to 300 epochs to reach peak performance, GPS achieves state-of-the-art (SOTA) results in significantly fewer iterations, typically between 32 and 60 epochs.
Performance and Data Efficiency
GPS outperformed major baselines, including CPS and PS-MT, across nearly all tested protocols. Notably, on the Pascal VOC dataset, a GPS model trained with only 50% of the labeled data outperformed a fully-supervised DeepLabv3+ model using 100% of the labels. This demonstrates the framework's ability to extract high-quality information from unlabeled data, effectively reducing the human labor required for dense annotation.
Conclusion
Guided Pseudo Supervision addresses the inherent instability of pseudo-labeling in semi-supervised semantic segmentation. By treating the teacher as an adaptable guide rather than a fixed authority, and by utilizing an adaptive ramp-up to gate student feedback, the framework achieves superior accuracy and faster convergence. While the system requires maintaining three networks, the resulting gains in data efficiency and training speed offer a compelling trade-off for large-scale segmentation tasks.
Summary updated