Training Vision Transformers for Semi-Supervised Semantic Segmentation
Lists
Only you see this
Abstract
We present S 4 Former, a novel approach to training Vision Transformers for Semi-Supervised Semantic Segmentation (S4). At its core, S 4 Former employs a Vision Transformer within a classic teacher-student framework, and then leverages three novel technical ingredients: PatchShuffle as a parameter-free perturbation technique, Patch-Adaptive Self-Attention (PASA) as a fine-grainedfeature modulation method, and the innovative Negative Class Ranking (NCR) regularization loss. Based on these regu-larization modules aligned with Transformer-specific char-acteristics across the image input, feature, and output di-mensions, S4Former exploits the Transformer's ability to capture and differentiate consistent global contextual information in unlabeled images. Overall, S4 Former not only defines a new state of the art in S4 but also maintains a streamlined and scalable architecture. Being readily compatible with existing frameworks, S4 Former achieves strong improvements (up to 4.9%) on benchmarks like Pascal VOC 2012, COCO, and Cityscapes, with varying numbers of labeled data. The code is at https://github.com/JoyHuYY1412/S4Former.
BibTeX
@inproceedings{hu2024training,
author = {Xinting Hu and Li Jiang and Bernt Schiele},
title = {{Training Vision Transformers for Semi-Supervised Semantic Segmentation}},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2024},
doi = {10.1109/cvpr52733.2024.00384},
}Loading related content...