Pull and concentrate: improving unsupervised semantic segmentation adaptation with cross- and intra-domain consistencies
Lists
Only you see this
Abstract
Unsupervised domain adaptation (UDA) is an important solution for the cross-domain problem in semantic segmentation. Existing segmentation UDA methods mainly consider the domain shift as the major challenge. This paper, from a novel viewpoint, disentangles the cross-domain problem into two negative factors beyond the domain shift. Specifically, we find that apart from the domain shift factor, the dispersed within-class distribution on the target domain is another factor that compromises cross-domain segmentation. This paper finds that the neglected target domain distribution dispersion is a challenge as crucial as the domain shift. In response to the joint of these two negative factors, we propose a “Pull-and-Concentrate” (PuCo) method comprised of two consistencies: (1) A cross-domain consistency “pulls” the source and target domain distribution (of the same class) close to each other based on a novel statistical style transfer. (2) An intra-domain consistency “concentrates” the within-class distribution on the target domain in a new unsupervised teacher-student method. Both consistencies have the advantage of being robust (or insulated) from pseudo-label noises. This advantage allows PuCo to bring consistent improvement over a battery of pseudo-label-based UDA methods. For example, on GTA5 to Cityscapes and SYNTHIA to Cityscapes, PuCo achieves \(60.3\%\) and \(57.2\%\) mean IoU, respectively. Code is available at https://github.com/Jarvis73/PuCo.
BibTeX
@inproceedings{zhang2023pull,
author = {Jian-Wei Zhang and Yifan Sun and Wei Chen},
title = {{Pull and concentrate: improving unsupervised semantic segmentation adaptation with cross- and intra-domain consistencies}},
booktitle = {Multim. Syst},
year = {2023},
url = {https://openreview.net/forum?id=jFwskeAZ9N},
}Loading related content...