Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation
Lists
Only you see this
Abstract
In this paper, we study the task of synthetic-to-real domain<br> generalized semantic segmentation, which aims to learn a model that<br> is robust to unseen real-world scenes using only synthetic data. The<br> large domain shift between synthetic and real-world data, including the<br> limited source environmental variations and the large distribution gap<br> between synthetic and real-world data, significantly hinders the model<br> performance on unseen real-world scenes. In this work, we propose the<br> Style-HAllucinated Dual consistEncy learning (SHADE) framework to<br> handle such domain shift. Specifically, SHADE is constructed based on<br> two consistency constraints, Style Consistency (SC) and Retrospection<br> Consistency (RC). SC enriches the source situations and encourages the<br> model to learn consistent representation across style-diversified samples.<br> RC leverages real-world knowledge to prevent the model from overfitting<br> to synthetic data and thus largely keeps the representation consistent<br> between the synthetic and real-world models. Furthermore, we present<br> a novel style hallucination module (SHM) to generate style-diversified<br> samples that are essential to consistency learning. SHM selects basis<br> styles from the source distribution, enabling the model to dynamically<br> generate diverse and realistic samples during training. Experiments show<br> that our SHADE yields significant improvement and outperforms stateof-<br> the-art methods by 5.05% and 8.35% on the average mIoU of three<br> real-world datasets on single- and multi-source settings, respectively.
BibTeX
@inproceedings{zhao2023style,
author = {Yuyang Zhao and Zhun Zhong and Na Zhao and Nicu Sebe and Gim Hee Lee},
title = {{Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation}},
booktitle = {Zenodo},
year = {2023},
doi = {10.5281/zenodo.7566098},
}Loading related content...