CDS: Cross-Domain Self-supervised Pre-training
Lists
Only you see this
Abstract
We present a two-stage pre-training approach that improves the generalization ability of standard single-domain pre-training. While standard pre-training on a single large dataset (such as ImageNet) can provide a good initial representation for transfer learning tasks, this approach may result in biased representations that impact the success of learning with new multi-domain data (e.g., different artistic styles) via methods like domain adaptation. We propose a novel pre-training approach called Cross-Domain Self-supervision (CDS), which directly employs unlabeled multi-domain data for downstream domain transfer tasks. Our approach uses self-supervision not only within a single domain but also across domains. In-domain instance discrimination is used to learn discriminative features on new data in a domain-adaptive manner, while cross-domain matching is used to learn domain-invariant features. We apply our method as a second pre-training step (after ImageNet pre-training), resulting in a significant target accuracy boost to diverse domain transfer tasks compared to standard one-stage pre-training.
BibTeX
@inproceedings{kim2021cds,
author = {Donghyun Kim and Kuniaki Saito and Tae-Hyun Oh and Bryan A. Plummer and Stan Sclaroff and Kate Saenko},
title = {{CDS: Cross-Domain Self-supervised Pre-training}},
booktitle = {IEEE/CVF International Conference on Computer Vision},
year = {2021},
doi = {10.1109/iccv48922.2021.00899},
}Loading related content...