FastSwap: A Lightweight One-Stage Framework for Real-Time Face Swapping

Sahng-Min Yoo, Tae-Min Choi, Jae-Woo Choi, Jong-Hwan Kim

Abstract

Recent face swapping frameworks have achieved high-fidelity results. However, the previous works suffer from high computation costs due to the deep structure and the use of off-the-shelf networks. To overcome such problems and achieve real-time face swapping, we propose a lightweight one-stage framework, FastSwap. We design a shallow network trained in a self-supervised manner without any manual annotations. The core of our framework is a novel decoder block, called Triple Adaptive Normalization (TAN) block, which effectively integrates the identity and pose information. Besides, we propose a novel data augmentation and switch-test strategy to extract the attributes from the target image, which further enables controllable attribute editing. Extensive experiments on VoxCeleb2 and wild faces demonstrate that our framework generates high-fidelity face swapping results in 123.22 FPS and better preserves the identity, pose, and attributes than other state-of-the-art methods. Furthermore, we conduct an in-depth study to demonstrate the effectiveness of our proposal.
BibTeX
@inproceedings{yoo2023fastswap,
  author = {Sahng-Min Yoo and Tae-Min Choi and Jae-Woo Choi and Jong-Hwan Kim},
  title = {{FastSwap: A Lightweight One-Stage Framework for Real-Time Face Swapping}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00355},
}

Loading related content...

Figure