Total Selfie: Generating Full-Body Selfies

Bowei Chen, Brian Curless, Ira Kemelmacher-Shlizerman, Steven M. Seitz

Abstract

Traditional selfies are limited by arm length, resulting in perspective distortion and narrow fields of view. We present "Total Selfie," a diffusion-based pipeline that synthesizes high-quality, full-body photographs from four close-up selfies (face, upper body, lower body, and shoes). By leveraging synthetic data training and per-capture fine-tuning, our system produces professional-grade third-person photos with realistic poses and undistorted features.

Introduction

Despite their ubiquity, selfies suffer from fundamental technical flaws: a restricted field of view, unnatural perspective distortion of facial features, and awkward compositions where one arm is often excluded. "Total Selfie" bridges the gap between a casual selfie and a professional third-person photograph, enabling users to generate full-body images in any pose and setting without a tripod or external photographer.

Methodology

The core of "Total Selfie" is an exemplar-based inpainting task powered by a multi-stage pipeline designed to handle the unique distortions of close-up photography.

Synthetic Data Generation

To overcome the lack of paired selfie/full-body datasets, the authors generated a synthetic dataset using Stable Diffusion. By applying homography transformations to warp sections of full-body images, they simulated the perspective and framing of real selfies. This allowed for the training of a robust selfie-conditioned inpainting model.

The Test-Time Pipeline

When processing user inputs, the system follows several critical steps:

  1. Face Undistortion: Close-up face shots are "flattened" using a model trained on 3D GAN-generated heads to correct perspective distortion.
  2. Automatic Target Pose Selection: The system identifies reference photos from the user's gallery with similar clothing to ensure the generated body proportions are anatomically accurate for that specific attire.
  3. Per-Capture Fine-Tuning: Similar to DreamBooth, the model is fine-tuned on the specific set of selfies to preserve the user's unique identity and clothing textures.
  4. Appearance Refinement: SDEdit and DreamBooth-based refinement are used to sharpen critical details like faces and shoes, which are often blurred in standard diffusion outputs.
Overview of the Total Selfie pipeline, illustrating the transition from input selfies and background to the final generated output via target pose selection.

Key Insights

A pivotal discovery in this work is the importance of face undistortion; correcting the "fisheye" effect of close-up lenses is essential for realism. Additionally, the authors found that clothing-aware pose selection significantly improves anatomical consistency, as different outfits (e.g., hoodies vs. tight shirts) drastically change perceived body shape. Finally, using Canny Edges for guidance proved superior to skeleton-based methods for capturing the natural drape and folds of clothing.

Evaluation and Results

"Total Selfie" was evaluated against state-of-the-art baselines like Paint-By-Example and DreamBooth. Quantitatively, the method outperformed all competitors across key metrics, achieving an LPIPS of 0.187, SSIM of 0.708, and FID of 139.4.

Two examples of generated full-body selfies showing realistic poses, consistent clothing textures, and high-fidelity facial details.

Qualitatively, "Total Selfie" excels at preserving specific facial expressions and clothing textures while seamlessly integrating the user into complex backgrounds with realistic lighting and shadows. Unlike baseline models that often produce generic features, this approach maintains the user's specific identity.

Limitations and Future Work

Current challenges include generating accurate "hard" shadows in direct sunlight due to a lack of 3D scene understanding. Furthermore, inferring exact body scale from disconnected selfies remains difficult. Future research will focus on automatic pose suggestions based on background context and improving 3D body shape inference from 2D inputs.

Summary updated

Abstract

We present a method to generate full-body selfies from photographs originally taken at arms length. Because self-captured photos are typically taken close up, they have lim-ited field of view and exaggerated perspective that distorts facial shapes. We instead seek to generate the photo some one else would take of you from a few feet away. Our approach takes as input four selfies of your face and body, a background image, and generates a full-body selfie in a de-sired target pose. We introduce a novel diffusion-based approach to combine all of this information into high-quality, well-composed photos of you with the desired pose and background.
BibTeX
@misc{chen2023total,
  author = {Bowei Chen and Brian Curless and Ira Kemelmacher-Shlizerman and Steven M. Seitz},
  title = {{Total Selfie: Generating Full-Body Selfies}},
  year = {2023},
  eprint = {2308.14740},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.14740},
}
Figure