Abstract
Traditional selfies are limited by arm length, resulting in perspective distortion and narrow fields of view. We present "Total Selfie," a diffusion-based pipeline that synthesizes high-quality, full-body photographs from four close-up selfies (face, upper body, lower body, and shoes). By leveraging synthetic data training and per-capture fine-tuning, our system produces professional-grade third-person photos with realistic poses and undistorted features.
Introduction
Despite their ubiquity, selfies suffer from fundamental technical flaws: a restricted field of view, unnatural perspective distortion of facial features, and awkward compositions where one arm is often excluded. "Total Selfie" bridges the gap between a casual selfie and a professional third-person photograph, enabling users to generate full-body images in any pose and setting without a tripod or external photographer.
Methodology
The core of "Total Selfie" is an exemplar-based inpainting task powered by a multi-stage pipeline designed to handle the unique distortions of close-up photography.
Synthetic Data Generation
To overcome the lack of paired selfie/full-body datasets, the authors generated a synthetic dataset using Stable Diffusion. By applying homography transformations to warp sections of full-body images, they simulated the perspective and framing of real selfies. This allowed for the training of a robust selfie-conditioned inpainting model.
The Test-Time Pipeline
When processing user inputs, the system follows several critical steps:
- Face Undistortion: Close-up face shots are "flattened" using a model trained on 3D GAN-generated heads to correct perspective distortion.
- Automatic Target Pose Selection: The system identifies reference photos from the user's gallery with similar clothing to ensure the generated body proportions are anatomically accurate for that specific attire.
- Per-Capture Fine-Tuning: Similar to DreamBooth, the model is fine-tuned on the specific set of selfies to preserve the user's unique identity and clothing textures.
- Appearance Refinement: SDEdit and DreamBooth-based refinement are used to sharpen critical details like faces and shoes, which are often blurred in standard diffusion outputs.
Key Insights
A pivotal discovery in this work is the importance of face undistortion; correcting the "fisheye" effect of close-up lenses is essential for realism. Additionally, the authors found that clothing-aware pose selection significantly improves anatomical consistency, as different outfits (e.g., hoodies vs. tight shirts) drastically change perceived body shape. Finally, using Canny Edges for guidance proved superior to skeleton-based methods for capturing the natural drape and folds of clothing.
Evaluation and Results
"Total Selfie" was evaluated against state-of-the-art baselines like Paint-By-Example and DreamBooth. Quantitatively, the method outperformed all competitors across key metrics, achieving an LPIPS of 0.187, SSIM of 0.708, and FID of 139.4.
Qualitatively, "Total Selfie" excels at preserving specific facial expressions and clothing textures while seamlessly integrating the user into complex backgrounds with realistic lighting and shadows. Unlike baseline models that often produce generic features, this approach maintains the user's specific identity.
Limitations and Future Work
Current challenges include generating accurate "hard" shadows in direct sunlight due to a lack of 3D scene understanding. Furthermore, inferring exact body scale from disconnected selfies remains difficult. Future research will focus on automatic pose suggestions based on background context and improving 3D body shape inference from 2D inputs.
Summary updated