Generating Fit Check Videos with a Handheld Camera

Bowei Chen, Brian Curless, Ira Kemelmacher-Shlizerman, Steven M. SeitzarXiv 2025 · DOI · arXiv

Introduction

Capturing high-quality "fit check" videos—short clips showcasing an outfit from multiple angles—is a popular social media trend that remains technically challenging. Traditional methods require tripods, specific framing, and multiple takes to achieve professional results. While mirror selfies are the most convenient way to document an outfit, they are limited by camera visibility and static poses. Researchers from the University of Washington have developed a system that transforms simple mirror selfies into dynamic, full-body videos using a phone's internal sensors, eliminating the need for complex setups.

Methodology

The proposed pipeline separates appearance, motion, and environment, then recomposes them using a novel video diffusion model. The process begins with the user capturing two mirror selfies (front and back) and a brief handheld motion recorded via the phone’s sensors.

System overview showing the transition from mirror selfies and IMU data to a generated video.

Motion and Background Retrieval

Estimating full-body motion from a shaky handheld camera is a notoriously noisy problem. To solve this, the system leverages the phone's Inertial Measurement Unit (IMU). By recording the phone's orientation and translation, the system uses Dynamic Time Warping (DTW) to retrieve a matching professional motion sequence from a pre-existing database. It then selects a background image with a ground plane that aligns with the retrieved motion's orientation.

Detail of the IMU-based motion and background retrieval process.

Multi-Reference Video Diffusion

To ensure the generated person accurately reflects the user's outfit from all sides, the model utilizes both front and back mirror selfies through several key innovations:

  • The "Second Frame" Trick: The back-view selfie is treated as the second frame of the video sequence during inference. This leverages the diffusion model’s inherent temporal consistency to propagate clothing details from the back view throughout the generated video.
  • Multi-Reference Attention: The researchers modified the spatial self-attention layers of the denoising network to concatenate features from both reference images. This allows the model to "look" at both the front and back of the person simultaneously while generating any given frame.
  • Image-Based Fine-Tuning: To address the common issue of blurriness in video diffusion, the model was fine-tuned on high-quality synthetic data. This stage specifically enhances sharpness and the generation of realistic shadows and reflections for better human-scene composition.

Key Results

The proposed method significantly outperformed state-of-the-art baselines like Animate Anyone and Champ across multiple metrics. On a custom self-captured dataset, it achieved an LPIPS of 0.381 and a PSNR of 16.80, demonstrating superior image quality and temporal consistency. Qualitative results showed the model is uniquely capable of rendering sharp text on clothing and realistic ground reflections, which are common failure points for existing models. In a user study, participants rated the method highest for garment accuracy and overall realism.

Limitations and Future Work

While highly effective, the model sometimes misjudges lighting directions in outdoor scenes, leading to misaligned shadows. Extremely intricate textures or complex hand movements can also result in slight blurring. Furthermore, because the IMU only tracks the hand holding the device, the system currently cannot model independent arm movements. Future iterations may integrate data from smartwatches to capture a broader range of motion.

Summary updated

Abstract

Self-captured full-body videos are popular, but most deployments require mounted cameras, carefully-framed shots, and repeated practice. We propose a more convenient solution that enables full-body video capture using handheld mobile devices. Our approach takes as input two static photos (front and back) of you in a mirror, along with an IMU motion reference that you perform while holding your mobile phone, and synthesizes a realistic video of you performing a similar target motion. We enable rendering into a new scene, with consistent illumination and shadows. We propose a novel video diffusion-based model to achieve this. Specifically, we propose a parameter-free frame generation strategy and a multi-reference attention mechanism to effectively integrate appearance information from both the front and back selfies into the video diffusion model. Further, we introduce an image-based fine-tuning strategy to enhance frame sharpness and improve shadows and reflections generation for more realistic human-scene composition.
BibTeX
@misc{chen2025generating,
  author = {Bowei Chen and Brian Curless and Ira Kemelmacher-Shlizerman and Steven M. Seitz},
  title = {{Generating Fit Check Videos with a Handheld Camera}},
  year = {2025},
  eprint = {2505.23886},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.23886},
}
Figure