Introduction
Capturing high-quality "fit check" videos—short clips showcasing an outfit from multiple angles—is a popular social media trend that remains technically challenging. Traditional methods require tripods, specific framing, and multiple takes to achieve professional results. While mirror selfies are the most convenient way to document an outfit, they are limited by camera visibility and static poses. Researchers from the University of Washington have developed a system that transforms simple mirror selfies into dynamic, full-body videos using a phone's internal sensors, eliminating the need for complex setups.
Methodology
The proposed pipeline separates appearance, motion, and environment, then recomposes them using a novel video diffusion model. The process begins with the user capturing two mirror selfies (front and back) and a brief handheld motion recorded via the phone’s sensors.
Motion and Background Retrieval
Estimating full-body motion from a shaky handheld camera is a notoriously noisy problem. To solve this, the system leverages the phone's Inertial Measurement Unit (IMU). By recording the phone's orientation and translation, the system uses Dynamic Time Warping (DTW) to retrieve a matching professional motion sequence from a pre-existing database. It then selects a background image with a ground plane that aligns with the retrieved motion's orientation.
Multi-Reference Video Diffusion
To ensure the generated person accurately reflects the user's outfit from all sides, the model utilizes both front and back mirror selfies through several key innovations:
- The "Second Frame" Trick: The back-view selfie is treated as the second frame of the video sequence during inference. This leverages the diffusion model’s inherent temporal consistency to propagate clothing details from the back view throughout the generated video.
- Multi-Reference Attention: The researchers modified the spatial self-attention layers of the denoising network to concatenate features from both reference images. This allows the model to "look" at both the front and back of the person simultaneously while generating any given frame.
- Image-Based Fine-Tuning: To address the common issue of blurriness in video diffusion, the model was fine-tuned on high-quality synthetic data. This stage specifically enhances sharpness and the generation of realistic shadows and reflections for better human-scene composition.
Key Results
The proposed method significantly outperformed state-of-the-art baselines like Animate Anyone and Champ across multiple metrics. On a custom self-captured dataset, it achieved an LPIPS of 0.381 and a PSNR of 16.80, demonstrating superior image quality and temporal consistency. Qualitative results showed the model is uniquely capable of rendering sharp text on clothing and realistic ground reflections, which are common failure points for existing models. In a user study, participants rated the method highest for garment accuracy and overall realism.
Limitations and Future Work
While highly effective, the model sometimes misjudges lighting directions in outdoor scenes, leading to misaligned shadows. Extremely intricate textures or complex hand movements can also result in slight blurring. Furthermore, because the IMU only tracks the hand holding the device, the system currently cannot model independent arm movements. Future iterations may integrate data from smartwatches to capture a broader range of motion.
Summary updated