Abstract
ControlHair is a hybrid framework that bridges the gap between deterministic physics simulators and photorealistic video diffusion models. By encoding complex physics parameters into per-frame geometry signals, it enables fine-grained control over hair movement in generated videos, addressing a significant limitation in current text-to-video models where physical dynamics are often imprecise or uncontrollable.
Introduction
Generating realistic hair in video remains a "final boss" challenge in computer graphics due to the sheer complexity of individual strands, material properties, and light-scattering effects. While modern video diffusion models produce stunning visuals, they lack precise physical "knobs." Text prompts are insufficient for specifying exact wind velocities or hair stiffness. ControlHair solves this by using a physics simulator as an intermediate "translator" between user intent and video generation.
Methodology
The ControlHair pipeline consists of three stages that decouple physical reasoning from visual rendering:
- Simulator-based Physics Encoding: A 3D hair model is estimated from a reference image and integrated into a physics simulator (e.g., Blender). Users can define explicit parameters such as mass, stiffness, damping, and wind. The simulator then generates precise 3D hair geometry for every frame.
- Per-frame Control Signal Extraction: The 3D strands are projected into 2D "strand maps" that encode orientation and shape, alongside sparse human poses. This creates a geometric "bridge" that the diffusion model can easily interpret.
- Conditioned Video Diffusion: These control signals and the reference image are fed into a video diffusion backbone (based on Wan 2.1), fine-tuned via LoRA to maintain identity and texture while following the geometric guidance.
Key Results and Insights
ControlHair significantly outperforms existing models in both precision and quality. Quantitatively, it achieved a PSNR of 17.15 in reconstruction tasks, compared to 11.65 for text-conditioned Wan 2.1 and 15.78 for pose-conditioned UniAnimate-DiT.
A key finding is the model's ability to "self-correct." Even if the initial 3D hair estimation is slightly off, the diffusion model uses the reference image to adjust hair length while still adhering to the simulated motion. This decoupling allows the model to be simulator-agnostic; the diffusion backbone doesn't need retraining if the physics engine is swapped.
Applications and Future Work
The framework enables several novel applications, including dynamic hairstyle try-ons, "bullet-time" effects where the camera rotates around frozen hair physics, and cinemagraphs where only the hair is animated.
Despite its success, the model's performance is tied to the quality of the initial 3D estimation and the physics simulator. Future work aims to incorporate multi-view priors for better 3D "grooming" and more advanced, data-driven simulators to further enhance motion realism.
Summary updated