ControlHair: Physically-based Video Diffusion for Controllable Dynamic Hair Rendering

Weikai Lin, Haoxiang Li, Yuhao ZhuarXiv 2025 · DOI · arXiv

Abstract

ControlHair is a hybrid framework that bridges the gap between deterministic physics simulators and photorealistic video diffusion models. By encoding complex physics parameters into per-frame geometry signals, it enables fine-grained control over hair movement in generated videos, addressing a significant limitation in current text-to-video models where physical dynamics are often imprecise or uncontrollable.

Introduction

Generating realistic hair in video remains a "final boss" challenge in computer graphics due to the sheer complexity of individual strands, material properties, and light-scattering effects. While modern video diffusion models produce stunning visuals, they lack precise physical "knobs." Text prompts are insufficient for specifying exact wind velocities or hair stiffness. ControlHair solves this by using a physics simulator as an intermediate "translator" between user intent and video generation.

Methodology

The ControlHair pipeline consists of three stages that decouple physical reasoning from visual rendering:

  1. Simulator-based Physics Encoding: A 3D hair model is estimated from a reference image and integrated into a physics simulator (e.g., Blender). Users can define explicit parameters such as mass, stiffness, damping, and wind. The simulator then generates precise 3D hair geometry for every frame.
  2. Per-frame Control Signal Extraction: The 3D strands are projected into 2D "strand maps" that encode orientation and shape, alongside sparse human poses. This creates a geometric "bridge" that the diffusion model can easily interpret.
Figure 2: A visualization of the control signals, including the 2D strand map and the sparse human pose skeleton used to guide the diffusion model.
  1. Conditioned Video Diffusion: These control signals and the reference image are fed into a video diffusion backbone (based on Wan 2.1), fine-tuned via LoRA to maintain identity and texture while following the geometric guidance.

Key Results and Insights

ControlHair significantly outperforms existing models in both precision and quality. Quantitatively, it achieved a PSNR of 17.15 in reconstruction tasks, compared to 11.65 for text-conditioned Wan 2.1 and 15.78 for pose-conditioned UniAnimate-DiT.

Figure 5: Qualitative results showing the generated video sequences (rows 1 and 3) alongside their corresponding driving strand maps (rows 2 and 4) for two different subjects.

A key finding is the model's ability to "self-correct." Even if the initial 3D hair estimation is slightly off, the diffusion model uses the reference image to adjust hair length while still adhering to the simulated motion. This decoupling allows the model to be simulator-agnostic; the diffusion backbone doesn't need retraining if the physics engine is swapped.

Applications and Future Work

The framework enables several novel applications, including dynamic hairstyle try-ons, "bullet-time" effects where the camera rotates around frozen hair physics, and cinemagraphs where only the hair is animated.

Despite its success, the model's performance is tied to the quality of the initial 3D estimation and the physics simulator. Future work aims to incorporate multi-view priors for better 3D "grooming" and more advanced, data-driven simulators to further enhance motion realism.

Summary updated

BibTeX
@misc{lin2025controlhair,
  author = {Weikai Lin and Haoxiang Li and Yuhao Zhu},
  title = {{ControlHair: Physically-based Video Diffusion for Controllable Dynamic Hair Rendering}},
  year = {2025},
  eprint = {2509.21541},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21541},
}
Figure