Strand-Accurate Multi-View Hair Capture

Giljoo Nam, Chenglei Wu, Min H. Kim, Yaser SheikhCVPR 2019 · OpenReview

Abstract

Hair is one of the most challenging objects to reconstruct due to its micro-scale structure and a large number of repeated strands with heavy occlusions. In this paper, we present the first method to capture high-fidelity hair geometry with strand-level accuracy. Our method takes three stages to achieve this. In the first stage, a new multi-view stereo method with a slanted support line is proposed to solve the hair correspondences between different views. In detail, we contribute a novel cost function consisting of both photo-consistency term and geometric term that reconstructs each hair pixel as a 3D line. By merging all the depth maps, a point cloud, as well as local line directions for each point, is obtained. Thus, in the second stage, we feature a novel strand reconstruction method with the mean-shift to convert the noisy point data to a set of strands. Lastly, we grow the hair strands with multi-view geometric constraints to elongate the short strands and recover the missing strands, thus significantly increasing the reconstruction completeness. We evaluate our method on both synthetic data and real captured data, showing that our method can reconstruct hair strands with sub-millimeter accuracy.

Introduction

Capturing the fine-grained geometry of human hair is a long-standing challenge in computer vision. While reconstruction of smooth surfaces like faces and bodies has reached high maturity, hair remains difficult due to its micro-scale structure, self-occlusions, and repetitive patterns. Standard Multi-View Stereo (MVS) algorithms typically assume the world is composed of small, locally flat planes. This "blobby" assumption fails for hair, which consists of thousands of thin, cylindrical strands, leading to noisy point clouds or smoothed-over "helmet hair" reconstructions.

Methodology

The proposed pipeline moves beyond surface-based primitives to treat hair as a collection of 3D line segments. The process consists of three main stages:

The three-stage pipeline: Line-based PatchMatch MVS, Hair Strand Reconstruction, and Hair Growing.
  1. Line-based PatchMatch MVS (LP-MVS): The core innovation is reformulating PatchMatch to use 3D line segments as the basic primitive. A novel cost function combines photo-consistency (color matching across views) with geometric-consistency (matching 3D line projections to 2D hair orientations detected via Gabor filters).
  2. Hair Strand Reconstruction: The initial output is a noisy cloud of 3D line segments. To refine this, a mean-shift-based fusion algorithm clusters these lines in a 4D space (position and direction), effectively "snapping" noisy data into clean, thin 3D curves.
  3. Multi-view Hair Growing: To address occlusions where strands are only partially visible, the method "grows" short segments by following detected 2D hair flow in the source images, ensuring the extended geometry remains constrained by the original photographs.

Results and Evaluation

The method was evaluated using both synthetic Maya/V-Ray renders and real-world data from a 70-camera capture rig.

Qualitative comparison showing the proposed method's ability to capture individual strands compared to the 'blobby' results of traditional COLMAP.

On synthetic data, the pipeline achieved sub-millimeter accuracy, with errors under 1.0mm for the vast majority of points. Qualitatively, the method excels at capturing "flying strands"—individual hairs that stick out from the main body—which are typically lost in standard reconstructions. Leave-one-out cross-validation confirmed that the reconstructed 3D strands align with pixel-level accuracy in unseen views.

Conclusion

By rethinking the fundamental primitives of 3D reconstruction from planes to lines, this work achieves unprecedented strand-level accuracy in hair capture. While challenges remain regarding scalp connectivity and inner-volume occlusion, this metric approach provides a significant leap over previous "plausible-looking" fitting methods, enabling high-fidelity digital hair for simulation and rendering.

Summary updated

Abstract

Hair is one of the most challenging objects to reconstruct due to its micro-scale structure and a large number of repeated strands with heavy occlusions. In this paper, we present the first method to capture high-fidelity hair geometry with strand-level accuracy. Our method takes three stages to achieve this. In the first stage, a new multi-view stereo method with a slanted support line is proposed to solve the hair correspondences between different views. In detail, we contribute a novel cost function consisting of both photo-consistency term and geometric term that reconstructs each hair pixel as a 3D line. By merging all the depth maps, a point cloud, as well as local line directions for each point, is obtained. Thus, in the second stage, we feature a novel strand reconstruction method with the mean-shift to convert the noisy point data to a set of strands. Lastly, we grow the hair strands with multi-view geometric constraints to elongate the short strands and recover the missing strands, thus significantly increasing the reconstruction completeness. We evaluate our method on both synthetic data and real captured data, showing that our method can reconstruct hair strands with sub-millimeter accuracy.
BibTeX
@inproceedings{nam2019strand,
  author = {Giljoo Nam and Chenglei Wu and Min H. Kim and Yaser Sheikh},
  title = {{Strand-Accurate Multi-View Hair Capture}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=lUY2Hu79xx},
}
Figure