TL;DR
This paper introduces a novel data-driven approach for reconstructing realistic 3D hair models from imperfect input data, even with significant outliers. By leveraging a database of simulated hair strands as structural priors, their method achieves robust and plausible hair capture, outperforming existing techniques in challenging scenarios.
What Problem Does This Paper Address?
Creating realistic 3D hair for digital characters is a notoriously difficult task. Unlike smooth objects, hair has incredibly fine, intricate structures that are prone to self-occlusion and vary wildly in style. Existing methods often rely on heuristics or manual cleanup, which are time-consuming and struggle with complex hairstyles, occlusions, or noisy input data (like point clouds from multi-view stereo). The core challenge is to reconstruct structurally plausible hair that accurately reflects the input while being robust to imperfections.
Methods
The authors propose a "data-driven hair capture framework" that uses simulated hair strands as a guide. Here's a breakdown of their approach:
- Input Processing: They start with multi-view images and reconstruct a 3D point cloud and an initial 3D orientation field.
- Cover Strand Generation: Based on the orientation field, they generate "cover strands" that roughly follow the hair's flow but are often short and disconnected due to input noise.
- Simulated Example Database: This is a key innovation. They generate a database of structurally plausible hair strands using a physics-based simulation (Super-Helices model). Crucially, these examples don't need to perfectly match the input; they serve as structural references. They optimize the simulation parameters and leverage symmetry to keep the database size manageable.
- Strand Fitting (Hough Transform Inspired): They use a voting-based fitting algorithm, inspired by the Hough transform, to match the cover strands to the simulated example strands. This process identifies structurally plausible configurations by clustering votes in a transformation space. This step is robust to outliers and doesn't require manual cleanup of the input point cloud.
- Ribbon Connection and Synthesis: The matched example strands guide the connection of cover strands into "ribbons," which are then assembled into "wisps." Finally, these wisps are used to synthesize the complete, realistic hair strands. For constrained hairstyles like ponytails, they allow users to provide a few sketch strokes to generate example strands, bypassing the need for simulation.
Datasets
The paper doesn't introduce new datasets but evaluates their method on various real-world hair capture scenarios, including:
- Set A: Complex, curly hairstyles captured with 66 synchronized cameras.
- Set B: Wigs with crossing wisps and varying strand lengths captured with 45 cameras.
- Set C: A challenging wig example from a previous state-of-the-art method's dataset.
- Ponytail Example: Demonstrates their approach with user-sketched input.
They also compare their results against the state-of-the-art method by Luo et al. [2013a] and a direct fitting approach from Bradley et al. [2013].
Key Insights and Ideas
- Simulated Examples as Structural Priors: The core idea is that a database of structurally plausible simulated strands, rather than perfectly matched ones, can guide reconstruction. This makes the method robust to noisy input.
- Voting-Based Fitting for Robustness: Using a Hough-transform-like voting scheme in a transformation space effectively clusters cover strands that align similarly to example strands, inherently handling outliers and occlusions.
- Bridging the Gap Between Cover Strands and Final Hair: The method provides a principled way to connect fragmented cover strands into coherent wisps using the structural information from the matched example strands.
- User Sketches for Constrained Styles: Allowing simple 2D sketches for specific hairstyles (like ponytails) offers a practical alternative to complex simulations when boundary conditions are critical.
Key Results
- Robustness to Outliers: The method successfully reconstructs complex hairstyles from unprocessed point clouds with significant outliers, outperforming Luo et al. [2013a] in challenging cases (Figure 1f vs. 1g).
- Structural Plausibility: The synthesized hair exhibits compelling ringlet structures and plausible inter-wisp connections, even for highly convoluted hairstyles.
- Efficiency: While the fitting step can take time (10-40 minutes), the overall pipeline is significantly less labor-intensive than manual cleanup methods. The simulation database generation is also relatively fast (5 minutes for 256 examples).
- Generality: The framework handles a wide range of hairstyles, from curly to straight, and can adapt to user input for constrained styles.
Limitations and Future Work
- Uniform Natural Stiffness/Curliness: The current simulation model assumes uniform physical properties, limiting its ability to capture hairstyles with artificial styling (perming, gluing). Future work could extend the simulation model to handle varying physical parameters.
- Fitting Bottleneck: The strand fitting step is computationally intensive. Future research aims to accelerate this process using more efficient shape analysis and matching algorithms.
- Limited Manual Input: While the method requires minimal user input, it's not fully automatic.
- Future Directions: The authors suggest exploring procedural priors for more complex styles (braids, updos), investigating methods for automatic inference of realistic simulation parameters, and developing cost-effective sensing devices for wider deployment. They also envision a large, general database of hairstyles for future learning and classification tasks.
Summary updated