TL;DR
This paper introduces a novel method for estimating hair appearance parameters from a single flash photograph. By framing the problem as an image retrieval task, it allows for the creation of realistic hair renderings in new lighting conditions with minimal user input, making it ideal for applications like character customization in games.
What Problem Does This Paper Address?
Creating realistic hair in computer graphics is challenging due to the complex interplay of light scattering, material properties, and fine geometric details. While advanced rendering models exist, manually tuning the numerous parameters to match a specific person's hair appearance is a time-consuming and error-prone process, even for experts. Existing image-based rendering methods often require complex capture setups with multiple cameras and lights. This paper aims to bridge this gap by enabling the estimation of hair appearance from a single, easily captured flash photograph, significantly simplifying the process for users and applications.
Methods
The core of the method involves recasting hair appearance estimation as an image retrieval problem.
- Hair Appearance Model: They propose a model with six free parameters: two melanin concentrations (for hair color), three reflectance lobe widths (controlling specular highlights and glints), and a geometric noise parameter (to avoid a "plastic-like" appearance).
- Melanin Model: They leverage biological knowledge of eumelanin and pheomelanin to create a more constrained and realistic absorption model, reducing the parameter space for hair color.
- Geometric Noise: A perturbation model is introduced to add small-scale variations to hair geometry, enhancing realism.
- Feature and Distance Metric: A luminance-weighted color distribution in the Lab color space is used as an image feature. The Earth Mover's Distance (EMD) is employed as a metric to compare these distributions, measuring the "work" required to transform one distribution into another. This metric was found to be perceptually validated.
- Dataset and Retrieval: A large dataset of rendered hair images with known appearance parameters is precomputed for various hairstyles. Given an input photo, its feature is computed, and the method searches this dataset for the rendered image with the closest feature match. The parameters of this best-matching image are then used as the estimated parameters for the input photo.
Datasets
- Synthetic Dataset: The authors generated a synthetic dataset consisting of precomputed tuples of (feature, parameters) for rendered hair images. This dataset was created by sampling the parameter space for seven different hairstyle geometries (three modeled in Maya, four captured from existing data). Datasets contained 4000-5000 sample points each.
- Input Photographs: The method was tested on a set of 64 input flash photographs of real hair.
Key Insights and Ideas
- Image Retrieval for Appearance: Framing appearance estimation as an image retrieval problem is a powerful simplification. Instead of directly solving for parameters, it leverages a precomputed database of known appearances.
- Perceptual Validation of Metric: The Earth Mover's Distance on luminance-weighted color distributions was shown to align well with human perception of hair similarity, a crucial step for a user-facing application.
- Melanin Model for Realism: Incorporating biological constraints on hair pigmentation significantly improves the realism and reduces the complexity of estimating hair color.
- Geometric Noise for Realism: The addition of a simple geometric noise parameter is surprisingly effective at removing an unrealistic "plastic-like" appearance from rendered hair.
- Robustness to Lighting: The method demonstrates robustness to variations in lighting conditions, a key advantage for single-photo capture.
Key Results
- The method successfully estimates hair appearance parameters from a single flash photograph, producing visually similar renderings.
- A perceptual evaluation showed that the EMD metric's predictions of similarity between renderings and photos were agreed upon by human observers with high accuracy (71.4% to 93.6% agreement across different similarity categories).
- The appearance estimation process takes approximately 20 seconds per photo using a dataset of 5000 images.
- The method was demonstrated on 64 photographs, with the best-match EMD distance ranging from 1.35 to 6.13.
Limitations and Future Work
- Limitations:
- The model does not account for dyed hair, sun-bleached or dyed highlights, or partial gray hair, which violate the constant absorption assumption.
- Visible scalp can affect estimations.
- The model does not handle wet or greasy hair.
- The current rendering implementation struggled with front-lit blond hair, possibly due to approximations in the hair model.
- The hairstyle geometry is manually selected from a limited set.
- Future Work:
- Extend the method to handle unstructured lighting conditions by incorporating lighting and white-point estimation techniques.
- Incorporate meso-level geometry estimation (e.g., clumps and wisps) to improve realism.
- Develop a way to estimate hair appearance from one or more photos taken in unstructured lighting.
- The authors suggest that the proposed parameter estimation approach could be applicable to other graphical phenomena.
- Enlarging the library of hairstyles could improve final rendered likeness.
Summary updated