Abstract
Colorization is the method of converting an image in grayscale to a fully color image. There are multiple methods to do the same. Old school methods used machine learning algorithms and optimization techniques to suggest possible colors to use. With advances in the field of deep learning, colorization results have improved consistently with improvements in deep learning architectures. The latest development in the field of deep learning is the emergence of generative adversarial networks (GANs) which is used to generate information and not just predict or classify. As part of this report, 2 architectures of recent papers are reproduced along with a novel architecture being suggested for general colorization. Following this, we propose the use of colorization by generating makeup suggestions automatically on a face. To do this, a dataset consisting of 1000 images has been created. When an image of a person without makeup is sent to the model, the model first converts the image to grayscale and then passes it through the suggested GAN model. The output is a generated makeup suggestion. To develop this model, we need to tweak the general colorization model to deal only with faces of people.
Introduction
Traditionally, image colorization was a manual, painstaking process involving "scribbling" colors onto regions or finding near-identical reference images. While deep learning has automated general colorization—turning old black-and-white photos into color—applying this technology to makeup suggestion presents a unique challenge. Existing makeup tools often require complex 3D facial modeling, expensive hardware, or manual user intervention. This research explores whether a fully automatic system can understand facial features well enough to suggest aesthetically pleasing makeup without human expertise in the loop.
Methods
The core innovation lies in treating makeup as a distribution of chrominance (color) over a luminance (brightness) map.
The Architecture
The proposed novel architecture improves upon previous models by replacing simple autoencoders with Generative Adversarial Networks (GANs). Specifically, DC-GANs are utilized for their ability to generate high-quality, realistic imagery.
Dual-Channel Prediction in LAB Space
Instead of predicting an RGB image directly, the model operates in the LAB color space. The 'L' channel (lightness) serves as the input. The system uses two separate DC-GANs: one to predict the 'A' channel (green-to-red) and one for the 'B' channel (blue-to-yellow). Using the LAB color space minimizes per-pixel correlation, making it easier for the neural network to learn colorization compared to the highly correlated RGB channels.
Feature Extraction
To provide contextual understanding, a DenseNet-161 (pretrained on ImageNet) is used as a backbone. For the specific makeup task, the model is refined to focus purely on facial features. The Generator creates the A and B channels, while a Discriminator attempts to distinguish the "fake" colorized face from a real photo with makeup. This adversarial training pushes the model toward more realistic results.
Datasets
The study utilizes two distinct datasets:
- ImageNet: A massive set of 600,000 images used to train and evaluate the model for general colorization tasks (landscapes, animals, etc.).
- Custom Makeup Dataset: The authors curated 1,000 high-resolution (256x256) images from Google and Flickr. These images were manually cropped to focus on key facial features (eyes, lips, skin) and selected to represent diverse ethnicities.
Results
The paper provides qualitative evidence of the model's success across general and specialized tasks.
General Colorization
The model successfully colorized complex scenes with high fidelity compared to ground truth images. Examples include natural landscapes and human activities, demonstrating the robustness of the underlying GAN architecture.
Makeup Generation
In facial tests, the model automatically applied distinct lipstick shades, eyeshadow, and skin-tone enhancements. The results appear natural and aligned with facial landmarks without requiring manual "hints." The model learns that "makeup" is essentially the delta between a bare face and a styled one in the chrominance space.
Conclusion and Future Work
This research demonstrates that makeup suggestion can be effectively framed as a specialized colorization task. While results are promising, future work could involve:
- Dataset Scaling: Expanding the 1,000-image dataset to improve GAN robustness and diversity.
- User Control: Allowing users to provide "hints" or select specific styles (e.g., "evening look") to guide the GAN's output.
- Quantitative Metrics: Implementing standardized metrics or user studies to measure the aesthetic quality of the suggested makeup.
Summary updated