Real-time Decolorization using Dominant Colors

Wei Hu, Wei Li, Fan Zhang, Qian Du

TL;DR

This paper introduces a novel, real-time decolorization algorithm that leverages a "dominant color hypothesis" to preserve image contrast more effectively than traditional methods. By focusing on key color distributions rather than complex pixel-wise analysis, the method achieves impressive speedups, enabling high-resolution image and video processing on standard CPUs.

What Problem Does This Paper Address?

Decolorization, the conversion of color images to grayscale, is a fundamental task with applications ranging from printing to digital art. The challenge lies in creating a grayscale image that not only removes color but also preserves the original image's visual structure and contrast. Existing methods often fall into two camps: those that preserve color information well but are computationally expensive, and those that are fast but produce unsatisfactory results, failing to differentiate between similar colors or losing important visual cues. This paper aims to bridge this gap by offering a method that is both fast and effective at preserving color contrast.

Methods

The core of the proposed method is the "dominant color hypothesis." Instead of analyzing every pixel or local neighborhood, the authors hypothesize that the quality of a grayscale conversion can be evaluated by how well the grayscale values of the image's dominant colors preserve their original contrast.

  1. Dominant Color Descriptor (DCD): The algorithm first extracts dominant colors from an input image using a technique inspired by MPEG's DCD. This involves identifying key color clusters and their spatial distribution.
  2. Weight Optimization: Similar to existing methods, the algorithm explores various linear combinations of RGB channels (represented by weights $w_r, w_g, w_b$) to produce grayscale outputs. However, instead of exhaustively searching all possibilities, it focuses on a set of 66 candidate weight combinations.
  3. Dominant Color Contrast Metric ($\xi$): A new metric is introduced to evaluate the quality of each grayscale conversion. This metric quantifies the contrast between dominant color pairs in the grayscale output, aiming to maximize perceivable color changes.
  4. Local Optimum and Fusion: The method identifies "good" candidates by finding local maxima of the $\xi$ metric. For enhanced robustness, a multi-pass fusion technique is employed, where different parameter choices for DCD extraction are combined to generate a more comprehensive set of candidates.

Datasets

The paper evaluates its method using the publicly available color-to-gray benchmark dataset from Cadłk (2008). They also use images frequently found in related research papers for comparison.

Key Insights and Ideas

  • Shifting Focus from Pixels to Dominant Colors: The most significant insight is that focusing on the contrast preservation of dominant colors, rather than intricate pixel-level or local region analysis, can be a more efficient and effective strategy for high-quality decolorization.
  • Contrast Preservation as the Key: The authors emphasize that preserving visual structure and contrast is paramount, and their dominant color hypothesis directly addresses this by ensuring that significant color differences in the original image remain discernible in grayscale.
  • Real-time Performance through Simplicity: By avoiding computationally intensive operations like extensive bilateral filtering or complex optimization, the proposed method achieves remarkable speed, making it suitable for real-time applications.
  • "Good" Candidates for Flexibility: The generation of multiple "good" candidates provides flexibility, allowing users to select the best result or enabling automated selection for video processing.

Key Results

  • Real-time Performance: The method can process high-resolution images (e.g., 1080p) and videos in real-time on a common CPU. For instance, a 1080p HD video can be decolorized in real-time.
  • High-Quality Results: The proposed method achieves results that are competitive with, and often superior to, state-of-the-art methods like JBFD. In comparisons, 100% of JBFD's ground truth user-preferred results were included in the proposed method's output, with over 90% ranked in the top five.
  • Robustness: The method demonstrates robustness across various image types and transition types in videos, although some minor consistency issues in living videos are acknowledged.
  • Computational Efficiency: The computational cost is significantly reduced compared to methods like JBFD. For example, extracting edges and DCDs for a 720p image takes only a few milliseconds, and the overall computation for the metric is considered near constant time.

Limitations and Future Work

  • Empirical Nature of Dominant Color Hypothesis: The authors acknowledge that the dominant color hypothesis is empirical and might not work for all specific images, particularly when local contrast is critical and not well represented by dominant colors.
  • Potential for Missing Local Contrast: In some cases, the high-ranked results might fail to preserve local contrast, as seen in some examples. However, the generation of multiple "good" results mitigates this risk.
  • Future Work:
    • Improving DCD extraction techniques and refining the measurement metric for better performance and robustness.
  • Exploring advanced color descriptors to represent complex color images more accurately.
  • Developing more sophisticated searching strategies to further alleviate consistency problems in video decolorization.

Summary updated

Abstract

Decolorization is the process to convert a color image or video to its grayscale version, and it has received great attention in recent years. An ideal decolorization algorithm should preserve the original color contrast as much as possible. Meanwhile, it should provide the final decolorized result as fast as possible. However, most of the current methods are suffering from either unsatisfied color information preservation or high computational cost, limiting their application value. In this paper, a simple but effective technique is proposed for real-time decolorization. Based on the typical rgb2gray() color conversion model, which produces a grayscale image by linearly combining R, G, and B channels, we propose a dominant color hypothesis and a corresponding distance measurement metric to evaluate the quality of grayscale conversion. The local optimum scheme provides several "good" candidates in a confidence interval, from which the "best" result can be extracted. Experimental results demonstrate that remarkable simplicity of the proposed method facilitates the process of high resolution images and videos in real-time using a common CPU.
BibTeX
@misc{hu2014real,
  author = {Wei Hu and Wei Li and Fan Zhang and Qian Du},
  title = {{Real-time Decolorization using Dominant Colors}},
  year = {2014},
  eprint = {1404.2728},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1404.2728},
}
Figure