Spectral sharpening by spherical sampling

Graham D. Finlayson, Javier Vázquez-Corral, Sabine Süsstrunk, María VanrellJournal of the Optical Society of America A 2012 · DOI

TL;DR

This paper introduces "Spherical Sampling," a novel method for finding optimal "sharp sensors" that improve color constancy and spectral calculations. By exploring the space of possible sensors on a sphere, it significantly outperforms previous methods in minimizing color differences and maximizing color ratio stability.

What Problem Does This Paper Address?

The paper tackles the challenge of color constancy, which is the ability of a visual system (or camera) to perceive the color of an object consistently, regardless of changes in illumination. A common model for this is the "von Kries adaptation," which assumes that illumination changes can be handled by scaling sensor responses. However, this model works best when sensors have narrow spectral support, leading to the concept of "sharp sensors." The problem is that finding these optimal sharp sensors, especially when optimizing for perceptually relevant metrics like color difference (CIE Delta E) or color ratio stability, is computationally difficult. Existing methods often rely on linear error metrics or are local optimizers, failing to guarantee the best possible solution. This paper aims to provide a principled and comprehensive approach to finding these optimal sharp sensors for various error criteria.

Methods

The core innovation is Spherical Sampling. The authors represent possible "sharp sensors" as points on the surface of a 3D sphere. By randomly sampling points on this sphere and evaluating the performance of the resulting sensor sets, they can explore a vast number of potential sharp sensor configurations. This method is then applied to optimize two key error functions:

  • CIE Delta E Minimization: This aims to find sharp sensors that minimize the perceptual difference between colors under different illuminations, using the CIE Lab color space.
  • Color Ratio Stability Maximization: This seeks sharp sensors that make color ratios (e.g., the ratio of red to green sensor responses) invariant to changes in illumination, a concept important in color invariant tasks.

The paper also extends this to the multispectral case, where they aim to find a basis set that simplifies spectral calculations and improves perceptual spectral recovery using a "factor model."

Datasets

The paper utilizes several datasets for evaluation:

  • 1995 reflectances from Barnard et al. [9]: This is a standard dataset for color research.
  • Illuminant Datasets: Four different illuminant datasets are used, including:
  • The 102 illuminant set from SFU [9].
  • Subsets of the SFU dataset (Dataset 2 with 11 lights, Dataset 3 with daylights).
  • A k-means clustered set of 10 illuminants derived from the 102 illuminants (Dataset 4).
  • Vrhel's 170 measured reflectances [29]: Used in the multispectral experiments.

Key Insights and Ideas

  • Spherical Sampling as a Global Optimizer: The key insight is that by representing sensors as points on a sphere and sampling it, the method can effectively explore the entire space of possible sharp sensors, overcoming the limitations of local optimization methods.
  • Perceptual Metrics are Crucial: The paper strongly advocates for optimizing directly for perceptually relevant metrics like CIE Delta E, rather than relying on simpler linear error measures, leading to more visually meaningful results.
  • Sharp Sensors Improve Color Constancy Significantly: The results demonstrate that optimizing for color constancy using sharp sensors, particularly with the spherical sampling approach, yields substantial improvements over existing methods.
  • Color Ratio Stability is Linked to Sensor Shape: The study reveals that sensors that maximize color ratio stability tend to be broader-band with smaller negative lobes, intuitively preventing small perturbations from causing large ratio errors.
  • Multispectral Extension is Powerful: The spherical sampling approach is successfully extended to the multispectral domain, improving the selection of basis sets for spectral calculations and perceptual recovery.

Key Results

  • CIE Delta E Improvement: Spherical sampling consistently delivered significantly improved Delta E error across all four illuminant datasets, with improvements up to 50% compared to the Chong et al. method.
  • Color Ratio Stability: The method achieved better average ratio error compared to other sharpening methods, with some optimized sensors showing over 50% improvement.
  • Multispectral Spectral Recovery: In the 6D multispectral case, spherical sampling achieved a mean Delta E of 1.27 (compared to 2.60 for sensor-based sharpening), demonstrating a significant increase in performance.

Limitations and Future Work

  • Dimensionality and Computational Cost: The authors acknowledge that as the dimensionality of the optimization problem increases, spherical sampling, like other search-based methods, may only be useful for probing "local" solutions. They suggest that for higher dimensions, combinatorial sampling might become computationally infeasible.
  • Sampling Resolution: While a 1-degree sampling resolution was found to be sufficient for 3D color spaces, the paper notes that for higher dimensional problems (like the 6D multispectral case), a coarser sampling resolution was necessary due to computational constraints. Future work could explore more efficient sampling strategies for higher dimensions.
  • Further Exploration of Error Functions: The paper presents a general framework for optimizing arbitrary objective functions. Future work could explore other perceptual metrics or error criteria beyond CIE Delta E and color ratio stability.

Summary updated

Abstract

There are many works in color that assume illumination change can be modeled by multiplying sensor responses by individual scaling factors. The early research in this area is sometimes grouped under the heading "von Kries adaptation": the scaling factors are applied to the cone responses. In more recent studies, both in psychophysics and in computational analysis, it has been proposed that scaling factors should be applied to linear combinations of the cones that have narrower support: they should be applied to the so-called "sharp sensors." In this paper, we generalize the computational approach to spectral sharpening in three important ways. First, we introduce spherical sampling as a tool that allows us to enumerate in a principled way all linear combinations of the cones. This allows us to, second, find the optimal sharp sensors that minimize a variety of error measures including CIE Delta E (previous work on spectral sharpening minimized RMS) and color ratio stability. Lastly, we extend the spherical sampling paradigm to the multispectral case. Here the objective is to model the interaction of light and surface in terms of color signal spectra. Spherical sampling is shown to improve on the state of the art.
BibTeX
@article{finlayson2012spectral,
  author = {Graham D. Finlayson and Javier Vázquez-Corral and Sabine Süsstrunk and María Vanrell},
  title = {{Spectral sharpening by spherical sampling}},
  journal = {Journal of the Optical Society of America A},
  year = {2012},
  doi = {10.1364/josaa.29.001199},
}

Loading related content...

Figure