Effects of chromatic image statistics on illumination induced color differences

Marcel P. Lucassen, Theo Gevers, Arjan Gijsenij, Niels DekkerJournal of the Optical Society of America A 2013 · DOI

TL;DR

This paper introduces a novel psychophysical method to measure human color fidelity under varying illuminations, focusing on the entire scene rather than individual objects. They found that the orientation of a scene's chromatic distribution significantly impacts perceived color fidelity, with better results when the illuminant change aligns with the scene's dominant chromatic axis. This work offers insights for improving computational color constancy algorithms by considering scene statistics.

What Problem Does This Paper Address?

Color constancy, the ability of our visual system to perceive stable object colors despite changes in illumination, is a long-standing research topic. However, a significant gap exists between how human color constancy is studied (focusing on individual object appearances) and how computational approaches are evaluated (often by estimating illuminant accuracy). This disconnect makes it difficult to directly compare and improve computational models based on human perception. The authors aim to bridge this gap by developing a psychophysical method that assesses the global color fidelity of an entire scene under different illuminations, providing a more direct link to computational methods that process entire images.

Methods

The core of the study is a new triad illuminant comparison method. Instead of comparing just two illuminations, observers are presented with a scene rendered under three illuminants simultaneously: one neutral reference and two chromatic test illuminants. They are asked to choose which of the two test illuminants yields better color fidelity compared to the reference. This allows for a more nuanced assessment of how different illuminations affect the overall appearance of a scene.

To model observer responses, the authors propose four computational models that vary in their incorporation of visual system processing:

  1. Reflected Light Signal: A purely physical measure based on the spectral power distribution of light.
  2. Chromatic Overlap: Measures the similarity of color gamuts rendered under test and reference illuminations.
  3. Scene Averaged Color Difference: Calculates the average color difference (using CIELAB, CIE94, and CIEDE2000 metrics) between corresponding image parts under test and reference illuminations.
  4. Color Image Quality Metrics: Utilizes spatial extensions of color difference metrics like S-CIELAB and a novel Toet-Lucassen metric that considers luminance, a, and b channels.

Datasets

The study utilizes three distinct datasets:

  • Data Set 1: Multicolor Test Scenes: Synthesized images composed of 900 color patches arranged in a matrix. These scenes have controlled ellipsoidal chromatic distributions in CIELAB space, with varying orientations. This dataset allows for precise manipulation of chromatic statistics.
  • Data Set 2: Natural Images: 25 natural images selected from a larger database. These images were processed to have chromatic distributions with specific orientations, mimicking Data Set 1 but with more complex, natural scene statistics.
  • Data Set 3: Scrambled Natural Images: Pixelated and spatially scrambled versions of the natural images from Data Set 2. This dataset was created to isolate the effect of chromatic distribution from image semantics (object recognition, scene meaning).

Key Insights and Ideas

  • Scene-Level Fidelity Matters: Perceived color fidelity is not just about individual colors but how the entire scene's color palette holds up under different lighting.
  • Chromatic Distribution Orientation is Key: The orientation of the scene's chromatic distribution (how colors are spread out in color space) significantly impacts how well colors are perceived under different illuminations.
  • Alignment is Optimal: Color fidelity is best when the direction of the illuminant change (e.g., from neutral to red) aligns with the dominant axis of the scene's chromatic distribution.
  • Semantics are Less Crucial Than Statistics: Scrambling natural images to remove semantic meaning didn't drastically alter the results, suggesting that the underlying chromatic statistics are more influential than object recognition.
  • Natural Scenes Favor Yellow/Blue: For natural scenes, yellow and blue illuminants generally led to higher perceived color fidelity compared to red and green, potentially indicating a preference for illuminant changes along the daylight locus.

Key Results

  • Multicolor Scenes: For synthesized scenes, color fidelity was highest when the illuminant change vector was parallel to the major axis of the scene's chromatic distribution. Red and blue illuminants showed higher fidelity than yellow and green for certain orientations.
  • Natural Scenes: Yellow and blue illuminants generally resulted in higher color fidelity than red and green. Scrambling natural images showed that semantic content had a limited impact on these findings.
  • Model Performance: The scene-averaged color difference models (particularly CIEDE2000) performed best in predicting observer responses, achieving up to 77% accuracy in predicting the observer's choice. The Reflected Light Signal model performed poorly, while Chromatic Overlap and S-CIELAB showed moderate performance.

Limitations and Future Work

  • Observer Variability: The authors acknowledge that observer variability exists, setting an upper limit on model performance.
  • Model Simplification: The models, while incorporating visual system aspects, are still simplifications. The comparable performance of different model types (e.g., Chromatic Overlap vs. AE00) suggests that the exact mechanism of visual processing remains complex and not fully captured.
  • Methodological Improvements: Future work could involve increasing the number of test illuminants to refine measurements and exploring successive presentation modes, which have shown promise in other color constancy studies.
  • Algorithm Tuning: The findings suggest that color constancy algorithms could be improved by incorporating knowledge of scene chromatic distributions, potentially leading to more perceptually accurate color correction.

Summary updated

Abstract

We measure the color fidelity of visual scenes that are rendered under different (simulated) illuminants and shown on a calibrated LCD display. Observers make triad illuminant comparisons involving the renderings from two chromatic test illuminants and one achromatic reference illuminant shown simultaneously. Four chromatic test illuminants are used: two along the daylight locus (yellow and blue), and two perpendicular to it (red and green). The observers select the rendering having the best color fidelity, thereby indirectly judging which of the two test illuminants induces the smallest color differences compared to the reference. Both multicolor test scenes and natural scenes are studied. The multicolor scenes are synthesized and represent ellipsoidal distributions in CIELAB chromaticity space having the same mean chromaticity but different chromatic orientations. We show that, for those distributions, color fidelity is best when the vector of the illuminant change (pointing from neutral to chromatic) is parallel to the major axis of the scene’s chromatic distribution. For our selection of natural scenes, which generally have much broader chromatic distributions, we measure a higher color fidelity for the yellow and blue illuminants than for red and green. Scrambled versions of the natural images are also studied to exclude possible semantic effects. We quantitatively predict the average observer response (i.e., the illuminant probability) with four types of models, differing in the extent to which they incorporate information processing by the visual system. Results show different levels of performance for the models, and different levels for the multicolor scenes and the natural scenes. Overall, models based on the scene averaged color difference have the best performance. We discuss how color constancy algorithms may be improved by exploiting knowledge of the chromatic distribution of the visual scene.
BibTeX
@article{lucassen2013effects,
  author = {Marcel P. Lucassen and Theo Gevers and Arjan Gijsenij and Niels Dekker},
  title = {{Effects of chromatic image statistics on illumination induced color differences}},
  journal = {Journal of the Optical Society of America A},
  year = {2013},
  doi = {10.1364/josaa.30.001871},
}

Loading related content...

Figure