Learning the eye of the beholder: Statistical modeling and estimation for personalized color perception

Xuanzhou Chen, Austin Xu, Jingyan Wang, Ashwin Pananjady

TL;DR

This paper introduces a novel statistical model for personalized color perception that unifies color-normal and color-blind individuals by modeling their unique color discrimination abilities. They propose an efficient data collection method using Perceptual Adjustment Queries (PAQs) and an algorithm to estimate a user's "copunctal point," which characterizes their color perception.

What Problem Does This Paper Address?

Traditional approaches to color vision deficiency (color blindness) rely on broad categories (e.g., red-green, blue-yellow) and standardized tests like Ishihara. However, empirical evidence suggests these categories are often oversimplifications of an individual's actual perception. The paper argues that color perception is highly individual and that existing classification methods fail to capture the fine-grained nuances of how people distinguish colors. This lack of personalization limits applications that aim to adapt to or correct for color vision differences. The core problem is to move beyond coarse classifications and develop a method to quantitatively model and estimate an individual's unique color perception profile.

Methods

The authors propose a unified model of color perception based on two key concepts:

  1. Copunctal Point: A central point in a color space from which "confusion lines" emanate. These lines represent directions where a user has difficulty distinguishing colors. For color-blind individuals, these confusion lines are a direct consequence of their specific deficiency.
  2. Confusion Ellipses: For any color, nearby colors within an ellipse are indistinguishable from that reference color. The major axis of this ellipse aligns with a confusion line.

Their approach involves a two-step process:

  • Metric Estimation: They use Perceptual Adjustment Queries (PAQs) to estimate a user's color discrimination ellipses. In a PAQ, a user adjusts a slider to find the first color on a path that is perceptibly different from a reference color. This is framed as a Mahalanobis metric learning problem.
  • Copunctal Point Estimation: Once ellipses (or more specifically, their major axes/confusion lines) are estimated for multiple reference colors, the copunctal point is estimated as the intersection of these lines. They develop an "error cone-based" estimator that accounts for noise and uncertainty in the metric estimates, providing theoretical guarantees on the accuracy of the copunctal point estimation.

Datasets

The paper does not explicitly mention using pre-existing public datasets for training or evaluation. Instead, they:

  • Simulate data: They generate synthetic data for numerical simulations to test their algorithms under controlled conditions, varying parameters like noise levels, eigenvalue gaps, and the number of reference colors.
  • Collect new data: They conducted a small user study, collecting data from 3 color-normal individuals and 1 color-blind individual. This data was used to fit chromaticity discrimination ellipses and visualize their findings.

Key Insights and Ideas

  • Unifying Model: The paper successfully unifies the concepts of confusion lines (associated with color blindness) and confusion ellipses (relevant to all individuals) under a single framework centered around the "copunctal point."
  • PAQs for Efficiency: Perceptual Adjustment Queries (PAQs) are shown to be a highly efficient and effective method for collecting color perception data, outperforming traditional triplet and paired comparison queries, especially under noisy conditions. This is a significant practical contribution for user studies.
  • Individualized Color Perception: The work strongly emphasizes that color perception is not a one-size-fits-all phenomenon. By estimating an individual's copunctal point, we can gain a much more nuanced understanding of their specific visual experience.
  • Theoretical Guarantees: The authors provide theoretical guarantees for their estimation algorithm, which is crucial for establishing the reliability of their approach.

Key Results

  • PAQ Advantage: Simulations demonstrated that PAQs significantly outperform noiseless triplet and paired comparison queries in estimating the copunctal point, even with moderate noise levels. This highlights the power of PAQs for practical data collection.
  • Error Bounds: The paper provides theoretical error bounds for copunctal point estimation, showing how accuracy depends on factors like noise variance, the number of measurements, and the "eigenvalue gap" of the estimated metrics.
  • Qualitative Visualization: Figure 3 provides compelling visualizations of estimated ellipses and confusion lines for individual users, illustrating how their color perception profiles differ, including noticeable differences for the color-blind individual.

Limitations and Future Work

  • Small User Study: The real-world data collection involved only four participants, which is a small sample size. Future work should involve larger-scale user studies to validate the findings across a broader population.
  • Computational Complexity: While the paper presents an algorithm, the computational complexity of solving the linear program for copunctal point estimation with a large number of reference points might be a consideration for real-time applications.
  • Generalizability of PAQ Model: The paper uses a specific noise model for PAQs. Exploring other noise models and their impact on estimation accuracy would be valuable.
  • Downstream Applications: The authors envision their work improving applications like image recoloring for color-blind individuals. Further research would be needed to integrate their personalized models into such applications and evaluate their effectiveness.

Summary updated

Abstract

Color perception has long remained an intriguing topic in vision and cognitive science. It is a common practice to classify a person as either "color-normal" or "color-blind", and that there are a few prevalent types. However, empirical evidence has repeatedly suggested that at best, categories for color-blindness only serve as approximations to real manifestations of it. To better understanding individual-level color perception, we propose a color perception model that unifies existing theories for color-normal and color-blind people, which posits a low-dimensional structure in color space according to which any given user distinguishes colors. We design an algorithm to learn this low-dimensional structure from user queries, and prove statistical guarantees on its performance. Taking inspiration from these guarantees, we design a novel data collection paradigm based on perceptual adjustment queries (PAQs) that efficiently infers a user’s color distinguishability profile from a small number of cognitively lightweight responses. In a host of simulations, PAQs offer significant advantages over the de facto method of collecting comparison-based similarity queries.
BibTeX
@inproceedings{chen2024learning,
  author = {Xuanzhou Chen and Austin Xu and Jingyan Wang and Ashwin Pananjady},
  title = {{Learning the eye of the beholder: Statistical modeling and estimation for personalized color perception}},
  booktitle = {Allerton},
  year = {2024},
  doi = {10.1109/allerton63246.2024.10735273},
}
Figure