Robust histogram construction from color invariants for object recognition

Theo Gevers, H.m.g. StokmanIEEE TPAMI 2004 · DOI

Abstract

An effective object recognition scheme is to represent and match images on the basis of histograms derived from photometric color invariants. A drawback, however, is that certain color invariant values become very unstable in the presence of sensor noise. To suppress the effect of noise for unstable color invariant values, in this paper, histograms are computed by variable kernel density estimators. To apply variable kernel density estimation in a principled way, models are proposed for the propagation of sensor noise through color invariant variables. As a result, the associated uncertainty is obtained for each color invariant value. The associated uncertainty is used to derive the parameterization of the variable kernel for the purpose of robust histogram construction. It is empirically verified that the proposed density estimator compares favorably to traditional histogram schemes for the purpose of object recognition.

Introduction

Color is a powerful cue for computer vision because specific mathematical transformations—known as color invariants—allow systems to recognize objects regardless of shadows, highlights, or changes in illumination. However, these invariants have a "dark side": they become mathematically unstable at specific points.

For example, Hue is extremely sensitive when an object is nearly grey (low saturation), and normalized RGB values fluctuate wildly when an image is very dark (near the "black point"). In these regions, even minimal sensor noise can cause massive, random jumps in the calculated color value. Traditionally, researchers discarded this data using arbitrary thresholds. This paper introduces a more robust, mathematical framework to handle this uncertainty without discarding information.

Methodology: Modeling Noise Propagation

The authors move away from traditional "hard-bin" histograms toward Variable Kernel Density Estimation (VKDE). This approach is built on two main pillars:

  1. Noise Propagation Modeling: Using a first-order Taylor series expansion, the authors derived analytical formulas to predict the variance (uncertainty) of a color invariant based on the estimated sensor noise in the original RGB values.
  2. Variable Kernels: Instead of a fixed histogram bin, every pixel is treated as a Gaussian "cloud" of probability. If a pixel’s color is mathematically stable, its kernel is narrow and sharp. If a pixel is near a singularity (like the black point), its kernel becomes wide and flat, spreading its influence across many bins to account for the high uncertainty.
Empirical verification of the noise propagation model, showing predicted vs. measured uncertainty.

Experimental Setup

The approach was evaluated using a core database of 500 target images featuring various objects such as tools, toys, and food products recorded under controlled lighting.

An example object (Fruit Cocktail can) from the 500-image core database used for evaluation.

To test robustness, the authors used a query set of 70 test images with varying orientations and distances. They also applied simulated Gaussian noise ($\sigma$ from 2 to 64) and conducted realistic Signal-to-Noise Ratio (SNR) tests by recording objects under decreasing light intensities.

Key Results and Insights

  • Don't Threshold, Smooth: The central insight is that unstable color values are not "bad" data to be deleted; they are simply "uncertain." By using a variable kernel, the system naturally down-weights the influence of noisy pixels without losing the information they provide.
  • Superior Recognition: In the presence of significant noise ($\sigma = 64$), the proposed VKDE method maintained much higher average ranking percentiles compared to traditional histograms or ad-hoc thresholding.
  • Low-Light Performance: In realistic SNR experiments, the VKDE method showed a much slower decay in performance as the lights dimmed compared to standard methods.
  • Model Accuracy: The authors empirically verified their noise propagation model, finding that the predicted uncertainty matched the actual measured uncertainty of the sensors within 1%.

Limitations and Future Work

The primary drawback of this method is computational cost; constructing a variable kernel density estimate is roughly 11 times slower than building a standard histogram. Additionally, at extremely low light levels (SNR < 12), 8-bit quantization causes the Gaussian noise model to break down, leading to a sharp drop in performance for all methods. Future research may apply this general framework to other non-linear image features, such as texture descriptors or geometric invariants.

Summary updated

Abstract

An effective object recognition scheme is to represent and match images on the basis of histograms derived from photometric color invariants. A drawback, however, is that certain color invariant values become very unstable in the presence of sensor noise. To suppress the effect of noise for unstable color invariant values, in this paper, histograms are computed by variable kernel density estimators. To apply variable kernel density estimation in a principled way, models are proposed for the propagation of sensor noise through color invariant variables. As a result, the associated uncertainty is obtained for each color invariant value. The associated uncertainty is used to derive the parameterization of the variable kernel for the purpose of robust histogram construction. It is empirically verified that the proposed density estimator compares favorably to traditional histogram schemes for the purpose of object recognition.
BibTeX
@article{gevers2004robust,
  author = {Theo Gevers and H.M.G. Stokman},
  title = {{Robust histogram construction from color invariants for object recognition}},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2004},
  doi = {10.1109/tpami.2004.1261083},
}
Figure