Hierarchical Color Learning in Convolutional Neural Networks

Chris Hickey, Byoung-Tak ZhangWorkshops @ CVPR 2020 · DOI

Abstract

This research investigates whether Convolutional Neural Networks (CNNs) exhibit a "Color Hierarchy" similar to the universal patterns observed in human linguistic evolution. By analyzing classification learning speeds and object detection recall, the study reveals that CNNs prioritize specific colors, such as red, while struggling with others like brown or gray. These findings suggest that artificial vision systems possess inherent developmental biases that mirror biological and cultural trends.

Introduction

In the field of linguistics, the "Color Hierarchy" theory suggests that as cultures evolve, they adopt color terms in a predictable, universal order (e.g., White/Black → Red → Green/Yellow → Blue → Brown). While CNNs are frequently compared to the human visual system, it has remained unclear whether they share this specific developmental bias. This paper addresses two main questions:

  1. Do CNNs learn to classify basic color categories in a fixed, hierarchical fashion?
  2. Does the color and "vividness" of an object impact the performance of object detection models?

Methodology

The researchers conducted two distinct experiments to explore these questions:

Experiment A: Color Classification

A standard CNN was trained to classify eight basic colors. To ensure the results were not an artifact of digital storage formats, the researchers tested five different color spaces: OPP, RGB, BGR, YCbCr, and YUV. Learning speed was measured by the number of training epochs required for a color to reach a recall threshold of 0.85.

Figure 1: Examples of the eight basic color categories used in Experiment A.

Experiment B: Object Detection

Using a Faster R-CNN architecture, the authors analyzed performance on the Modanet street fashion dataset. This dataset was enhanced by mapping object pixels to 267 unique NBS-ISCC color labels (e.g., "Vivid Pink," "Pale Green"), allowing for a granular analysis of how specific hues and descriptive qualities affect detection success.

Key Results and Analysis

The "Red" Advantage

Across all tests and color spaces, Red was consistently the easiest color for the network to learn and the most reliably detected. As shown in the data below, Red reached the target recall threshold significantly faster than any other color, mirroring human biological and linguistic trends where red is often the most salient non-achromatic color.

Figure 6: Learning speed (epochs to 0.85 recall) for eight basic colors across various color spaces.

In Experiment A, Red was learned in as few as 17.49 epochs (in YCbCr space). In contrast, colors lower on the proposed hierarchy, such as Brown, Orange, and Gray, consistently required between 32 and 37 epochs to reach the same performance level. This consistency across color spaces suggests the bias is inherent to the CNN learning process itself.

The Impact of Saturation and Value

Experiment B demonstrated that it isn't just the hue that matters, but also the "vividness" of the color. Red clothing items achieved a mean recall of 0.751, roughly 10% higher than the recall for pink, brown, orange, or gray items. Furthermore, adjectives implying high colorfulness resulted in better performance: objects described as "Brilliant" had a mean recall of 0.762, whereas "Pale" objects lagged behind at 0.594.

Conclusion

The study confirms that CNNs follow a hierarchical pattern in color learning that closely aligns with human linguistic evolution. The "Red Advantage" and the superior detection of vivid colors highlight a significant bias in artificial vision. These insights offer a path for future work, suggesting that model sensitivity to "harder" colors like gray or pale hues can be improved through targeted data augmentation or optimized color space selection.

Summary updated

BibTeX
@inproceedings{hickey2020hierarchical,
  author = {Chris Hickey and Byoung-Tak Zhang},
  title = {{Hierarchical Color Learning in Convolutional Neural Networks}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00200},
}
Figure