RGB Color Model Aware Computational Color Naming and Its Application to Data Augmentation

Zipei Yan, Linchuan Xu, Atsushi Suzuki, Jing Wang, Jiannong Cao, Jun HuangIEEE Big Data 2022 · DOI

Abstract

Computational Color Naming (CCN) aims to map raw RGB pixel values to semantic color labels. Traditional methods struggle with the sparsity of labeled color data and often produce disjointed color boundaries. We introduce ColorMLP, a neural network architecture that leverages Graph Attention Networks (GATs) to encode the inherent relationships of the RGB additive color model. By "understanding" color physics, ColorMLP achieves superior generalization. Furthermore, we propose Partial Color Jitter (PCJ), a semantic data augmentation method that uses ColorMLP to selectively jitter specific color regions, enhancing model performance on benchmarks like ImageNet while maintaining visual realism.

Introduction

Mapping a numeric triplet like [255, 200, 0] to the word "orange" is a fundamental task in computer vision known as Computational Color Naming. Despite its apparent simplicity, CCN is hindered by three major issues:

  1. Data Scarcity: Only a fraction of the 16.7 million possible RGB colors are labeled.
  2. Poor Generalization: Models often create "patchy" boundaries that do not align with human perception.
  3. Augmentation Artifacts: Standard "Color Jitter" (CJ) applies random shifts globally, often resulting in semantically nonsensical images (e.g., green oranges or purple skies) that can mislead training.

Methods: ColorMLP and PCJ

The researchers developed a two-pronged approach to address these challenges.

ColorMLP Architecture

ColorMLP is a Multi-Layer Perceptron constrained by the "physics" of the RGB model. Instead of learning weights purely from data, it uses three Graph Attention Networks (rGAT, gGAT, bGAT) to encode the additive relationships between primary colors (e.g., Red + Green = Yellow). These GATs generate the weights for the MLP's second linear layer, ensuring the model's internal logic respects the structure of the RGB cube.

The ColorMLP architecture uses GATs to regularize the weight generation process based on RGB color relationships.

Partial Color Jitter (PCJ)

Leveraging the precise color boundaries of ColorMLP, the authors introduced Partial Color Jitter. Unlike standard CJ, which shifts the entire image's color space, PCJ identifies pixels belonging to a specific semantic category (e.g., "Sky" or "Pineapple") and applies jitter only to those pixels. This maintains the realism of the rest of the scene while providing targeted diversity for training.

Comparison of standard Color Jitter (CJ) vs. Partial Color Jitter (PCJ). PCJ allows for selective augmentation of specific semantic regions like the sky or the fruit.

Experimental Results

The proposed methods were evaluated across multiple benchmarks:

  • CCN Accuracy: ColorMLP achieved 90.37% accuracy in 5-fold cross-validation on a consolidated dataset of ~178,000 pixel-color pairs, significantly outperforming standard MLPs and SVMs.
  • ImageNet Performance: Integrating PCJ into a ResNet-50 training pipeline on ImageNet-ILSVRC2012 reduced the top-1 error rate from 23.30% to 22.46%, a significant margin for such a well-studied benchmark.
  • Robustness: PCJ consistently outperformed other state-of-the-art augmentation techniques like ISDA on CIFAR-10 and CIFAR-100.

Conclusion

By injecting domain knowledge of the RGB color model into neural architectures, ColorMLP provides a more logical and continuous mapping of the color space. This semantic awareness enables Partial Color Jitter, which proves that "where" augmentation is applied is as critical as "how much." Future work will focus on expanding the color vocabulary beyond the 11 basic names and refining graph-based relationships for more complex interactions.

Summary updated

BibTeX
@inproceedings{yan2022rgb,
  author = {Zipei Yan and Linchuan Xu and Atsushi Suzuki and Jing Wang and Jiannong Cao and Jun Huang},
  title = {{RGB Color Model Aware Computational Color Naming and Its Application to Data Augmentation}},
  booktitle = {IEEE Big Data},
  year = {2022},
  doi = {10.1109/bigdata55660.2022.10020750},
}
Figure