Introduction
Color Vision Deficiency (CVD), often colloquially called color blindness, affects millions of people worldwide. It creates significant daily challenges, from navigating traffic signals to identifying ripe produce in agricultural settings. For instance, a farmer with CVD might struggle to distinguish a ripe red tomato from an unripe green one against a backdrop of green leaves.
Physiologically, CVD is caused by deficiencies in the spectral sensitivity of the eye's cone cells (S, M, and L cones), which act as a biological filter on incoming light.
While previous technological solutions exist, they often rely on simplistic linear models that focus on contrast enhancement—making objects look different from one another—rather than color restoration—making objects look like their "true" colors. EnColor aims to bridge this gap by using deep learning to transform images so that a person with CVD perceives a result as close as possible to what a person with normal trichromatic vision sees.
Methods
The core of EnColor is a Deep Convolutional Encoder-Decoder architecture designed for non-linear color mapping.
- The Network: The encoder uses two convolutional layers to compress the input image into an abstract, low-dimensional representation. The decoder then uses two transposed convolutional layers to "up-sample" this representation back into a full-sized image with modified color values.
- The CVD Simulator: The most novel aspect of the training process is the inclusion of a "CVD Simulator" in the loss calculation. This simulator uses established physiological models (linear transformation matrices) to mimic how a person with a specific type of CVD (like deuteranopia) would see an image.
- The Training Loop: During training, an original image ($I$) is passed through EnColor to produce a corrected image ($I'$). This corrected image is then passed through the CVD Simulator ($S$) to produce $S(I')$. The loss function calculates the Mean Absolute Error (MAE) between the simulated corrected image $S(I')$ and the original image $I$.
Essentially, the model is trained to answer the question: "How should I change this image so that when a color-blind person looks at it, it looks 'normal' to them?"
Experimental Setup and Datasets
The researchers focused their evaluation on a practical, high-impact domain: agriculture. They utilized the Laboro Tomato Dataset, which contains over 600 high-quality images of ripe and unripe tomatoes in various lighting conditions and stages of growth. The authors used 600 images for training, 43 for validation, and 150 for testing, with all images resized to 648 × 484 to optimize computation.
Key Results
EnColor prioritizes "color retention," aiming for a naturalistic look that preserves the original intent of the visual information rather than just maximizing contrast.
- Quantitative Performance: EnColor achieved a 16.8% improvement in color restoration (measured via Mean Squared Error) compared to "Fuzzy logic" baselines and a 29.1% improvement over standard linear models.
- Structural Integrity: Using the Structural Similarity Index (SSIM), the authors demonstrated that EnColor maintains the texture and feel of the original image better than most heuristic-based methods.
- Human Subjective Assessment: A survey of 43 participants ranked EnColor the highest for both "contrast" and "ease of identifying ripe tomatoes." On a scale of 1-5 (where 1 is best), EnColor scored 1.59 for contrast and 1.45 for ripeness identification.
Discussion and Future Work
EnColor demonstrates that end-to-end non-linear mapping can significantly improve visual accessibility. By using the CVD simulator as a "teacher" within the loss function, the model automatically adapts to different types of CVD without manual parameter tuning.
However, limitations remain. CVD is a spectrum, and future work is needed to develop methods that can automatically identify an individual's specific "intensity" of CVD to provide bespoke correction. Additionally, the authors suggest that incorporating Generative Adversarial Networks (GANs) or including SSIM directly in the loss function could further improve the visual quality and sharpness of the corrected images.
Summary updated