Object-Sorting-by-Color in a Variety of Lighting Conditions Using Neural Networks and Lego Mindstorms Robot

Natasa Lazetic, Jianna ZhangAAAI 2006

Introduction

Color recognition is a fundamental yet complex task in computer vision. While humans effortlessly adjust for varying illumination, machines often struggle with "color constancy"—the ability to perceive the same color consistently under different light sources. A red object appears drastically different to a camera sensor under fluorescent office lighting compared to direct afternoon sunlight. This variability, combined with sensor differences and the subjective nature of human color perception, makes hard-coded color-sorting algorithms fragile in real-world environments.

This paper presents a prototype robotic system built using LEGO Mindstorms that addresses these challenges through Artificial Neural Networks (ANNs). By training a model on diverse visual data, the researchers aim to create a robust sorting mechanism suitable for industrial and agricultural applications, such as grading fruit or sorting manufactured parts.

System Architecture

The hardware platform is based on the LEGO Mindstorms Robotics Invention System 2.0. The physical setup consists of a conveyor belt, a camera for image acquisition, and a series of mechanical pushers (P1–P4) to divert objects into specific bins based on their classification.

Schematic diagram of the LEGO Mindstorms sorting system.

While the LEGO RCX brick manages the mechanical components, the computationally intensive image processing is offloaded to a connected PC. The authors propose three primary techniques for data preprocessing before feeding information into the ANN:

  1. Spatial Downsampling: Images are reduced from 352x288 pixels to a 15x15 grid via pixel averaging, simplifying the input while preserving the dominant color profile.
  2. Color Vector Approximation: A region-growing algorithm isolates the object from the background. The system then generates a frequency vector of color classifications for every pixel, making the model less sensitive to non-homogeneous lighting.
  3. Texture Analysis: For complex biological objects like fruit, statistical image texture is used to capture surface details with a significantly reduced input size.

Dataset and Training

To train and validate the ANN, the researchers developed a custom dataset using LEGO components of various colors (red, green, yellow, blue, and white). To ensure the system's robustness against environmental changes, images were captured across a wide spectrum of conditions, including indoor and outdoor locations, varying weather (sunny vs. cloudy), and different times of day.

Examples of red LEGO components used for training and testing.

The dataset is split into 75% for training and 25% for testing. This variety allows the ANN to learn the underlying patterns of a color rather than relying on specific RGB values that might shift with the light.

Key Insights and Discussion

The study highlights the utility of LEGO Mindstorms as a high-fidelity, low-cost prototyping environment for industrial concepts. A key advantage of using ANNs over traditional algorithms is their ability to generalize from "noisy" data. For instance, by converting an image into a "color count" vector, the system focuses on the dominant spectral signature rather than the exact orientation or position of the object on the belt.

Preliminary Results and Future Work

As a work in progress, the project aims for a classification accuracy of at least 90%. This target is supported by existing literature in agricultural ANN applications, where orange sorting has reached 93% accuracy and weed identification has hit 97.6%.

Future development will focus on:

  • Biological Variance: Transitioning from uniform LEGO bricks to fruits like apples and oranges, which exhibit higher natural variance in color and texture.
  • Geometric Integration: Incorporating shape recognition to distinguish between objects with similar colors but different forms.
  • Industrial Scaling: Moving the logic from the LEGO prototype to high-speed industrial robotics.

Summary updated

BibTeX
@inproceedings{lazetic2006object,
  author = {Natasa Lazetic and Jianna Zhang},
  title = {{Object-Sorting-by-Color in a Variety of Lighting Conditions Using Neural Networks and Lego Mindstorms Robot}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure