Automatic Classification Algorithm for Raw Materials using Mean Shift Clustering and Stepwise Region Merging in Color

Kim SangJun, Joon-Young Kwak, Byoung Chul KoJournal of Broadcast Engineering 2016 · DOI

Abstract

In this paper, we propose a classification model by analyzing raw material images recorded using a color CCD camera to automatically classify good and defective agricultural products such as rice, coffee, and green tea. To resolve the problems of existing human-dependent commercial products, we propose a vision-based automatic raw material classification combining mean shift clustering and stepwise region merging algorithms. The image is divided into N cluster regions by applying the mean-shift clustering algorithm. Representative regions are selected, and a stepwise region-merging method is applied to integrate similar cluster regions by comparing both color and positional proximity. The merged objects are expressed in 2D color distributions (RG, GB, BR), and a threshold is used to detect good and defective products based on color distribution ellipses. Experimental results show improved classification accuracy and reduced need for artificial manipulation compared to existing methods.

Introduction

In the agricultural and recycling industries, classifying raw materials—such as separating high-quality coffee beans from defective ones—has traditionally relied on skilled manual labor. However, human performance degrades over time due to fatigue and the repetitive nature of the task. While automated systems exist, they face several technical hurdles:

  1. Color Complexity: Early systems used monochromatic data, which fails for materials with diverse color profiles like coffee or green tea.
  2. Parameter Sensitivity: Standard algorithms like K-means require pre-defined cluster counts and are highly sensitive to initial conditions.
  3. Manual Calibration: Many commercial systems require manual adjustment of color threshold boundaries, which is inefficient and prone to error when material color distributions shift.

The proposed system aims to create a "hands-off," noise-resistant solution capable of handling complex color distributions automatically within a physical sorting environment.

Layout of the automatic sorting machine.

Proposed Methodology

The authors propose a multi-stage computer vision pipeline designed for object-centric classification rather than simple pixel-wise analysis. This approach allows the system to build statistical models based on entire objects, making it more robust to noise.

1. Preprocessing

The system removes the background and applies morphological erosion (using a 5x5 kernel) to clean up the edges of the raw materials. This ensures that background pixels do not bleed into the material's color profile, enhancing the purity of the training data.

2. Mean Shift Clustering

Unlike K-means, Mean Shift is a non-parametric clustering technique. It finds local maxima in the color-position probability density without requiring a pre-defined number of clusters. This makes it ideal for segmenting objects with irregular shapes and unpredictable color variations.

3. Stepwise Region Merging

Because a single bean might be split into multiple clusters due to highlights or shadows, a merging step is implemented. A "seed" region is selected and merged with neighboring clusters based on Euclidean color similarity and spatial proximity.

4. 2D Color Distribution & Ellipse Thresholding

Once objects are merged, their pixels are mapped into three 2D color spaces: RG, GB, and BR. The system calculates the centroid and the tilt angle (using the least moment of inertia) to draw an automatic threshold ellipse around the "good" material. Any future material falling outside these ellipses is flagged as defective.

Proposed vision-based classification pipeline.

Experimental Evaluation

The algorithm was evaluated using 14 color images collected from a commercial sorting environment, containing 62 rice grains, 14 coffee beans, and 11 plastic pieces. Performance was benchmarked against a standard K-means algorithm and a commercial sorting system from Daewon GSI Co., Ltd.

Key Results

  • Proposed Method: Achieved an average accuracy of 84.9%.
  • Commercial System: Achieved 41.4%. The proposed method improved upon this by roughly 40% by automating the thresholding process that usually requires manual human adjustment.
  • K-means Clustering: Achieved only 3.47%. K-means failed due to high sensitivity to noise and random initial cluster centers, leading to poor extraction of material regions.

The system performed best on coffee and rice but was slightly less accurate on plastic due to more diverse and unpredictable color profiles.

Conclusion

This paper introduces a robust pipeline for automatic material sorting. By combining Mean Shift clustering with stepwise region merging, the system automatically defines optimal color thresholds, significantly outperforming existing solutions. Future work will focus on optimizing the algorithm for faster classification rates and smaller memory footprints to meet the demands of real-world industrial sorting, where thousands of items pass per minute.

Summary updated

Abstract

본 논문에서는 카메라로부터 입력된 영상으로부터 쌀, 커피, 녹차 등 다양한 원료를 양품과 불량품으로 자동 분류하기 위한 분류 모델을 제안한다. 현재 농산물 원료 분류를 위해서 주로 숙달된 노동력의 육안 선택에 의존하고 있지만 작업시간이 길어질수록 반복적인 작업에 의해 분류 능력이 현저히 떨어지는 문제점이 있다. 노동력에 부분적으로 의존하는 기존 제품의 문제점을 해결하기 위해, 본 논문에서는 평균-이동 클러스터링 알고리즘과 단계별 영역 병합 알고리즘을 결합하는 비전기반 자동 원료 분류 알고리즘을 제안한다. 우선 입력 원료 영상에서 평균-이동 클러스터링 알고리즘을 적용하여 영상을 N개의 클러스터 영역으로 분할한다. 다음단계에서 N개의 클러스터 영역 중에서 대표 영역을 선택하고 이웃 영역들의 영역의 색상과 위치 근접성을 기반으로 단계별 영역 병합 알고리즘을 적용하여 유사한 클러스터 영역을 병합한다. 병합된 원료 객체는 RG, GB, BR의 2D 색상 분표로 표현되고, 병합된 원료 객체에 대해 색상 분포 타원을 만든다. 이후 미리 실험적으로 설정된 임계값을 적용하여 원료를 양품과 불량품을 구분한다. 다양한 원료 영상에 대해 본 논문에서 제안하는 알고리즘을 적용한 결과 기존의 클러스터링 알고리즘이나 상업용 분류 방법에 비해 사용자의 인위적 조작이 덜 필요하고 분류성능이 우수한 결과를 나타냄을 알 수 있었다. In this paper, we propose a classification model by analyzing raw material images recorded using a color CCD camera to automatically classify good and defective agricultural products such as rice, coffee, and green tea, and raw materials. The current classifying agricultural products mainly depends on visual selection by skilled laborers. However, classification ability may drop owing to repeated labor for a long period of time. To resolve the problems of existing human dependant commercial products, we propose a vision based automatic raw material classification combining mean shift clustering and stepwise region merging algorithm. In this paper, the image is divided into N cluster regions by applying the mean-shift clustering algorithm to the foreground map image. Second, the representative regions among the N cluster regions are selected and stepwise region-merging method is applied to integrate similar cluster regions by comparing both color and positional proximity to neighboring regions. The merged raw material objects thereby are expressed in a 2D color distribution of RG, GB, and BR. Third, a threshold is used to detect good and defective products based on color distribution ellipse for merged material objects. From the results of carrying out an experiment with diverse raw material images using the proposed method, less artificial manipulation by the user is required compared to existing clustering and commercial methods, and classification accuracy on raw materials is improved.
BibTeX
@article{sangjun2016automatic,
  author = {Kim SangJun and Joon-Young Kwak and Byoung Chul Ko},
  title = {{Automatic Classification Algorithm for Raw Materials using Mean Shift Clustering and Stepwise Region Merging in Color}},
  journal = {Journal of Broadcast Engineering},
  year = {2016},
  doi = {10.5909/jbe.2016.21.3.425},
}
Figure