Deep Learning and Machine Vision for Food Processing: A Survey

Lili Zhu, Petros Spachos, Erica Pensini, Konstantinos N. PlataniotisarXiv 2021 · OpenReview

TL;DR

This survey provides a comprehensive overview of how machine vision, powered by traditional and deep learning techniques, is revolutionizing food processing. It highlights the critical role of these technologies in ensuring food safety, quality, and efficiency, addressing the limitations of labor-intensive traditional methods.

What Problem Does This Paper Address?

The food industry faces increasing demands for quality and safety, coupled with the challenges of labor-intensive processes, rising costs, and the need for greater efficiency. Traditional methods for food quality control and inspection are often time-consuming and destructive. This paper addresses the need for automated, accurate, and non-destructive solutions by surveying the application of machine vision (MV) and image processing (IP) techniques, particularly those leveraging machine learning (ML) and deep learning (DL), across various stages of food processing.

Methods

The paper surveys a wide range of machine vision techniques and image processing methods. These include:

  • Image Acquisition: Various methods for capturing images, such as stereo systems for depth perception, remote sensing (RS) for spectral analysis, hyperspectral imaging (HSI) for detailed spectral information, X-ray imaging for foreign object detection, thermal imaging for temperature assessment, and magnetic resonance imaging (MRI) for internal structure analysis.
  • Image Processing: This is broken down into three levels:
  • Low-level processing: Pre-processing techniques like noise removal, contrast enhancement, and image filtering to improve image quality.
  • Intermediate-level processing: Segmentation, representation, and description of image features, including watershed segmentation, Otsu's method, and texture analysis.
  • High-level processing: Image recognition and interpretation using statistical methods or deep learning to classify and understand image content.
  • Machine Learning Approaches:
    • Traditional ML: Supervised (e.g., Support Vector Machines (SVM), Logistic Regression, K-Nearest Neighbors (KNN), Decision Trees, Random Forests), unsupervised (e.g., K-means, Fuzzy C-means), and reinforcement learning methods. These often involve manual feature extraction.
  • Deep Learning (DL): Architectures like Artificial Neural Networks (ANN), Back-Propagation (BP) networks, Convolutional Neural Networks (CNN), Fully Convolutional Networks (FCN), and Recurrent Neural Networks (RNN). DL excels at automatic feature learning from raw data.

The paper categorizes applications based on their function: food safety and quality evaluation, packaging monitoring, and foreign object detection.

Datasets

The paper does not explicitly introduce or benchmark new datasets. Instead, it surveys existing research that utilizes various datasets specific to the food products and tasks being investigated. Examples include images of apples, blueberries, peaches, mangoes, chicken fillets, and various other food items, captured using different imaging modalities.

Key Insights and Ideas

  • Machine Vision as a Game-Changer: MV systems offer a powerful, non-destructive, and efficient alternative to traditional, labor-intensive food inspection methods, leading to reduced costs and improved accuracy.
  • The Power of Deep Learning: Deep learning, with its ability to automatically learn complex features, is increasingly outperforming traditional ML methods in accuracy and robustness, especially for complex tasks like defect detection and classification.
  • Multimodality is Key: Combining different imaging techniques (e.g., hyperspectral, X-ray, thermal) and feature extraction methods often leads to more comprehensive and accurate results than using a single modality.
  • Beyond Visuals: While visual cues are primary, the paper acknowledges the challenge of correlating appearance with smell, a crucial aspect of food quality that imaging alone cannot capture.
  • The "Curse of Dimensionality": For high-dimensional data (common in hyperspectral imaging), dimensionality reduction techniques are essential for effective ML model training and performance.

Key Results

The paper summarizes numerous studies, highlighting high accuracy rates across various applications:

  • Food Safety & Quality: Accuracies often exceed 90% for tasks like defect detection (e.g., bruised apples at 98%), quality grading (e.g., mango grading with SVM, MADM achieving 87-89%), and classification of food types or conditions.
  • Foreign Object Detection: High accuracy rates (e.g., 99.5% for segmentation and 95% for classification of foreign objects in walnuts) are reported using advanced techniques like deep learning and X-ray imaging.
  • Processing Efficiency: MV systems can significantly boost processing speed, with some systems handling up to 50 potatoes per second or sorting 2800 food samples per hour.

Limitations and Future Work

Limitations:

  • Environmental Sensitivity: MV systems are highly dependent on consistent lighting and a clean environment, which can be challenging to maintain in dynamic food processing settings. This impacts robustness and reliability.
  • Feature Extraction Challenges: Detecting subtle color changes or unusual shapes remains difficult. Furthermore, imaging technologies currently struggle to correlate visual data with olfactory information, a critical factor in food quality.
  • Computational Demands: Processing large volumes of image data requires significant storage and computational power, increasing system costs, especially when integrating multiple imaging technologies.
  • Maturity of Integrated Systems: While MV systems are advancing, the integration of decision execution modules (like robot arms) is not yet fully mature for large-scale, intelligent food production. Real-time requirements can also be a challenge for current mechanical systems.

Future Work:

  • Algorithm Improvement: Continued development of more efficient and robust image processing algorithms is crucial for enhancing processing speed and reliability.
  • Embedded Vision Systems: The trend towards compact, fast, and low-cost embedded vision systems will drive wider adoption of MV in food processing.
  • Multimodal Integration & IoT: Further research into integrating multiple technologies and leveraging the Internet of Things (IoT) for smart systems is needed. Edge computing can help reduce resource usage and latency in image processing and interpretation.
  • Addressing Olfactory Detection: Developing methods to capture or infer olfactory information would be a significant advancement.

Summary updated

Abstract

The quality and safety of food is an important issue to the whole society, since it is at the basis of human health, social development and stability. Ensuring food quality and safety is a complex process, and all stages of food processing must be considered, from cultivating, harvesting and storage to preparation and consumption. However, these processes are often labour-intensive. Nowadays, the development of machine vision can greatly assist researchers and industries in improving the efficiency of food processing. As a result, machine vision has been widely used in all aspects of food processing. At the same time, image processing is an important component of machine vision. Image processing can take advantage of machine learning and deep learning models to effectively identify the type and quality of food. Subsequently, follow-up design in the machine vision system can address tasks such as food grading, detecting locations of defective spots or foreign objects, and removing impurities. In this paper, we provide an overview on the traditional machine learning and deep learning methods, as well as the machine vision techniques that can be applied to the field of food processing. We present the current approaches and challenges, and the future trends.
BibTeX
@misc{zhu2021deep,
  author = {Lili Zhu and Petros Spachos and Erica Pensini and Konstantinos N. Plataniotis},
  title = {{Deep Learning and Machine Vision for Food Processing: A Survey}},
  year = {2021},
  eprint = {2103.16106},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.16106},
}
Figure