Real-Time Detection of Cook Assistant Overalls Based on Embedded Reasoning

Qinghua Sheng, Haixiang Sheng, Peng Gao, Zhu Li, Haibing Yin2021 · DOI

TL;DR

This paper presents a real-time system for detecting cook assistant overalls on embedded devices, specifically targeting the Hi3559A processor. By optimizing the YOLOv3 model through techniques like network pruning, segmentation, and pipelining, they achieve a recognition speed of 28 FPS with only 5.5W of power consumption, making deep learning more feasible for resource-constrained environments.

What Problem Does This Paper Address?

The increasing prevalence of deep learning in computer vision has led to significant advancements in areas like image and speech recognition. However, deploying these powerful models on embedded devices, which are characterized by limited computational power and strict power constraints, remains a major challenge. Traditional approaches often rely on high-performance GPUs and cloud servers, leading to high costs, power consumption, and bandwidth requirements. This paper addresses the need for efficient, on-device deep learning for specific applications, such as identifying whether kitchen staff are wearing appropriate attire (cook assistant overalls) for food safety supervision. This is crucial for initiatives like the "transparent kitchen and stoves" movement, where real-time monitoring of food processing is desired but hindered by the limitations of current embedded systems.

Methods

The authors propose a multi-pronged approach to optimize a YOLOv3 model for embedded deployment on the Hi3559A processor:

  • Network Pruning: They employ L1 regularization and a "cutting" scheme to remove less important network layers and weights. This reduces model size and computational load. They found a 40% pruning ratio to be optimal, reducing computation by 58.59% with only a 0.19% accuracy drop.
  • Model Segmentation and Quantization: The Hi3559A has two neural network inference engines (NNIEs). Instead of parallel deployment, they opt for a serial deployment where the network is split into two unbalanced parts, each running on a separate NNIE. This is done to balance the reasoning time between the two networks, especially after quantization. Quantization converts floating-point numbers to fixed-point (8-bit) to reduce memory footprint, though it adds computational overhead.
  • Pipelining: Inspired by CPU pipelining, they divide the entire processing pipeline (image acquisition, decoding, reasoning, post-processing, video coding) into stages that can execute concurrently. This significantly reduces the overall processing cycle time from 92ms to approximately 30ms, improving hardware utilization.
  • Multithreaded Optimization and Circle Buffer: The system utilizes multithreading for independent tasks, with shared memory for data exchange. To ensure stability and prevent memory fragmentation, a circle buffer mechanism is implemented. This buffer manages memory efficiently, avoiding the need for mutual exclusion locks and improving system stability.

Datasets

The paper utilizes a self-built "cook dataset" for training and evaluation. This dataset consists of 33,254 images collected from 37 different kitchen scenes. The dataset was further expanded through augmentation techniques like tilt, mirroring, Gaussian blur, and histogram equalization, resulting in a final dataset of 43,255 images. The images are categorized into eight classes, representing different colors of overalls.

Key Insights and Ideas

  • Embedded Deep Learning is Achievable: The core insight is that with careful optimization, complex deep learning models like YOLOv3 can be effectively deployed on low-power embedded devices, opening up new application possibilities.
  • Pipelining is Crucial for Throughput: The paper demonstrates that by breaking down the deep learning pipeline into sequential stages that can run in parallel, significant improvements in processing speed can be achieved, even on resource-constrained hardware.
  • Unbalanced Segmentation for Balance: The authors highlight the importance of balancing the computational load between different parts of the network, especially after quantization, to avoid bottlenecks and maximize overall system performance.
  • Fixed-Point Arithmetic for Efficiency: Leveraging the Hi3559A's faster processing of fixed-point numbers and adapting post-processing steps to work with them significantly reduces computational complexity and improves speed.
  • Circle Buffer for Stability: The implementation of a circle buffer is key to ensuring the long-term stability and reliability of the embedded system, preventing issues like memory fragmentation and dirty data.

Key Results

  • Recognition Speed: The optimized system achieves a recognition speed of approximately 28 frames per second (FPS), exceeding the design requirement of 25 FPS.
  • Power Consumption: The system operates with a low power consumption of only 5.5W.
  • Accuracy: The system demonstrates good accuracy, with an average accuracy rate of 98.29% in on-site field tests. The mean average precision (mAP) on the COCO dataset is reported to be around 90.25%.
  • Model Compression: Network pruning reduced computation by 58.59% with a minimal accuracy loss of 0.19%.
  • System Stability: The system proved stable during long-term operation, with frame rates remaining consistent and no invalid memory access observed after six hours of testing.

Limitations and Future Work

  • Dataset Specificity: The primary dataset used is self-built and specific to kitchen overalls. While augmented, its diversity might be limited compared to larger, general-purpose datasets.
  • Hardware Dependency: The optimizations are tailored to the Hi3559A embedded processor. Transferring these optimizations to different hardware architectures might require further adjustments.
  • Future Work: The authors plan to incorporate newer software acceleration solutions and integrate them with existing hardware acceleration techniques to achieve even better performance.

Summary updated

BibTeX
@misc{sheng2021real,
  author = {Qinghua Sheng and Haixiang Sheng and Peng Gao and Zhu Li and Haibing Yin},
  title = {{Real-Time Detection of Cook Assistant Overalls Based on Embedded Reasoning}},
  year = {2021},
  doi = {10.3390/s21238069},
}
Figure