A Single-Camera-Based Three-Dimensional Velocity Field Measurement Method for Granular Media in Mass Finishing with Deep Learning

J. H. Zou, Chunyue Tian, Yiqun Liu, Junfei Ding, Wenhui LiSensors 2024 · DOI

TL;DR

This paper introduces a novel single-camera system that uses deep learning to accurately measure the 3D velocity field of granular media in mass finishing. This method significantly outperforms traditional Digital Image Correlation (DIC) in both accuracy (especially in the Z-direction) and speed, making it a more practical tool for understanding and optimizing surface finishing processes.

What Problem Does This Paper Address?

Mass finishing is a critical surface treatment process that relies on the complex motion of granular media. Understanding this granular flow is key to optimizing the process, but accurately measuring it is challenging. Existing methods are often expensive, require multiple cameras or specialized probes, are limited to single points, or interfere with the flow itself. This leaves a gap for a more accessible, comprehensive, and non-intrusive method to capture the full 3D velocity field of granular media in dynamic environments like mass finishing machines.

Methods

The core innovation is a single-camera system that leverages a trichromatic mask to capture three different perspectives of the granular flow from a single viewpoint. This effectively simulates a multi-camera setup without the need for precise synchronization. To process these multi-perspective images and reconstruct the 3D flow, the researchers integrate deep learning, specifically a lightweight optical flow estimation network called LiteFlowNet. LiteFlowNet replaces traditional, computationally intensive optical flow algorithms, drastically reducing processing time. The system then uses triangulation to convert 2D image data into 3D spatial positions, allowing for the calculation of velocity vectors.

Datasets

The paper does not explicitly mention training datasets for the deep learning model. However, the evaluation and performance analysis rely on data generated from controlled experiments using a checkerboard calibration board for camera calibration and spherical granular media (and later, walnut shell granular media) subjected to precise translations on a high-precision XYZ stage. This experimental setup serves as the benchmark for evaluating the proposed method against traditional DIC.

Key Insights and Ideas

  • Single Camera, Multiple Perspectives: The trichromatic mask is a clever way to achieve multi-view stereo reconstruction with just one camera, simplifying hardware and synchronization requirements.
  • Deep Learning for Speed and Accuracy: Replacing traditional optical flow with LiteFlowNet is a game-changer. It not only speeds up computation dramatically but also shows improved accuracy, particularly in the challenging Z-direction.
  • Bridging the Gap: This work effectively combines traditional optical techniques (like triangulation) with modern deep learning, demonstrating a powerful synergy for complex measurement tasks.
  • Mass Finishing Relevance: The successful application in a bowl-type vibratory finishing machine validates the method's practical utility in a real-world industrial scenario.

Key Results

  • Accuracy: The proposed method achieved an average error of less than 0.07 mm in measuring spatial displacement in the X and Z directions.
  • Speed: The calculation speed of the proposed method was 1291.67% faster than the traditional DIC algorithm under the same conditions.
  • Z-Direction Improvement: While both methods were accurate in the X-direction, the proposed method showed a significant accuracy improvement in the Z-direction, with an average error 656.63% better than DIC.
  • Feasibility: Experiments in a bowl-type vibratory finishing machine successfully captured the 3D flow of granular media, demonstrating the system's practical applicability.

Limitations and Future Work

  • Z-Direction SD: The authors note that standard deviations (SDs) in the Z-direction were larger than in X and Y. This is attributed to factors like triangulation geometry, lighting, and potential optical distortions, suggesting areas for further refinement.
  • Generalizability: While tested on granular media in mass finishing, further validation across a wider range of granular materials and dynamic scenarios would be beneficial.
  • Deep Learning Training Data: The paper doesn't detail the training data for LiteFlowNet, which could be a point of interest for reproducibility or further model development.

The paper highlights the significant potential of integrating deep learning with traditional optical techniques for practical engineering measurements, paving the way for more advanced and efficient measurement systems in various fields.

Summary updated

Abstract

Surface treatment processes such as mass finishing play a crucial role in enhancing the quality of machined parts across industries. However, accurate measurement of the velocity field of granular media in mass finishing presents significant challenges. Existing measurement methods suffer from issues such as complex and expensive equipment, limited to single-point measurements, interference with the flow field, and lack of universality in different scenarios. This study addresses these issues by proposing a single-camera-based method with deep learning to measure the three-dimensional velocity field of granular flow. We constructed a complete measurement system and analyzed the accuracy and performance of the proposed method by comparing the measurement results with those of the traditional DIC algorithm. The results show that the proposed method is very accurate in measuring spatial displacement, with an average error of less than 0.07 mm and a calculation speed that is 1291.67% of the traditional DIC algorithm under the same conditions. Additionally, experiments in a bowl-type vibratory finishing machine demonstrate the feasibility of the proposed method in capturing the three-dimensional flow of granular media. This research not only proposed a novel method for three-dimensional reconstruction and velocity field measurement using a single-color camera, but also demonstrated a way to combine deep learning with traditional optical techniques. It is of great significance to introduce deep learning to improve traditional optical techniques and apply them to practical engineering measurements.
BibTeX
@inproceedings{zou2024single,
  author = {J. H. Zou and Chunyue Tian and Yiqun Liu and Junfei Ding and Wenhui Li},
  title = {{A Single-Camera-Based Three-Dimensional Velocity Field Measurement Method for Granular Media in Mass Finishing with Deep Learning}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24154790},
}
Figure