Segmentation of tobacco shred point cloud and 3-D measurement based on improved PointNet++ network with DTC algorithm

Yihang Wang, Hai-Ying Zheng, Jie Yang, Yan Wang, Li Wang, Qunfeng NiuFrontiers in Plant Science 2025 · DOI

TL;DR

Researchers have developed an improved PointNet++ deep learning architecture and a novel "Dimension Transformation Calculation" (DTC) algorithm to accurately measure the 3D dimensions of tobacco shreds. By combining high-precision laser scanning with semantic segmentation, the system achieves over 95% segmentation precision and measures complex, flexible filaments with significantly higher accuracy than traditional 2D imaging methods.

What Problem Does This Paper Address?

In the tobacco industry, the physical dimensions (length, width, and thickness) of tobacco shreds—including cut stems, silk, and reconstituted shreds—are critical. These dimensions directly influence cigarette combustibility, smoke characteristics, and the production of harmful substances.

However, measuring these tiny, flexible filaments is a nightmare for traditional computer vision. Tobacco shreds are rarely flat; they bend, curl, and overlap. Traditional 2D imaging methods only capture a "top-down" projection, which ignores thickness entirely and drastically underestimates the true length of a curled filament. While 3D laser scanning can capture the raw data, segmenting individual shreds from background noise and calculating the true "unrolled" length of a curved 3D object remains a significant technical hurdle.

Methods

The authors propose a hybrid pipeline consisting of high-precision data acquisition and a specialized deep learning model.

  1. Hardware & Data Acquisition: They used a Keyence LJ-X8080 laser scanning system to capture point clouds of both the upper and lower surfaces of tobacco shreds. Because these surfaces don't overlap, they used standard gauge blocks as reference points for a custom registration algorithm to "sandwich" the shred data together.
  2. Improved PointNet++: The core segmentation engine is based on PointNet++, but with three key upgrades:
  3. Weighted Cross-Entropy Loss: Since background noise often outweighs the actual tobacco data, they assigned higher weights to the shred classes to prevent the model from becoming biased toward the background.
  4. Cosine Annealing: This optimizes the learning rate during training, allowing the model to converge faster and find more stable local minima.
  5. KNN Multi-Scale Grouping (MSG): Instead of standard ball-querying, they used K-Nearest Neighbors to better capture the subtle, complex morphologies of thin filaments.
  6. DTC Algorithm: To calculate dimensions, they developed the Dimension Transformation Calculation. It maps the 3D point cloud to a 2D grayscale image to extract a "skeleton" (the centerline of the shred), then traces that skeleton back into 3D space to calculate the true curvilinear length.

Datasets

The researchers constructed a custom multi-category point cloud dataset.

  • Categories: Cut Stem, Tobacco Silk, Reconstituted Tobacco Shred, Expanded Tobacco Silk, and Background Noise.
  • Scale: Each category contains over 1,500 labeled samples.
  • Augmentation: They employed global random displacement, rotation, and background removal to expand the dataset fourfold, ensuring the model could handle shreds in any orientation.

Key Insights and Ideas

  • The 2D-3D Bridge: The most clever part of the paper is the DTC algorithm. Tracing a 3D skeleton directly in a sparse point cloud is computationally expensive and error-prone. By projecting to 2D to find the topology and then "re-inflating" those coordinates back to the 3D point cloud, they get the best of both worlds: the simplicity of 2D pathfinding and the accuracy of 3D spatial data.
  • Handling Class Imbalance: In many agricultural and industrial point cloud tasks, the "target" is a tiny fraction of the "scene." The use of weighted loss functions here is a vital reminder that raw accuracy is a poor metric when the background is 90% of your data.
  • Non-Contact Precision: By using laser scanning, the researchers avoid the "compression error" inherent in manual measurements with vernier calipers, which often squash the soft tobacco shreds and yield inaccurate thickness readings.

Key Results

The improvements to the PointNet++ architecture yielded substantial gains over the baseline:

  • Segmentation Accuracy: Precision jumped from 84.27% to 95.13%, and Recall increased from 83.63% to 97.68%.
  • Measurement Error: The relative errors for length and width were kept under 5% and 7%, respectively.
  • Thickness Accuracy: Using the gauge block validation, the system achieved a thickness measurement relative error of just 1.12%.
  • Efficiency: The prediction time for a single point cloud file was roughly 0.326 seconds, making it viable for real-world quality control lines.

Limitations and Future Work

While the results are impressive, the paper notes a few areas for growth:

  • Hardware Dependency: The system relies on a high-end XYZ three-axis displacement platform and specific laser sensors, which may be cost-prohibitive for smaller-scale operations.
  • Material Generalization: While the authors suggest this could work for other flexible materials (like textiles or films), the current model is specifically tuned for tobacco.
  • Real-time Integration: Future work aims to integrate this into fully automated cigarette production lines, which would require even faster processing and the ability to handle higher-density, overlapping shreds in real-time.

Summary updated

Abstract

This study also provides a new idea for implementing three-dimensional measurements of other flexible materials.
BibTeX
@inproceedings{wang2025segmentation,
  author = {Yihang Wang and Hai-Ying Zheng and Jie Yang and Yan Wang and Li Wang and Qunfeng Niu},
  title = {{Segmentation of tobacco shred point cloud and 3-D measurement based on improved PointNet++ network with DTC algorithm}},
  booktitle = {Frontiers in Plant Science},
  year = {2025},
  doi = {10.3389/fpls.2024.1508449},
}
Figure