Tolerance Information Extraction for Mechanical Engineering Drawings – A Digital Image Processing and Deep Learning-based Model

Yuanping Xu, Chaolong Zhang, Zhijie Xu, Chao Kong, Dan Tang, Xin Deng, Tukun Li, Jin JinCIRP 2024 · DOI

Introduction

In the modern landscape of Industry 4.0, manufacturing processes increasingly rely on digital twins and Computer-Aided Tolerance (CAT) software. However, a significant hurdle remains: millions of "legacy" mechanical engineering drawings (MEDs) that exist only as paper blueprints or static image files. Manually transcribing the complex Geometrical Tolerance Specification Callouts (GTSCs) from these documents into digital systems is a labor-intensive bottleneck, prone to errors that can necessitate intervention in up to 40% of cases.

Example of a mechanical engineering drawing with complex geometrical tolerance specification callouts (GTSCs).

The Challenge of Automated Extraction

Automating the recognition of GTSCs is technically demanding. These callouts are not simple text; they are structured blocks containing a mix of specialized symbols, numbers, and letters. They are often arranged in multi-line formats, frequently tilted due to scanning or photography angles, and often obscured by noise or overlapping leader lines from the drawing itself.

Proposed Methodology: A Hybrid Pipeline

To address these challenges, the researchers developed a multi-stage pipeline that integrates traditional digital image processing with advanced deep learning. The workflow is designed to handle the structural complexity of engineering documents while maintaining high recognition accuracy.

The integrated workflow for GTSC detection, extraction, and semantic recognition.

1. Pre-processing and Localization

The initial stage focuses on cleaning the input image and isolating the tolerance blocks. Gaussian filtering and binarization are used to eliminate noise. The system then employs morphological operations and Hough line detection to identify the rectangular frames of the GTSCs amidst the clutter of a full drawing.

2. Tilt Correction and Segmentation

For drawings captured via mobile devices, the model applies perspective transformation formulas to correct any tilt. Once straightened, the system performs advanced segmentation:

  • Multi-line Splitting: Hough transforms identify and remove horizontal separator lines within multi-line blocks.
  • The "Drop-Fall" Algorithm: To solve the problem of "touching characters"—where ink bleeds together or characters touch the box borders—the researchers implemented a heuristic that simulates a water drop falling through gaps to find the optimal segmentation path.

3. Recognition via Inception-AlexNet

The core of the recognition engine is Inception-AlexNet, a modified version of the classic AlexNet architecture. By replacing standard convolutional layers with Inception modules, the model can perform parallel convolutions of different sizes. This allows it to capture multi-scale features, which is critical for distinguishing between similar-looking technical symbols (such as different types of run-out or orientation tolerances). The addition of Batch Normalization (BN) and a reduced fully connected layer (2048 neurons) helps prevent overfitting and improves generalization.

Experimental Results

The model was trained and tested on a dataset of over 20,000 augmented images covering 106 categories of symbols, numbers, and letters.

  • Recognition Accuracy: The Inception-AlexNet achieved a 98.3% accuracy rate for individual character recognition, outperforming the standard AlexNet by 1.8%.
  • System-Wide Performance: When evaluated on the entire pipeline—from localization through to semantic recognition—the system achieved a success rate of 96.73% on real-world technical documents.
  • Efficiency: The architecture demonstrated faster convergence and lower training loss, suggesting its suitability for industrial deployment.

Conclusion and Future Work

This research provides a robust solution for digitizing legacy engineering data, bridging the gap between paper-based documentation and modern automated manufacturing. While the system handles complex interferences and leader lines effectively, future work will focus on further "lightening" the model for real-time use on mobile devices and integrating it directly into automated online measurement systems.

Summary updated

BibTeX
@inproceedings{xu2024tolerance,
  author = {Yuanping Xu and Chaolong Zhang and Zhijie Xu and Chao Kong and Dan Tang and Xin Deng and Tukun Li and Jin Jin},
  title = {{Tolerance Information Extraction for Mechanical Engineering Drawings – A Digital Image Processing and Deep Learning-based Model}},
  booktitle = {CIRP},
  year = {2024},
  doi = {10.1016/j.cirpj.2024.01.013},
}
Figure