An Automated Engineering Assistant: Learning Parsers for Technical Drawings

Dries Van Daele, Nicholas Decleyre, Herman Dubois, Wannes Meert

Abstract

Manufacturing operations rely heavily on technical drawings, yet these documents often exist as "dark data"—searchable only by basic metadata rather than their rich internal content. We present a hybrid AI framework that combines deep learning with symbolic logic to parse complex technical drawings. By employing Inductive Logic Programming (ILP) for relational table understanding and self-supervised Siamese networks for visual geometry, our system enables engineers to perform content-based searches, saving significant time in design retrieval tasks.

Introduction

Engineers frequently need to locate historical designs based on specific geometric profiles or material requirements. However, traditional search methods are limited to keywords or titles, forcing manual exploration of thousands of files. Technical drawings present a unique challenge: they contain both unstructured visual geometry (CAD views) and structured relational data (parts lists and material tables). While convolutional neural networks (CNNs) excel at shape recognition, they struggle with the strict logic of tables. Conversely, hand-coded parsers are too brittle for the variety of templates found in decades of archives.

The Hybrid Pipeline

To address these challenges, we developed a modular pipeline that treats technical drawings as multi-modal documents.

The hybrid pipeline for technical drawing analysis, combining image segmentation, object recognition, and symbolic parsing.
  1. Segmentation & Classification: Drawings are first partitioned using DBSCAN clustering. A CNN then classifies segments into categories such as "tables" or "2D CAD drawings."
  2. Symbolic Table Parsing via ILP: For structured data, we utilize Inductive Logic Programming. Unlike standard classifiers, ILP learns logical rules to define cell relationships (e.g., identifying a 'header' based on its position relative to keywords). A bootstrapping strategy allows the system to learn simple labels first, using them to resolve complex, recursive structures like multi-row material lists.
  3. Visual Feature Extraction: To process CAD views without extensive manual labeling, we employ a Siamese ResNet-50 architecture. By training the model to recognize design identity across transformations (rotation, shifting), it learns robust visual features in a self-supervised manner.
  4. Similarity Search: The system integrates tabular and visual features using a weighted geometric mean, allowing for queries based on "partial designs"—such as a sketch combined with specific material constraints.

Experimental Evaluation

The system was evaluated using a real-world dataset from Saint-Gobain Mobility, comprising 5,000 archived drawings. A subset of 318 drawings was used for segment classification, while only 30 drawings were required to train the ILP table parser, demonstrating the "small data" efficiency of the symbolic approach.

Key Results

The ILP-based parser demonstrated exceptional performance, achieving a perfect F1 score (1.0) on most table column labels. This significantly outperformed traditional Gradient Boosting Machines, which failed to capture the recursive logic of descriptions.

Performance comparison showing the F1 score improvement using ILP with bootstrapping versus standard ILP as the number of training drawings increases.

Furthermore, the initial segment classifier reached 96.8% accuracy. In practical deployment, engineers reported that the tool saves 15 to 30 minutes per task by streamlining the exploration of historical design spaces.

Conclusion

By bridging the gap between neural "perception" and symbolic "reasoning," this system transforms static technical drawings into searchable assets. Future work will focus on automatically extracting design rules from historical data to provide real-time feedback on new engineering constraints.

Summary updated

BibTeX
@inproceedings{daele2021automated,
  author = {Dries Van Daele and Nicholas Decleyre and Herman Dubois and Wannes Meert},
  title = {{An Automated Engineering Assistant: Learning Parsers for Technical Drawings}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17783},
}
Figure