Abstract
Engineering firms possess vast archives of legacy technical drawings that are difficult to search due to limited metadata. This paper presents a hybrid AI system combining deep learning and Inductive Logic Programming (ILP) to automatically parse and index these documents. By extracting visual CAD features and structured tabular data, the system enables content-based similarity searches, facilitating design reuse and adaptation.
Introduction
Engineering firms sit on decades' worth of legacy technical drawings that are notoriously difficult to search. Because these documents are usually only indexed by basic metadata (like title or date), engineers struggle to find previous designs that could be reused, adapted, or used to complete a partial design. Writing hard-coded software parsers to extract rich tabular data (like bills of materials) and visual CAD data from these drawings is expensive and brittle due to subtle variations in design templates over time. This research proposes an "automated engineering assistant" capable of learning to extract rich data from minimal expert annotations, unlocking the ability to search for designs based on deep structural and visual similarity.
Methodology
The authors propose a hybrid neuro-symbolic pipeline that leverages the strengths of both neural networks and logic-based AI to process complex documents.
Segmentation and Classification
The process begins by isolating visual elements using DBSCAN. A small Convolutional Neural Network (CNN) then classifies these segments into categories such as 2D CAD drawings, tabular data, or irrelevant segments.
Symbolic Parsing with ILP
To parse complex, non-standard tables, the system employs Inductive Logic Programming (ILP). A novel bootstrapping strategy allows the model to learn simple rules first (e.g., identifying a header row) and uses those as background knowledge to learn more complex, recursive rules (e.g., parsing indexed rows in a materials table). This hierarchy prevents overfitting on small datasets.
Handling Noisy OCR with ProbLog
To fix Optical Character Recognition (OCR) errors on custom part codes, the system uses Statistical Relational Learning via ProbLog. By treating OCR outputs as probability distributions and applying "virtual evidence" (e.g., injecting domain knowledge that a specific column should only contain numbers), the system drastically improves text recognition where standard dictionaries fail.
Visual Feature Extraction and Similarity Search
Lacking labeled visual data, the authors use a self-supervised approach. They train a modified ResNet-50 to predict if two augmented image pairs represent the same design. The final 64-neuron layer becomes a robust, translation-invariant feature vector. Finally, tabular similarity (calculated via Frequent Pattern mining) and visual similarity (via cosine similarity) are combined into a user-tunable weighted geometric mean for search.
Key Results
The system was evaluated on a proprietary dataset of 5,000 technical drawings from Saint-Gobain Mobility.
- Classification: The initial CNN segment classifier achieved 100% accuracy on a test set of 500 segments.
- Parsing: The bootstrapped ILP method achieved a perfect F1 score for parsing complex materials tables when trained on more than three drawings, significantly outperforming standard ILP.
- Visual Features: The ResNet-50 model achieved 96.8% accuracy on the binary classification task, proving the feature vector successfully captures unique design identifiers.
Conclusion
Combining data-driven neural networks for visual data with knowledge-driven symbolic logic for structured data provides a more complete and interpretable understanding of complex documents. Future work will focus on moving beyond search to automated reasoning, using extracted representations to verify engineering constraints and flag anomalous designs during the drafting process.
Summary updated