Towards Learning-Aided Configuration in 3D Printing: Feasibility Study and Application to Defect Prediction

Benoit Amand, Maxime Cordy, Patrick Heymans, Mathieu Acher, Paul Temple, Jean-Marc Jézéquel

TL;DR

This paper explores using machine learning to automatically detect defects in 3D printable models by learning from existing configurations. They found that while standard classifiers achieve high accuracy, they struggle with boundary cases, suggesting a need to combine machine learning with explicit constraint knowledge for robust 3D model configuration.

What Problem Does This Paper Address?

Configurators are tools that help users tailor products by selecting options and setting parameters. A crucial aspect of configurators is ensuring that the chosen configuration is valid and will result in a functional product. This is typically achieved through explicit logical constraints that define valid parameter combinations. However, in some domains, like the 3D printing industry, capturing this configuration knowledge is challenging or even infeasible. Parametric 3D models often lack explicit constraints, meaning users can create configurations that are technically impossible to print correctly, leading to wasted time and resources. This paper addresses this gap by investigating whether machine learning can learn to predict valid configurations in the absence of explicit constraints, specifically for 3D printing.

Methods

The authors developed a toolchain to tackle this problem. First, they created an OpenSCAD parser to extract parameters and their value domains from 3D object models. Next, they employed a sampling strategy to generate a representative set of configurations. For numeric parameters with interval domains, they sampled 20 equidistant values. For enumerated domains, they included all possible values. The Cartesian product of these values formed the training set.

To determine the validity of each sampled configuration (the "oracle"), they utilized the diagnostic capabilities of Slic3r, a tool that can identify common issues in 3D object models. This process generated labeled data (valid/invalid configurations). Finally, they trained 17 different state-of-the-art classification algorithms on this data to predict the validity of unseen configurations. They evaluated these classifiers using standard metrics like accuracy, precision, recall, and F1-score.

Datasets

The study used a dataset of 5057 OpenSCAD parametric models gathered from Thingiverse. After filtering out models that were too computationally expensive to process or yielded trivial results (all valid or all invalid configurations), 31 models were selected for detailed analysis. For each of these 31 models, a training set and a test set of configurations were generated. The test set was designed to be distinct from the training set, with numeric parameters sampled using 21 equidistant values to ensure no overlap.

Key Insights and Ideas

  • Machine learning shows promise but isn't a silver bullet: Standard classification algorithms can achieve high accuracy in predicting valid 3D print configurations, demonstrating the potential of learning-based approaches.
  • Boundary cases are the Achilles' heel: Classifiers struggle significantly when a single parameter value or a specific combination of parameter values acts as a sharp boundary between valid and invalid configurations. This is particularly evident in models with continuous parameter ranges.
  • The need for hybrid approaches: The limitations observed with boundary cases strongly suggest that a purely learning-based approach is insufficient. Combining machine learning with explicit constraint knowledge (from experts or other sources) is likely necessary for robust and accurate configuration assistance.
  • Randomness is a major challenge: Models where the outcome is heavily influenced by a random number generator (like the "Blizzard of Unique Snowflakes" example) are particularly difficult for machine learning to handle, as there's no inherent correlation between parameter values and validity.

Key Results

Overall, the classifiers achieved high average scores across the 31 models, with average F1-scores rarely falling below 0.93. For instance, the Random Committee classifier achieved an average F1-score of 0.9735. However, a detailed analysis of four specific models revealed significant weaknesses.

  • Hopper Upgrade Extension: A single parameter value (32) acted as a delimiter. If this value was not in the training set, classifiers struggled to extrapolate and often misclassified configurations near this boundary, resulting in a false positive for the value 32.
  • Holesphere: The validity depended on the relative values of two parameters. Classifiers could draw a boundary between valid and invalid regions, but their accuracy was limited without knowing the exact mathematical equation defining this boundary.
  • BLTouch mounting adapter: A specific horizontal offset value (2.5) led to invalid configurations. If this value was not in the training set, classifiers failed to identify it.
  • Blizzard of Unique Snowflakes: This model, heavily reliant on a random seed, showed very poor performance, with F1-scores as low as 0.25 for some SVM classifiers.

Limitations and Future Work

The authors acknowledge several limitations:

  • Model selection bias: They discarded models requiring substantial computation, potentially missing more challenging cases.
  • Sampling strategy: The assumption of uniform distribution of diversity over parameter space might not hold true for all models.
  • Oracle completeness: Slic3r, while useful, might not be a perfect oracle, potentially leading to mislabeled configurations.
  • Hyperparameter tuning: The choice of classifiers and their hyperparameters was not exhaustively optimized.

Future work directions include:

  • Integrating expert knowledge: Developing methods to combine machine learning with explicit knowledge from 3D printing experts.
  • Mining knowledge from forums: Extracting configuration knowledge from online communities like Thingiverse.
  • Improving the testing oracle: Exploring more robust and potentially costly methods for validating configurations.
  • Integrating into existing platforms: Showcasing the value of learning-aided configuration by integrating it into tools like Thingiverse's Customizer.
  • Formalizing learned constraints: Investigating how to represent and enforce learned constraints, especially given the limitations of the OpenSCAD language.

Summary updated

Abstract

Configurators rely on logical constraints over parameters to aid users and determine the validity of a configuration. However, for some domains, capturing such configuration knowledge is hard, if not infeasible. This is the case in the 3D printing industry, where parametric 3D object models contain the list of parameters and their value domains, but no explicit constraints. This calls for a complementary approach that learns what configurations are valid based on previous experiences. In this paper, we report on preliminary experiments showing the capability of state-of-the-art classification algorithms to assist the configuration process. While machine learning holds its promises when it comes to evaluation scores, an in-depth analysis reveals the opportunity to combine the classifiers with constraint solvers.
BibTeX
@inproceedings{amand2019towards,
  author = {Benoit Amand and Maxime Cordy and Patrick Heymans and Mathieu Acher and Paul Temple and Jean-Marc Jézéquel},
  title = {{Towards Learning-Aided Configuration in 3D Printing: Feasibility Study and Application to Defect Prediction}},
  booktitle = {VaMoS},
  year = {2019},
  doi = {10.1145/3302333.3302338},
}

Loading related content...

Figure