Can CNNs Construct Highly Accurate Model Efficiently with Limited Training Samples?

Yu Li, Hu Wang, Juanjuan LiuarXiv 2017 · arXiv

Introduction

In modern engineering design, such as the development of automotive structures or aircraft, engineers frequently encounter systems defined by hundreds of design variables. Optimizing these designs requires "metamodels"—simplified mathematical approximations of computationally expensive simulations like Finite Element Analysis (FEA). However, traditional metamodeling techniques like Kriging or Radial Basis Functions (RBF) suffer from the "curse of dimensionality": as the number of variables increases, computational costs explode while accuracy diminishes. This paper investigates whether Convolutional Neural Networks (CNNs) can serve as high-performance metamodels capable of handling strongly nonlinear, high-dimensional systems.

Methodology: The CNN Metamodel

The authors propose a CNN-based architecture to bridge the gap in high-dimensional regression. While typically used for image recognition, CNNs are well-suited for engineering data due to their ability to identify local features and correlations.

The core innovation lies in the "Image Analogy." By reshaping a vector of design variables (e.g., 324 control points) into a matrix (e.g., an 18x18 grid), the CNN treats the design space like an image. This allows the network to capture complex interactions between variables that traditional linear or kernel-based methods often miss.

Figure 1: Flowchart of the CNN-based metamodeling process, including sample generation, training, and the specific CNN architecture.

As shown in Figure 1, the process involves three main stages:

  1. Getting Samples: Generating training and testing datasets from high-fidelity simulations.
  2. Training: Normalizing input variables and iteratively optimizing the CNN using the Adam optimizer to minimize a loss function.
  3. Testing: Evaluating the trained model's predictions against known labels to calculate error metrics.

The CNN architecture itself consists of alternating convolutional and pooling layers for feature extraction and dimensionality reduction, followed by fully connected layers for final numerical prediction.

Performance and Scalability

The researchers evaluated the CNN against traditional methods using standard mathematical benchmarks (up to 784 dimensions) and real-world engineering cases. A key finding was the CNN's remarkable stability across dimensions. While error rates for methods like Kriging-based KPLS often spike as dimensions increase, the CNN maintained a low relative error of approximately 1.2% even at 784 dimensions.

Figure 5: Comparison of performance metrics (R2, RAAE, and RMAE) between the proposed CNN (orange) and a baseline method (blue) across varying dimensions.

Figure 5 highlights the trade-offs in CNN performance:

  • Global Accuracy: The CNN (orange bars) consistently achieves higher $R^2$ values and lower Relative Average Absolute Error (RAAE) compared to the baseline. This indicates superior performance in capturing the overall behavior of the design space.
  • Local Error: Conversely, the Relative Maximum Absolute Error (RMAE) is often higher for the CNN. This suggests that while the model is globally accurate, it may occasionally struggle with extreme edge-case outliers.

Engineering Application: IGA Tubular Structure

To demonstrate practical viability, the method was applied to an IsoGeometric Analysis (IGA) model of a tubular structure involving 324 design variables. Using Particle Swarm Optimization guided by the CNN metamodel, the researchers found an optimal design with a maximum stress only 1.072% different from the high-fidelity simulation result. This confirms that CNNs can effectively replace expensive simulations in real engineering workflows.

Conclusion

CNNs represent a powerful tool for overcoming the curse of dimensionality in engineering design. By leveraging spatial feature extraction, they provide stable, globally accurate metamodels for high-dimensional, nonlinear problems. While further work is needed to improve local maximum error and sample efficiency, the CNN-based approach offers a scalable alternative to traditional surrogate modeling techniques.

Summary updated

BibTeX
@misc{li2017can,
  author = {Yu Li and Hu Wang and Juanjuan Liu},
  title = {{Can CNNs Construct Highly Accurate Model Efficiently with Limited Training Samples?}},
  year = {2017},
  eprint = {1712.01639},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1712.01639},
}
Figure