A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study

Milad Ramezankhani, Amir Nazemi, Apurva Narayan, Heinz Voggenreiter, Mehrtash Harandi, Rudolf J. Seethaler, Abbas S. Milani

TL;DR

This paper introduces a novel "multi-fidelity physics-informed" framework that leverages knowledge from less accurate, but abundant, "low-fidelity" models to improve the accuracy of "high-fidelity" models, especially when high-fidelity data is scarce. This approach significantly enhances the performance of physics-informed neural networks (PINNs) for complex engineering problems like composite material heat transfer during manufacturing.

What Problem Does This Paper Address?

Modern manufacturing, particularly in fields like aerospace, relies on complex processes that are difficult to model accurately. Physics-informed neural networks (PINNs) offer a promising way to learn these complex relationships by incorporating governing physical laws directly into the neural network training. However, training PINNs for highly nonlinear systems is challenging, requiring extensive hyperparameter tuning, complex network architectures, and costly, time-consuming training processes. Furthermore, obtaining large amounts of high-fidelity data (e.g., from expensive experiments or detailed simulations) is often infeasible. This paper addresses the challenge of building accurate predictive models for complex engineering systems when high-fidelity data is limited, by proposing a framework that effectively utilizes readily available, but less precise, low-fidelity data.

Methods

The core of the proposed solution is a Data-driven Multi-fidelity Physics-informed Framework (MFPINN). This framework builds upon the concept of transfer learning and multi-fidelity learning. Instead of using standard neural networks, it employs two interconnected Physics-informed Neural Networks (PINNs):

  • Low-fidelity PINN (PINN_L): This network is trained using available low-fidelity data and the governing physical laws (PDEs) relevant to the low-fidelity system.
  • High-fidelity PINN (PINN_H): This network learns from the predictions of PINN_L, along with the high-fidelity data and its corresponding governing physical laws.

Key methodological innovations include:

  • Transfer Learning with PINNs: Leveraging the knowledge learned by PINN_L as an auxiliary source for training PINN_H, reducing reliance on extensive high-fidelity data.
  • Adaptive Weighting: An adaptive weighting method is used to dynamically adjust the contribution of different loss terms (physics loss, data loss) during training. This helps to manage stiff gradient flow dynamics and conflicting gradients, leading to more stable and efficient optimization.
  • Data-driven Strategy for Divergence: A specific strategy is introduced to address subdomains where low- and high-fidelity behaviors diverge significantly. This involves strategically adding high-density labeled data points in these error-prone regions to guide the model.

Datasets

The study uses a case study involving the heat transfer of composite materials during an autoclave cure cycle.

  • Low-fidelity data: This data is generated using ABAQUS commercial software to simulate the heat transfer of a composite part (Composite 1) and solve its governing PDEs. The paper implies this data is more abundant.
  • High-fidelity data: This data is also generated via ABAQUS simulations for a different composite material (Composite 2) with distinct thermal properties. The paper frames this as the "target" system where high-fidelity data is scarce.
  • Labeled Data: For evaluating the impact of labeled data, specific subsets of the simulated data (e.g., 10, 50, 100, 200, 400 points) from Composite 2 were used for training. A "high-density cloud" of 30 labeled points was specifically used in the cooldown phase to address divergence.

Key Insights and Ideas

  • Bridging the Fidelity Gap: The core insight is that by explicitly incorporating physics and learning the correlation between low- and high-fidelity models, we can achieve high accuracy even with limited high-fidelity data. This is more powerful than simply using a low-fidelity model as a prior.
  • PINNs as Building Blocks: Using PINNs for both low- and high-fidelity components allows the framework to inherently respect physical laws, making it more robust than using standard neural networks for multi-fidelity learning.
  • Adaptive Weighting is Crucial: The adaptive weighting strategy is vital for managing the complex, multi-objective loss function of PINNs, preventing training instabilities and improving convergence.
  • Targeted Data Injection: The data-driven strategy of adding focused labeled data in regions of divergence is a practical and effective way to fine-tune the model where it struggles most.

Key Results

The proposed MFPINN framework demonstrated superior performance compared to conventional PINNs:

  • Reduced Error: The MFPINN model achieved a significantly lower relative L² error. For instance, the MFPINN with labeled data achieved a relative L² error of 1.7 (× 10⁻²), compared to 12.9 (× 10⁻²) for a vanilla PINN and 5.1 (× 10⁻²) for a PINN trained with labeled data.
  • Improved Prediction Accuracy: The framework successfully predicted the temperature distribution of the composite material during the cure cycle, with noticeable improvements in accuracy, especially in regions where low- and high-fidelity behaviors diverged. The model was able to correct errors that were previously around 15°C down to less than 6°C.
  • Effective Use of Low-Fidelity Knowledge: Even without any labeled high-fidelity data, the MFPINN model showed significant improvement over a vanilla PINN by leveraging the low-fidelity predictions.

Limitations and Future Work

  • Divergence Regions: While the data-driven strategy helps, the paper notes that the model's performance can decline towards the end of the time domain where low- and high-fidelity curves diverge. This suggests that identifying and addressing these divergence regions prior to training is an important aspect for future work.
  • Generalizability: The study focuses on a specific heat transfer problem. Further research could explore the applicability of this framework to other complex engineering systems and different types of PDEs.
  • Data Generation Costs: While the framework reduces the need for high-fidelity data, the generation of low-fidelity data (even if synthetic) can still be computationally intensive. Future work might explore more efficient ways to generate or leverage low-fidelity information.

Summary updated

Abstract

Despite the successful implementations of physics-informed neural networks in different scientific domains, it has been shown that for complex nonlinear systems, achieving an accurate model requires extensive hyperparameter tuning, network architecture design, and costly and exhaustive training processes. To avoid such obstacles and make the training of physics-informed models less precarious, in this paper, a data-driven multi-fidelity physics-informed framework is proposed based on transfer learning principles. The framework incorporates the knowledge from low-fidelity (auxiliary) systems and limited labeled data from target (actual) system to significantly improve the performance of conventional physics-informed models. While minimizing the efforts of designing a complex task-specific network for the problem at hand, the proposed settings guide the physics-informed model towards a fast and efficient convergence to a global optimum. An adaptive weighting method is utilized to further enhance the optimization of the model's composite loss function during the training process. A data-driven strategy is also introduced for maintaining high performance in subdomains with significant divergence between low- and high-fidelity behaviours. The heat transfer of composite materials undergoing a cure cycle is investigated as a case study to demonstrate the proposed framework's superior performance compared to conventional physics-informed models.
BibTeX
@inproceedings{ramezankhani2022data,
  author = {Milad Ramezankhani and Amir Nazemi and Apurva Narayan and Heinz Voggenreiter and Mehrtash Harandi and Rudolf J. Seethaler and Abbas S. Milani},
  title = {{A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study}},
  booktitle = {ICPS},
  year = {2022},
  url = {https://openreview.net/forum?id=jl9YpKMYoz},
}
Figure