Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process

Wei Chen, Northwestern University, Yigitcan Comlek, Sandipp Krishnan Ravi, Piyush Pandita, Sayan Ghosh, Liping Wang, Wei ChenarXiv 2024 · DOI · arXiv

TL;DR

This paper introduces a novel framework for fusing data from multiple sources that have different input spaces, a common challenge in engineering. By first mapping these heterogeneous inputs into a common space and then using a Latent Variable Gaussian Process (LVGP) model, the authors achieve more accurate predictions and better interpretability than existing methods, especially in data-scarce scenarios.

What Problem Does This Paper Address?

In engineering, we often have data from various sources (simulations, experiments, different fidelities) that are crucial for building accurate models. However, these sources frequently have different input parameters, making it difficult to combine them into a single, unified model. This "heterogeneity" in input spaces limits our ability to leverage all available information, leading to less accurate predictions and hindering design optimization. Existing data fusion techniques struggle when sources don't share the same input variables, creating a gap in our ability to build comprehensive models from diverse data.

Methods

The proposed framework tackles heterogeneous data fusion in two stages:

  1. Heterogeneous Mapping (Input Mapping Calibration - IMC): This stage transforms the input spaces of different data sources into a common reference space. The paper uses a linear transformation (Ax + b) to achieve this, optimizing the transformation parameters by minimizing the difference between the outputs of the sources. This is a crucial step to align disparate input domains.
  2. Multi-Source Data Fusion (Latent Variable Gaussian Process - LVGP): After mapping, the data from all sources (now in a common input space) is fed into an LVGP model. What's novel here is that LVGP treats each data source as a qualitative variable. It then learns a low-dimensional latent space that captures the underlying similarities and differences between these sources, going beyond just the input mapping. This allows the model to understand how each source contributes to the overall prediction and provides interpretability through the latent space and a dissimilarity metric.

Datasets

The paper demonstrates its framework on three engineering case studies:

  • Cantilever Beam Design: Data for Rectangular Beams (RB), Hollow Rectangular Beams (HRB), and Hollow Circular Beams (HCB) were used. RB had the most data and served as the reference source.
  • Ellipsoidal Void Modeling: Data for 2D Ellipse (2DE), 3D Ellipse (3DE), and 3D Ellipse with Y-Rotation (3DER) were used, varying in complexity and simulation fidelity (elastic vs. plastic analysis). 2DE served as the reference.
  • Manufacturing Process Fusion of Ti6Al4V Alloys: Data from Laser Powder Bed Fusion (LPBF), Electron Beam Melting (EBM), and Friction Stir Welding (FSW) were used, with no common input parameters across sources. LPBF was the reference source.

The paper does not introduce new benchmark datasets but uses existing data from these engineering problems to validate its approach.

Key Insights and Ideas

  • Bridging the Input Gap: Linear mapping (IMC) is effective in aligning heterogeneous input spaces, but it doesn't fully capture all source-specific nuances.
  • LVGP as a Source-Aware Model: Treating data sources as qualitative variables in LVGP is key. It allows the model to learn source-specific relationships and dependencies that simple input mapping misses.
  • Interpretability through Latent Space: The learned latent space provides valuable insights into how different sources relate to each other. Sources with similar underlying characteristics (e.g., similar design shapes or manufacturing processes) cluster together.
  • Power in Data Scarcity: The framework shines when one or more sources have limited data. By transferring knowledge from data-rich sources, LVGP significantly improves predictions for data-poor sources.
  • Beyond Accuracy: Understanding Differences: The dissimilarity metric derived from the latent space helps quantify the differences between sources, offering a deeper understanding of the data fusion process.

Key Results

The proposed heterogeneous multi-source data fusion framework consistently outperformed conventional methods across all three case studies:

  • Improved Predictive Accuracy: LVGP achieved lower Normalized Root Mean Squared Error (NRMSE) compared to standard Gaussian Process (GP) models and source-unaware IMC+GP approaches. For instance, in the cantilever beam study, LVGP reduced NRMSE by 87.5% for training and 42.4% for testing data across all sources.
  • Superior Performance in Data-Scarce Scenarios: For the source with the least data (e.g., HCB in the beam study, 3DER in the void study, FSW in the alloy study), LVGP showed significant improvements. For example, it improved NRMSE by 52.3% over a single-source GP model for the HCB.
  • Interpretability: The latent space analysis revealed meaningful relationships between sources. For example, in the beam study, RB and HRB were close in the latent space, while HCB was further away, reflecting their structural similarities.
  • Generalizability: LVGP models demonstrated better generalization across all sources compared to source-unaware models.

Limitations and Future Work

  • Linear Mapping Assumption: The current IMC uses a linear transformation. The authors acknowledge that non-linear or kernel-based transformations could offer greater accuracy but come with increased complexity.
  • Computational Cost: While not explicitly detailed as a limitation, optimizing transformations and training LVGP models can be computationally intensive.
  • Future Work:
    • Non-linear/Kernel Mappings: Exploring more advanced mapping techniques for IMC.
  • Cost-Aware Bayesian Optimization: Integrating adaptive sampling to address data deficiency and cost more effectively.
  • Broader Applications: Extending the framework to areas like digital twins, multi-task learning, and federated learning.

Summary updated

BibTeX
@misc{university2024heterogenous,
  author = {Wei Chen, Northwestern University and Yigitcan Comlek and Sandipp Krishnan Ravi and Piyush Pandita and Sayan Ghosh and Liping Wang and Wei Chen},
  title = {{Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process}},
  year = {2024},
  eprint = {2407.11268},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.11268},
}
Figure