Multi-Fidelity Data Integration

19 concepts19 papers1 proposal

When building predictive models of physical, environmental, or engineering systems, researchers face a fundamental data collection dilemma. On one hand, they can gather high-fidelity data: precise physical experiments, regulatory-grade sensor measurements, or fine-grained numerical simulations. This data is highly accurate but painfully expensive and sparse. On the other hand, they can collect low-fidelity data: coarse computer simulations, crowdsourced observations, or cheap consumer sensors. This data is abundant and affordable but riddled with noise and systematic errors.

As highlighted by Kelling et al. and Peterson et al. in ecological monitoring, scientific discovery increasingly relies on breaking this tension. The goal of multi-fidelity data integration is to build a bridge between these worlds, leveraging the massive volume of low-fidelity data to learn global trends while anchoring the final predictions to high-fidelity ground truths.

Why Naive Data Fusion Fails

The simplest approach to combining multi-source data is to pool all observations into a single dataset and train a standard machine learning model. This strategy fails for two primary reasons:

  1. Volume Imbalance: Because low-fidelity data is cheap, it often outnumbers high-fidelity data by orders of magnitude. A standard loss function will optimize for the overwhelming volume of low-fidelity data, treating the sparse high-fidelity points as mere outliers.
  2. Systematic Bias: Low-fidelity data rarely suffers from simple, zero-mean random noise. Instead, it exhibits systematic biases. For example, a coarse simulation might consistently underestimate a fluid dynamics property, or a low-cost air quality sensor might drift based on ambient humidity.

If a model treats low-fidelity approximations and high-fidelity ground truths identically, it cannot allocate computational or sampling resources strategically (Baheri et al.). As Burnaev et al. point out, failing to explicitly model these systematic biases misleads standard classifiers and regression models, degrading rather than improving performance.

The Core Mechanism: Learning the Discrepancy

To successfully integrate multi-fidelity data, we must shift our objective. Instead of learning the high-fidelity function $f_{\text{high}}(x)$ entirely from scratch, we model it as a transformation of the low-fidelity function $f_{\text{low}}(x)$.

In its simplest linear form, the relationship can be written as:

$$f_{\text{high}}(x) = \rho(x) \cdot f_{\text{low}}(x) + \delta(x)$$

Here, $\rho(x)$ is a scaling factor, and $\delta(x)$ is a discrepancy function that explicitly captures the systematic bias of the low-fidelity source.

By structuring the problem this way, the model uses the abundant low-fidelity data to learn the rough, global topology of the system. It then uses the sparse high-fidelity data solely to learn $\rho(x)$ and $\delta(x)$. This requires significantly fewer high-fidelity samples than trying to learn the entire complex surface from scratch.

This mechanism is frequently implemented using Multi-Fidelity Gaussian Processes (MFGPs). By placing a joint prior over both fidelity levels, the model naturally propagates uncertainty from the low-fidelity data to the final high-fidelity predictions.

[Low-Fidelity Data]  --->  Learn Global Topology f_low(x)
                                   |
                                   v
[High-Fidelity Data] --->  Learn Discrepancy δ(x)  ---> [High-Fidelity Prediction]

McDonald et al. leverage this exact framework for robotic coverage control, allowing autonomous agents to map environments efficiently by fusing cheap, global satellite imagery with expensive, on-site physical samples. Similarly, Meng et al. use MFGPs to discover the underlying differential equations of physical systems, using the Gaussian process's built-in uncertainty quantification to weight regression problems and handle heavily corrupted data.

From an algorithmic perspective, this information transfer can be viewed as a variance-reduction technique. Sun et al. formalize this using Vector-Valued Control Variates (vv-CVs). By using matrix-valued Stein kernels, they transfer information between related integration tasks, using cheap low-fidelity proxies to systematically lower the variance of high-fidelity Monte Carlo estimates.

Fusing Data with Unknown Reliability

The classic multi-fidelity formulation assumes that we know the exact hierarchy of our data sources—we know precisely which sensor is "high-fidelity" and which is "low-fidelity." But in real-world deployments, data often arrives as a heterogeneous soup from hundreds of uncalibrated sources, such as a mix of professional drone sensors and consumer smartphones (Nex et al.).

When the fidelity levels are unknown, models must dynamically infer the reliability of each data point. This requires modeling heteroscedastic noise—noise that varies across different samples or sources.

Hong et al. address this with HePPCAT, a probabilistic Principal Component Analysis (PCA) framework. Instead of assuming a uniform noise floor, HePPCAT jointly estimates the underlying low-dimensional signal and the specific noise level of each individual sample. This prevents highly corrupted, low-cost measurements from drowning out the true signal during dimensionality reduction.

In spatial contexts, such as urban air quality monitoring, we can use structural and geometric constraints to infer fidelity. Yu et al. introduce a self-supervised "fidelity score" within a geometry-aware graph neural network. The network automatically learns to trust high-quality regulatory stations over dense, noisy consumer sensors without ever needing an explicit, clean "ground truth" dataset for training.

Similarly, Zheng et al. resolve the discrepancy between low-accuracy satellite data and sparse ground sensors by treating the Earth's surface as a continuous manifold. Enforcing spatial smoothness across this manifold allows the model to accurately "fill in the gaps" where high-fidelity data is missing.

Scaling to High Dimensions and Deep Surrogates

While Gaussian processes offer elegant uncertainty quantification, they scale poorly to high-dimensional outputs or massive datasets due to their $O(N^3)$ computational complexity. To scale multi-fidelity integration to complex engineering systems, researchers have turned to deep learning architectures.

Yang et al. develop conditional deep surrogate models designed specifically for stochastic, high-dimensional multi-fidelity systems, bypassing the computational bottlenecks of non-parametric methods. To preserve Bayesian uncertainty benefits at scale, Bhouri et al. introduce Randomized Prior Networks (RPNs) into multi-fidelity Bayesian optimization. This allows the surrogate model to handle massive vectors, images, or partial differential equation (PDE) solutions while efficiently propagating uncertainty across fidelity levels.

This scalability is vital for modern scientific discovery platforms. As highlighted by Cisse et al., navigating the trade-offs between cheap numerical simulations and expensive physical experiments via language-based Bayesian optimization assistants represents a major frontier in automating scientific workflows.

Resilience to Intermittent Data Streams

A final, practical hurdle in multi-fidelity systems is operational robustness. In the real world, high-fidelity data streams are prone to sudden outages or irregular sampling intervals.

Evaluating how deep learning models handle these gaps, Gauch et al. show that when transitioning between local sensors and coarse satellite estimates in hydrological forecasting, a "masked mean" architecture performs best. By averaging the embeddings of whatever data sources are currently available, the model maintains high predictive accuracy even when the high-fidelity data stream completely drops out.

Whether solving differential equations (Karniadakis et al.) or tracking global environmental changes, the core philosophy of multi-fidelity integration remains the same: do not treat all data as equal. By explicitly modeling the discrepancy, noise structure, and relative reliability of heterogeneous data streams, we can build models that are both globally comprehensive and locally precise.

Key Foundational Work

Accurate Integration of Aerosol Predictions by Smoothing on a Manifold (2014) — contributed 1 concept elements to this direction.
A Human/Computer Learning Network to Improve Biodiversity Conservation and Research (2013) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 19 papers
  2. Version 1 · 2026-10-04 · 23 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_multi_fidelity_data_integration_9f9e85ca,
  title = {multi-fidelity data integration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-fidelity-data-integration/rdr_9f9e85ca803442ffa8d98c5de8eb1e0d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_multi_fidelity_data_integration_9f9e85ca_v2,
  title = {multi-fidelity data integration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-fidelity-data-integration/rdr_9f9e85ca803442ffa8d98c5de8eb1e0d/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake