Knowledge Transfer in Engineering Fleets: Hierarchical Bayesian Modelling for Multi-Task Learning

Lawrence A. Bull, Maharshi Harshadbhai Dhada, Olof Steinert, Tony Lindgren, Ajith Kumar Parlikad, Andrew B. Duncan, Mark GirolamiarXiv 2022 · arXiv · OpenReview

Abstract

A population-level analysis is proposed to address data sparsity when building predictive models for engineering infrastructure. Utilising an interpretable hierarchical Bayesian approach and operational fleet data, domain expertise is naturally encoded and shared between different sub-groups representing use-type, component, or operating conditions. By exploiting domain expertise to constrain the model via informed prior distributions, the methodology automatically shares information between similar assets. This approach improves survival analysis for truck fleets and power prediction for wind farms by learning correlated functions over the fleet in a combined inference. Groups with incomplete data automatically borrow statistical strength from data-rich groups, enabling Bayesian transfer learning that remains physically interpretable.

Introduction

In structural health monitoring and asset management, data is frequently sparse, incremental, and expensive. Traditional predictive modeling often faces a dichotomy: building independent models for each asset, which are unreliable with limited data, or pooling all assets into a single "average" model, which ignores critical individual differences. This paper introduces a hierarchical Bayesian framework that treats different sub-groups of assets as related tasks in a multi-task learning (MTL) setup, allowing assets to "borrow statistical strength" from one another while preserving individual nuances.

The Challenge of Data Sparsity

The primary motivation for this work is the extreme imbalance in operational data. In a typical engineering fleet, some assets may have years of history, while others—such as a newly commissioned truck or a turbine in a unique location—may have only a handful of observations.

Table 1: Distribution of observations across eight truck sub-groups, highlighting extreme data sparsity in groups $N_5$ through $N_8$.

As shown in Table 1, sub-groups can range from 180 observations to as few as a single data point ($N_8 = 1$). Standard statistical methods fail in the latter case, but by leveraging correlations across the fleet, meaningful predictions can still be generated.

Methodology: Hierarchical Bayesian Multi-Task Learning

The authors employ Hierarchical Bayesian Modelling (HBM) framed as an MTL problem. Unlike "black-box" machine learning models, this approach encodes engineering expertise directly into the model structure through domain-informed priors (e.g., Gompertz distributions for survival or segmented linear regression for power curves).

Figure 2: Plate notation for the hierarchical Bayesian multi-task learning framework, illustrating how individual observations $y_{ik}$ are influenced by both group-specific ($\alpha_k$) and population-wide ($\beta, \sigma$) parameters.

Key technical components include:

  • Mixed-Effects Models: The framework distinguishes between fixed effects (parameters shared across the entire population, like physics-based maximum power) and random effects (parameters that vary between sub-groups).
  • Semi-Parametric Adjustments: B-splines are integrated into the hierarchy to capture complex, non-linear behaviors that simple physics models might miss.
  • Probabilistic Inference: Markov Chain Monte Carlo (MCMC) sampling via the Stan programming language is used to estimate posterior distributions, providing a full quantification of uncertainty.

Key Results and Discussion

The methodology was validated on two real-world datasets: Scania CV truck fleet survival data and Visualwind SCADA power data.

  1. Predictive Accuracy: In the truck survival case study, the MTL approach achieved a predictive log-likelihood of 409.57, significantly outperforming Single-Task Learning (355.47) and Complete Pooling (-43.49).
  2. Variance Reduction: The model achieved a 90% average reduction in standard deviation for linear weights in sparse truck domains. For wind turbines, uncertainty in maximum power estimation for data-poor units was reduced by up to 82%.
  3. Robustness: The model successfully made meaningful predictions for the truck sub-group with only one training data point by leveraging the population-level knowledge learned from the rest of the fleet.

Conclusion

By using parametric models, the learned "knowledge" remains human-readable, allowing engineers to inspect which assets share information and for which physical parameters. This decision-centric approach to uncertainty enables operators to perform "Value of Information" analysis, deciding whether to invest in more sensors or act on current model certainties. Future work will explore unsupervised clustering to automatically categorize assets into sub-fleets, further automating the knowledge transfer process.

Summary updated

Abstract

A population-level analysis is proposed to address data sparsity when building predictive models for engineering infrastructure. Utilising an interpretable hierarchical Bayesian approach and operational fleet data, domain expertise is naturally encoded (and appropriately shared) between different sub-groups, representing (i) use-type, (ii) component, or (iii) operating condition. Specifically, domain expertise is exploited to constrain the model via assumptions (and prior distributions) allowing the methodology to automatically share information between similar assets, improving the survival analysis of a truck fleet and power prediction in a wind farm. In each asset management example, a set of correlated functions is learnt over the fleet, in a combined inference, to learn a population model. Parameter estimation is improved when sub-fleets share correlated information at different levels of the hierarchy. In turn, groups with incomplete data automatically borrow statistical strength from those that are data-rich. The statistical correlations enable knowledge transfer via Bayesian transfer learning, and the correlations can be inspected to inform which assets share information for which effect (i.e. parameter). Both case studies demonstrate the wide applicability to practical infrastructure monitoring, since the approach is naturally adapted between interpretable fleet models of different in situ examples.
BibTeX
@misc{bull2022knowledge,
  author = {Lawrence A. Bull and Maharshi Harshadbhai Dhada and Olof Steinert and Tony Lindgren and Ajith Kumar Parlikad and Andrew B. Duncan and Mark Girolami},
  title = {{Knowledge Transfer in Engineering Fleets: Hierarchical Bayesian Modelling for Multi-Task Learning}},
  year = {2022},
  eprint = {2204.12404},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.12404},
}
Figure