Comparison of parametric and non‐parametric Bayesian inference for fusing sensory estimates in physiological time‐series analysis

Tingting Zhu, Hamza Javed, David A. CliftonHealthcare 2021 · DOI

TL;DR

This paper tackles the challenge of reliably estimating physiological signals like respiratory rate (RR) from wearable sensors, where multiple algorithms provide noisy and potentially biased estimates. The authors propose and compare two Bayesian methods to fuse these estimates, demonstrating that their models can infer more accurate ground truth than existing state-of-the-art approaches, even with incomplete data.

What Problem Does This Paper Address?

Wearable sensors are generating vast amounts of physiological data, promising personalized healthcare. However, automatically labeling this data (e.g., identifying abnormal respiratory rates) is crucial but difficult. Current automated algorithms are often less reliable than human experts, and their outputs can vary significantly due to inter- and intra-subject variability. When "ground truth" labels are unavailable, it's hard to know which algorithm to trust or how to combine their outputs for a precise estimate. This paper addresses the problem of fusing estimates from multiple, potentially imperfect, algorithms to achieve a more accurate and robust inference of the underlying physiological signal, specifically respiratory rate.

Methods

The core of this work lies in applying and comparing two Bayesian graphical models for fusing sensory estimates:

  • Independent Annotator Model (IAM): This is a parametric, fully Bayesian model that treats each algorithm's output as an independent estimate of the ground truth. It jointly models the bias and precision of each annotator (algorithm) and infers the latent ground truth.
  • Correlated Annotator Model (CAM): This model extends the IAM by incorporating a multivariate normal distribution to account for potential correlations between annotators. This allows for penalizing algorithms that are highly correlated but have large variances and biases, while giving more weight to expert annotators with small variances and biases.

Both models utilize Gibbs sampling for inference, which is particularly useful for handling incomplete data (i.e., when some algorithms don't provide estimates for all time points). The authors also compare their methods against established parametric (EM-R, STAPLE) and non-parametric (Hierarchical Gaussian Processes - HGPs) approaches.

Datasets

The proposed models were validated on two publicly available biomedical datasets:

  • CapnoBase dataset: Contains 42 PPG recordings from 42 subjects (paediatric and adult), each 8 minutes long.
  • BIDMC dataset: Contains PPG recordings from 53 adult subjects, also 8 minutes long.

These datasets provide physiological time-series data (specifically, PPG recordings) from which respiratory rate (RR) can be estimated. The authors extracted three respiratory-induced modulation time-series (Amplitude Modulation, Baseline Wander, and Frequency Modulation) from each PPG recording and used two established methods (Fourier spectral analysis and AR modeling) to generate multiple RR estimates per window, simulating the scenario of having multiple "algorithms" or annotators.

Key Insights and Ideas

  • The Power of Joint Modeling: By jointly estimating the bias and precision of each algorithm, the Bayesian models can better understand the reliability of each source and thus infer a more accurate ground truth.
  • Correlation Matters: Explicitly modeling correlations between algorithms (CAM) can lead to improved RR estimates compared to assuming independence (IAM), especially when algorithms are not truly independent.
  • Bayesian Robustness: The strongly Bayesian approach, particularly with Gibbs sampling, provides a principled way to handle missing data, which is a common issue in real-world sensor applications due to sensor failures or intermittent operation.
  • Beyond Simple Averaging: The proposed methods offer a more sophisticated way to fuse estimates than naive averaging, accounting for the individual characteristics of each estimation algorithm.

Key Results

  • Outperforming State-of-the-Art: Both the IAM and CAM models demonstrated improved accuracy in estimating respiratory rate compared to existing methods, including EM-R, STAPLE, and Hierarchical Gaussian Processes (HGPs).
  • CAM's Advantage: The CAM model achieved the lowest Mean Absolute Error (MAE) on the CapnoBase dataset, suggesting its ability to leverage inter-algorithm correlations is beneficial.
  • IAM's Performance: The IAM model also performed well, particularly on the BIDMC dataset, and both proposed models consistently outperformed HGPs.
  • Confidence Intervals: The Bayesian models provide 95% confidence intervals for their estimates, offering a measure of uncertainty that is valuable for clinical interpretation. HGPs, in contrast, showed larger uncertainty due to biased algorithms.
  • Efficiency: The models are deemed efficient enough for real-time applications.

Limitations and Future Work

  • Model Complexity: While the authors note that their parametric models have a complexity of O(n³) (where n is the number of samples), similar to HGPs, further optimization of inference steps could reduce computational complexity.
  • Assumption of Gaussianity: The models assume that biases are drawn from a Gaussian distribution. The authors acknowledge that this choice might be dataset-dependent, and other distributions could be explored.
  • Further Correlation Modeling: While CAM models correlations, the paper suggests that exploring more complex correlation structures could be a direction for future work.
  • Generalizability: While validated on RR estimation, future work could explore the application of these fusion techniques to other physiological time-series data and different types of wearable sensor data.

Summary updated

Abstract

The rapid proliferation of wearable devices for medical applications has necessitated the need for automated algorithms to provide labelling of physiological time-series data to identify abnormal morphology. However, such algorithms are less reliable than gold-standard human expert labels (where the latter are typically difficult and expensive to obtain), due to their large inter- and intra-subject variabilities. Actions taken in response to these algorithms can therefore result in sub-optimal patient care. In a typical scenario where only unevenly sampled continuous or numeric estimates are provided, without access to the "ground truth", it is challenging to choose which algorithms to trust and which to ignore, or even how to merge the outputs from multiple algorithms to form a more precise final estimate for individual patients. In this work, the novel application of two previously proposed parametric fully-Bayesian graphical models is demonstrated for fusing labels from (i) independent and (ii) potentially correlated algorithms, validated on two publicly available datasets for the task of respiratory rate (RR) estimation. These unsupervised models aggregate RR labels and estimate jointly the assumed bias and precision of each algorithm. Fusing estimates in this way may then be used to infer the underlying ground truth for individual patients. It is shown that modelling the latent correlations between algorithms improves the RR estimates, when compared to commonly employed strategies in the literature. Finally, it is demonstrated that the adoption of a strongly Bayesian approach to inference using Gibbs sampling results in improved estimation over the current state-of-the-art (e.g. hierarchical Gaussian processes) in physiological time-series modelling.
BibTeX
@inproceedings{zhu2021comparison,
  author = {Tingting Zhu and Hamza Javed and David A. Clifton},
  title = {{Comparison of parametric and non‐parametric Bayesian inference for fusing sensory estimates in physiological time‐series analysis}},
  booktitle = {Healthcare},
  year = {2021},
  doi = {10.1049/htl2.12003},
}
Figure