BayesLDM: A Domain-Specific Language for Probabilistic Modeling of Longitudinal Data

Karine Tung, Steven De La Torre, Mohamed El Mistiri, Rebecca Braga De Braganca, Eric B. Hekler, Misha Pavel, Daniel E. Rivera, Pedja Klasnja, Donna Spruijt-Metz, Benjamin M. Marlin

TL;DR

This paper introduces BayesLDM, a domain-specific language and compiler for probabilistic modeling of longitudinal data. BayesLDM simplifies the specification of complex time-series models, particularly Dynamic Bayesian Networks (DBNs), and automatically generates optimized inference code, significantly accelerating the modeling workflow.

What Problem Does This Paper Address?

Modeling longitudinal data, especially in domains like mobile health, presents significant challenges. These include dealing with extensive missingness, high data heterogeneity, and limited data volumes. Traditional approaches often require extensive manual coding for efficient probabilistic inference, which is time-consuming and error-prone. The authors aim to bridge the gap between conceptual model specification and computationally efficient inference code, making complex longitudinal data analysis more accessible and faster.

Methods

BayesLDM is a two-part system:

  1. High-Level Modeling Language: This language allows users to specify probabilistic models, particularly Bayesian Networks and Dynamic Bayesian Networks (DBNs), using an intuitive, Python-like syntax. It includes features for declarative specification of complex time-series dependencies and array indexing.
  2. Optimizing Compiler: This compiler takes the model specification and available data to generate optimized probabilistic program code. It leverages underlying inference libraries (NumPyro in this work) and applies optimizations like implicit replication (using plate) and recurrence (using scan) for efficient sampling. Crucially, it automates Bayesian imputation for missing data.

Datasets

The paper uses benchmark models from the "Statistical Rethinking" book for evaluating representational efficiency. For the case study, they use real data from a physical activity intervention trial in the mobile health domain, collected over 38 weeks with weekly observations of physical activity and related behavioral constructs from 10 participants.

Key Insights and Ideas

  • Abstraction for Complexity: BayesLDM's core innovation is abstracting away the low-level details of probabilistic inference code generation. This allows researchers to focus on model specification and interpretation rather than implementation intricacies.
  • Efficiency Through Domain Specificity: By focusing on longitudinal data and DBNs, BayesLDM can apply specific optimizations (like scan and plate) that are not easily accessible or are more cumbersome to implement in general-purpose probabilistic programming languages.
  • Automated Missing Data Handling: The system's ability to automatically handle missing data through Bayesian imputation is a significant practical advantage, reducing a common pain point in real-world data analysis.
  • Bridging the Gap: BayesLDM effectively translates high-level model ideas into executable, optimized code, accelerating the iterative process of model development and refinement.

Key Results

  • Representational Efficiency: BayesLDM models were found to be significantly more compact, requiring 2-4 times fewer lines of code and 4-6 times fewer characters compared to Stan, PyMC3, and NumPyro for benchmark models.
  • Inference Scalability: For an AR(1) model with missing data, BayesLDM's optimized inference code (using scan) achieved inference times comparable to or faster than optimized NumPyro implementations and significantly faster than Stan and PyMC3, especially as data length increased.
  • Case Study: Applying BayesLDM to mobile health data demonstrated its utility in modeling complex relationships and handling missingness, leading to insights about the effectiveness of different hypothesized relationships in physical activity planning. The DBN model provided a substantially better fit than a simpler AR(1) model.

Limitations and Future Work

  • Meta-Optimization: The authors suggest that BayesLDM could potentially compile into implementations for different probabilistic programming languages based on model and data characteristics, a direction for future work.
  • Discrete Variables: While NUTS (the default sampler) is for continuous variables, BayesLDM supports discrete variables using NumPyro's variable elimination, but this could be further explored.
  • Model Complexity vs. Data Volume: In the case study, the added complexity of a multi-level extension (DBN-ML) did not always yield a better fit according to BIC, suggesting that the benefits of complexity are data-dependent.
  • Further Model Refinement: The authors note that BayesLDM can be used to extend models by incorporating additional observed variables, analyzing residuals, or modeling missing data mechanisms.

Summary updated

Abstract

In this paper we present BayesLDM, a system for Bayesian longitudinal data modeling consisting of a high-level modeling language with specific features for modeling complex multivariate time series data coupled with a compiler that can produce optimized probabilistic program code for performing inference in the specified model. BayesLDM supports modeling of Bayesian network models with a specific focus on the efficient, declarative specification of dynamic Bayesian Networks (DBNs). The BayesLDM compiler combines a model specification with inspection of available data and outputs code for performing Bayesian inference for unknown model parameters while simultaneously handling missing data. These capabilities have the potential to significantly accelerate iterative modeling workflows in domains that involve the analysis of complex longitudinal data by abstracting away the process of producing computationally efficient probabilistic inference code. We describe the BayesLDM system components, evaluate the efficiency of representation and inference optimizations and provide an illustrative example of the application of the system to analyzing heterogeneous and partially observed mobile health data.
BibTeX
@inproceedings{tung2022bayesldm,
  author = {Karine Tung and Steven De La Torre and Mohamed El Mistiri and Rebecca Braga De Braganca and Eric B. Hekler and Misha Pavel and Daniel E. Rivera and Pedja Klasnja and Donna Spruijt-Metz and Benjamin M. Marlin},
  title = {{BayesLDM: A Domain-Specific Language for Probabilistic Modeling of Longitudinal Data}},
  booktitle = {CHASE},
  year = {2022},
  url = {https://openreview.net/forum?id=FaJbTdcqrX},
}
Figure