BayesLDM: A Domain-specific Modeling Language for Probabilistic Modeling of Longitudinal Data

Karine Tung, Steven De La Torre, Mohamed El Mistiri, Rebecca Braga De Braganca, Eric B. Hekler, Misha Pavel, Daniel E. Rivera, Pedja Klasnja, Donna Spruijt-Metz, Benjamin M. Marlin

Introduction

Modeling multivariate longitudinal data is a cornerstone of modern behavioral science and mobile health (mHealth) research. However, researchers in these fields face a "triple threat" of challenges: pervasive missing data, high inter-individual heterogeneity, and general data scarcity. While general-purpose Probabilistic Programming Languages (PPLs) like Stan, PyMC3, and NumPyro offer the flexibility to address these issues, they often impose a steep learning curve and a significant "implementation gap." Translating a conceptual model into verbose, low-level code is error-prone and frequently results in computationally inefficient implementations, particularly when handling complex temporal dependencies.

To bridge this gap, we present BayesLDM, a domain-specific language (DSL) and optimizing compiler designed specifically for Dynamic Bayesian Networks (DBNs). BayesLDM simplifies the modeling process by providing high-level abstractions that automatically handle missing data and optimize temporal loops for high-performance inference.

Methods: The BayesLDM System

The BayesLDM framework consists of two primary components: a declarative DSL and an optimizing compiler.

Declarative DSL

The DSL employs a Python-like syntax that allows users to define variables, indices, and probability distributions with minimal boilerplate. It is specifically tailored for DBNs, enabling the concise definition of temporal dependencies (e.g., $x[t]$ conditioned on $x[t-1]$). This high-level abstraction allows researchers to focus on the model structure rather than the underlying implementation details.

Optimizing Compiler

The core of BayesLDM is its compiler, built using the textX parser. The compiler transforms the high-level model text into an Abstract Syntax Tree (AST), which is then used to generate optimized Python code targeting the NumPyro library.

The BayesLDM system architecture, illustrating the pipeline from high-level model text to optimized inference execution.

The compiler implements several key innovations:

  • Automatic Imputation: The system automatically detects missing values (NaN) in the input data and generates the necessary logic to treat these as latent variables. This abstracts away the complex marginalization mathematics typically required for Bayesian imputation.
  • Structural Optimizations: The compiler leverages NumPyro’s plate primitive for independent variables and the scan primitive for temporal sequences. By using scan, the underlying JAX engine can compile time-series loops into efficient XLA (Accelerated Linear Algebra) kernels, avoiding the significant overhead associated with standard Python loops.

Evaluation and Results

We evaluated BayesLDM across benchmark models, synthetic datasets, and a real-world mHealth case study.

Code Compactness

BayesLDM significantly reduces the cognitive load on researchers. In tests using standard models from Statistical Rethinking, BayesLDM models required an average of only 7.3 lines of code, compared to 23.3 lines for equivalent Stan models. Overall, BayesLDM code was approximately 50% more compact than both Stan and NumPyro implementations.

Inference Performance

Scalability tests using synthetic autoregressive (AR(1)) data demonstrated that BayesLDM matches the performance of hand-optimized NumPyro code. As illustrated in the runtime comparison below, for longer time series ($T=300$), BayesLDM maintains a significant speed advantage over Stan and PyMC3, which often struggle with the computational overhead of long temporal sequences.

Runtime comparison for T=300 across varying missingness rates, demonstrating BayesLDM's efficiency relative to other PPL backends.

Scientific Utility

In an mHealth case study involving physical activity data, BayesLDM enabled rapid iteration through a hierarchy of models. This flexibility allowed for the discovery that a multi-level DBN model provided a superior fit for behavioral data compared to a simple AR(1) model, helping to validate hypotheses regarding the influence of planning on physical activity.

Conclusion

BayesLDM demonstrates that high-level abstractions in probabilistic modeling do not have to come at the cost of performance. By understanding the specific structure of longitudinal models, the BayesLDM compiler can apply expert-level optimizations automatically. Future work will focus on expanding distribution support, improving the handling of discrete missing data, and exploring meta-optimization techniques to select the most efficient PPL backend for a given model.

Summary updated

Abstract

In this paper we present BayesLDM, a library for Bayesian longitudinal data modeling consisting of a high-level modeling language with specific features for modeling complex multivariate time series data coupled with a compiler that can produce optimized probabilistic program code for performing inference in the specified model. BayesLDM supports modeling of Bayesian network models with a specific focus on the efficient, declarative specification of dynamic Bayesian Networks (DBNs). The BayesLDM compiler combines a model specification with inspection of available data and outputs code for performing Bayesian inference for unknown model parameters while simultaneously handling missing data. These capabilities have the potential to significantly accelerate iterative modeling workflows in domains that involve the analysis of complex longitudinal data by abstracting away the process of producing computationally efficient probabilistic inference code. We describe the BayesLDM system components, evaluate the efficiency of representation and inference optimizations and provide an illustrative example of the application of the system to analyzing heterogeneous and partially observed mobile health data.
BibTeX
@inproceedings{tung2022bayesldm,
  author = {Karine Tung and Steven De La Torre and Mohamed El Mistiri and Rebecca Braga De Braganca and Eric B. Hekler and Misha Pavel and Daniel E. Rivera and Pedja Klasnja and Donna Spruijt-Metz and Benjamin M. Marlin},
  title = {{BayesLDM: A Domain-specific Modeling Language for Probabilistic Modeling of Longitudinal Data}},
  booktitle = {CHASE},
  year = {2022},
  url = {https://openreview.net/forum?id=FaJbTdcqrX},
}
Figure