The Libra toolkit for probabilistic models

Daniel Lowd, Amirmohammad Rooshenas

Abstract: The Libra Toolkit is an open-source software package designed for learning and inference in discrete probabilistic models. It distinguishes itself from existing toolkits by focusing on tractable models like sum-product networks and arithmetic circuits, which allow for efficient exact inference. Libra provides a comprehensive suite of tools for structure learning and inference, leveraging OCaml for a combination of safety and high performance. Benchmarks show that Libra is competitive with state-of-the-art C++ libraries in both speed and accuracy.

Introduction: Addressing the Tractability Gap

While numerous open-source toolkits exist for probabilistic graphical models—such as BNT, libDAI, and OpenGM2—most are restricted to traditional representations like Bayesian Networks (BNs) or Markov Networks (MNs). In these standard models, exact inference is frequently computationally intractable (NP-hard).

The Libra Toolkit was developed to fill a critical gap: the lack of accessible software for tractable probabilistic models. These models are specifically structured to ensure that exact inference can be performed efficiently. Furthermore, Libra addresses the need for robust structure learning algorithms that can exploit local structures, such as decision trees or conjunctive features, which are often overlooked by existing tools.

Comparison of Libra with other probabilistic modeling toolkits across representation, inference, and learning capabilities.

Methods and Representations

Libra provides a modular suite of command-line programs for both learning and inference, implemented in OCaml to balance high-level safety with native machine-code performance. This architecture allows users to combine different learning and inference components seamlessly.

Representations and Learning

Beyond standard BNs, MNs, and Dependency Networks (DNs), Libra uniquely supports Sum-Product Networks (SPNs) and Arithmetic Circuits (ACs). Its standout feature is its suite of structure learning algorithms, including the ID-SPN algorithm for SPNs and methods for learning tractable BN/MN structures via ACs. It also includes unique methods for learning MN structure from DNs, providing a versatile toolkit for diverse modeling needs.

Inference

The toolkit offers both exact inference (e.g., AC variable elimination) and approximate inference (Gibbs sampling, mean field, loopy belief propagation, and max-product). Unlike tabular methods that iterate through all states, Libra's inference engines are designed to exploit local structure—such as tree-structured factors—to significantly accelerate processing, especially in high-treewidth models that would otherwise be computationally prohibitive.

Empirical Performance

To evaluate performance, Libra was benchmarked against libDAI, a highly optimized C++ library, using synthetic grid-structured Markov Networks ranging from 4x4 to 80x80. These networks are standard benchmarks for inference due to their complex connectivity.

Runtime comparison between Libra and libDAI for Belief Propagation and Gibbs sampling on grid-structured Markov Networks.

The results demonstrate that Libra’s implementations of Belief Propagation (BP) and Gibbs sampling are at least as fast as libDAI. Notably, as the grid size increases, Libra maintains high efficiency, even outperforming libDAI in some Gibbs sampling scenarios. Furthermore, the accuracy between the two toolkits remained equivalent under identical parameter settings, proving that Libra provides a high-performance alternative without sacrificing precision.

Conclusion and Future Work

Libra successfully lowers the barrier to entry for using tractable models that guarantee efficient inference. While currently limited to discrete domains and lacking support for dynamic Bayesian networks or EM-based learning with missing data, its modular architecture is designed for community expansion. Future developments aim to integrate continuous variables and broader model types to further extend the toolkit's utility in the machine learning community.

Summary updated

BibTeX
@article{lowd2015libra,
  author = {Daniel Lowd and Amirmohammad Rooshenas},
  title = {{The Libra toolkit for probabilistic models}},
  journal = {J. Mach. Learn. Res.},
  year = {2015},
}
Figure