Scalable Score Computation for Learning Multinomial Bayesian Networks over Distributed Data

Praveen Rao, Anas Katib, Kobus Barnard, Charles A. Kamhoua, Kevin A. Kwiat, Laurent NjillaWorkshops @ AAAI 2017 · OpenReview

Abstract

In this paper, we focus on the problem of learning a Bayesian network over distributed data stored in a commodity cluster. Specifically, we address the challenge of computing the scoring function over distributed data in a scalable manner, which is a fundamental task during learning. We propose a novel approach designed to achieve: (a) scalable score computation using the principle of gossiping; (b) lower resource consumption via a probabilistic approach for maintaining scores using the properties of a Markov chain; and (c) effective distribution of tasks during score computation (on large datasets) by synergistically combining well-known hashing techniques. Through theoretical analysis, we show that our approach is superior to a MapReduce-style computation in terms of communication bandwidth. Further, it is superior to the batch-style processing of MapReduce for recomputing scores when new data are available.
BibTeX
@inproceedings{rao2017scalable,
  author = {Praveen Rao and Anas Katib and Kobus Barnard and Charles A. Kamhoua and Kevin A. Kwiat and Laurent Njilla},
  title = {{Scalable Score Computation for Learning Multinomial Bayesian Networks over Distributed Data}},
  booktitle = {Workshops @ AAAI},
  year = {2017},
  url = {https://openreview.net/forum?id=rJWpVk-OZr},
}

Loading related content...

Figure