Stochastic relational processes: Efficient inference and applications

Ingo Thon, Niels Landwehr, Luc De RaedtMachine Learning 2011 · DOI · OpenReview

Abstract

Modeling dynamic environments with high relational complexity and large numbers of objects remains a significant challenge in artificial intelligence. This paper introduces CPT-L (Causal Probabilistic Time-Logic), a framework that integrates the expressive power of relational logic with the computational efficiency of Markov processes. By employing Binary Decision Diagrams (BDDs) and a novel "partially lifted" inference algorithm, CPT-L enables efficient modeling and exact inference in large-scale sequential domains, such as massively multiplayer online games (MMOGs), which were previously computationally prohibitive for statistical relational learning.

Introduction

Artificial Intelligence researchers often face a trade-off when modeling dynamic systems: standard sequence models like Hidden Markov Models (HMMs) or Dynamic Bayesian Networks (DBNs) are computationally efficient but struggle to represent relational complexity—worlds where the number of objects and their relationships change over time. Conversely, Statistical Relational Learning (SRL) techniques like Markov Logic are highly expressive but often fail to scale to real-world sequential data. CPT-L addresses this gap by targeting environments that are inherently relational, strongly stochastic, and involve a large number of objects.

The CPT-L Framework

CPT-L is a variant of CP-logic (Causal Probabilistic Logic) that treats transitions between world states as causal processes. It defines a probability distribution over sequences of relational interpretations under a Markov assumption. By focusing on causal rules (e.g., "If player A attacks city C, it succeeds with probability $p$"), the model is more intuitive for human specification and more efficient for machine processing than purely correlational models.

Efficient Inference via BDDs and Partial Lifting

To manage the potential exponential explosion of world transitions, CPT-L transforms the inference problem into a Boolean formula represented compactly by Binary Decision Diagrams (BDDs). This allows probability calculations to scale linearly with the BDD size.

The framework's primary innovation is partially lifted inference. Instead of grounding the entire model into a massive propositional network, the algorithm solves logical components at the first-order level first. By identifying which outcomes are logically determined by the current state before building the BDD, the computational workload is drastically reduced. Furthermore, the authors adapted the Expectation-Maximization (EM) algorithm to work directly with these BDD structures for efficient parameter learning.

Experimental Evaluation

The authors evaluated CPT-L across three distinct domains: a stochastic Blocks World with probabilistic "jiggling," real-world IRC chat room data monitored via PieSpy, and a live server of the MMOG Travian.

Relational representation of the Travian MMOG domain, showing alliances, cities, and conquest actions.

In the Travian domain, the model tracked 25,000 active players, their cities, and alliances over several months. As shown in the figure above, the domain is represented through logical predicates that define cities, alliances, and conquest events. This relational structure allows the model to capture complex interactions that propositional models would miss.

Key Results

  • Massive Speedups: The partially lifted inference algorithm achieved a 50x speedup over basic grounded versions in the Travian domain.
  • Scalability: CPT-L successfully performed exact inference in domains that would require a ground Bayesian Network with approximately 800,000 nodes.
  • Accuracy: In a meta-alliance classification task, CPT-L achieved 82.22% accuracy, rivaling state-of-the-art discriminative systems like BOOSTEDREAL while utilizing a simpler, more interpretable generative model.
  • Predictive Performance: The model predicted player conquest actions significantly better than random baselines, maintaining performance even several time steps into the future.

Conclusion and Future Work

CPT-L demonstrates that combining causal modeling with efficient BDD-based inference can bridge the gap between expressiveness and efficiency in relational domains. A current limitation is the assumption of full observability; future work will explore partial observability through techniques like particle filters. Additionally, while parameters are learned efficiently, the authors aim to automate structure learning—discovering the logical rules themselves from raw data—by integrating standard rule learners like Progol.

Summary updated

Abstract

One of the goals of artificial intelligence is to develop agents that learn and act in complex environments. Realistic environments typically feature a variable number of objects, relations amongst them, and non-deterministic transition behavior. While standard probabilistic sequence models provide efficient inference and learning techniques for sequential data, they typically cannot fully capture the relational complexity. On the other hand, statistical relational learning techniques are often too inefficient to cope with complex sequential data. In this paper, we introduce a simple model that occupies an intermediate position in this expressiveness/efficiency trade-off. It is based on CP-logic (Causal Probabilistic Logic), an expressive probabilistic logic for modeling causality. However, by specializing CP-logic to represent a probability distribution over sequences of relational state descriptions and employing a Markov assumption, inference and learning become more tractable and effective. Specifically, we show how to solve part of the inference and learning problems directly at the first-order level, while transforming the remaining part into the problem of computing all satisfying assignments for a Boolean formula in a binary decision diagram.We experimentally validate that the resulting technique is able to handle probabilistic relational domains with a substantial number of objects and relations.
BibTeX
@article{thon2011stochastic,
  author = {Ingo Thon and Niels Landwehr and Luc De Raedt},
  title = {{Stochastic relational processes: Efficient inference and applications}},
  journal = {Machine Learning},
  year = {2011},
  doi = {10.1007/s10994-010-5213-8},
}
Figure