A Factor Graph Description of Deep Temporal Active Inference

Bert de Vries, Karl J. FristonFrontiers Comput. Neurosci 2017 · DOI

TL;DR

This paper introduces Forney-style factor graphs (FFGs) as a powerful and intuitive framework for describing and simulating deep temporal active inference (DTAI) models. By leveraging FFGs, the authors demonstrate how complex active inference processes, including policy selection, can be elegantly represented and computationally executed through message passing, offering a biologically plausible computational theory for how agents interact with their environment.

What Problem Does This Paper Address?

Active inference, a framework rooted in the Free Energy Principle, posits that biological agents maintain their existence by minimizing variational free energy under a generative model of their environment. While the theoretical underpinnings are well-established, simulating and understanding complex active inference processes, especially those involving deep temporal dependencies and hierarchical control, presents significant computational and representational challenges. Specifically, specifying these models and automating the inference (free energy minimization) process for them is difficult. Existing approaches often require manual derivation of inference equations for each new model, hindering rapid exploration and simulation. This paper aims to bridge this gap by providing a unified graphical framework (FFGs) that simplifies model specification and automates inference for complex active inference models.

Methods

The core contribution of this paper is the application and exposition of Forney-style factor graphs (FFGs) for active inference. FFGs are a type of graphical model that visually represent the factorization of a probability distribution.

  • Model Specification: FFGs provide an intuitive way to depict the generative model of an agent's environment and its internal processes. Variables are represented by edges, and functions (factors) by nodes. The paper shows how to construct FFGs for linear Gaussian dynamical systems (LGDS) and then extends this to multi-scale hierarchical dynamical (MSHD) systems, culminating in a deep temporal active inference (DTAI) model.
  • Inference Automation: FFGs facilitate automated inference through message passing algorithms, primarily the sum-product algorithm. This algorithm iteratively passes "messages" (representing beliefs or probability distributions) between nodes and edges. The paper details how this message passing can be used to compute marginal probabilities, which is equivalent to minimizing free energy.
  • Closing-the-Box Procedure: A key technique demonstrated is the "closing-the-box" procedure, which allows for hierarchical abstraction by treating subgraphs as composite nodes. This is crucial for managing the complexity of deep and hierarchical models.
  • Variational Message Passing (VMP): For situations where exact inference is intractable (e.g., continuous variables or large state spaces), the paper introduces variational message passing, an approximate inference method derived from minimizing free energy.
  • Deep Temporal Active Inference (DTAI) Model: The paper specifically constructs an FFG for a DTAI model, which involves multiple layers of temporal processing. A key aspect is the representation of a "generative policy model" that specifies desired future states and observations, and how expected free energy minimization within this policy model drives action selection.

Datasets

Not discussed in the paper. The paper focuses on theoretical modeling and simulation rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • FFGs as a Unified Framework: FFGs offer a single, elegant graphical language to represent both the probabilistic generative model and the inference process for active inference, bridging the gap between model specification and computation.
  • Biologically Plausible Inference: The message-passing algorithms on FFGs provide a computationally efficient and, in principle, biologically plausible mechanism for how neuronal systems might implement free energy minimization.
  • Hierarchical Abstraction with FFGs: The "closing-the-box" technique in FFGs naturally supports the hierarchical organization of complex systems, allowing for the representation of multi-scale dynamical systems and deep inference processes.
  • Policy Selection as Free Energy Minimization: The paper clearly visualizes how expected free energy minimization, a core component of active inference for action selection, is nested as an inference subtask within the overall generative model.
  • Automated Inference: FFGs pave the way for automated inference tools, reducing the burden on researchers to manually derive complex update equations for novel active inference models.

Key Results

The paper does not present quantitative results from simulations on specific tasks. Instead, its key results are:

  • Demonstration of FFG for DTAI: The paper successfully constructs and illustrates FFGs for increasingly complex models, culminating in a detailed representation of a deep temporal active inference system.
  • Visualizing Inference Processes: The FFGs provide clear visual representations of how message passing algorithms execute inference for these complex models, making the computational mechanisms more transparent.
  • Conceptual Framework: The primary outcome is the establishment of FFGs as a robust and intuitive formalism for describing and simulating active inference, particularly for deep and hierarchical temporal models.

Limitations and Future Work

  • Computational Intractability: While FFGs facilitate automation, simulating very large or complex models can still be computationally demanding, especially for loopy graphs where exact inference is not guaranteed. The authors mention that iterative message passing is often used in practice for loopy graphs.
  • Toolbox Development: The authors acknowledge that a quality FFG simulation toolbox is not yet readily available and that their team is developing one, with a hope for release in 2018. This suggests that practical implementation and widespread adoption are still in progress.
  • Inference Scheduling: The paper touches upon the challenge of "inference scheduling" in hierarchical systems, noting that there is little theory about the optimal strategy for message passing between layers. This is identified as an unexplored area.
  • Learning: While the paper briefly mentions learning as updating priors, a detailed treatment of learning mechanisms within the FFG framework for active inference is not provided.
  • Future Work: The authors express hope that their work will generate interest in the neuroscience community to explore factor graphs further for complex generative neural models. They also anticipate the release of their FFG simulation toolbox.

Summary updated

BibTeX
@article{vries2017factor,
  author = {Bert de Vries and Karl J. Friston},
  title = {{A Factor Graph Description of Deep Temporal Active Inference}},
  journal = {Frontiers Comput. Neurosci.},
  year = {2017},
  doi = {10.3389/fncom.2017.00095},
}
Figure