VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming

Eleonora Misino, Giuseppe Marra, Emanuele Sansone

Introduction

Standard conditional generative models, such as conditional Variational Autoencoders (VAEs), tightly couple their latent representations to the specific labels provided during training. For instance, if a model is trained on images of two handwritten digits labeled with their mathematical sum, its latent space learns to represent "the sum" rather than the individual digits themselves. This creates a rigid system: if tasked with generating an image of two digits conditioned on their multiplication or subtraction, standard models fail. They must be completely retrained on a newly labeled dataset because they haven't learned the underlying symbolic structure.

To address this, researchers introduce VAEL, a novel neuro-symbolic generative model that bridges VAEs with Probabilistic Logic Programming. By using a logic program to decouple the latent representation from the specific training task, VAEL can perform entirely new generative tasks zero-shot, while also drastically improving data efficiency.

The VAEL Architecture

VAEL features an end-to-end differentiable architecture that factorizes the generative process into perceptual and logical components:

  1. The Split Latent Space: An encoder maps an input image $x$ to two distinct latent vectors: a subsymbolic vector $z$ (capturing continuous features like style and background) and a symbolic vector $z_{sym}$.
  2. The Neuro-Symbolic Bridge: Instead of mapping $z_{sym}$ directly to a label, a Multi-Layer Perceptron (MLP) maps $z_{sym}$ to the probabilities of logical facts within a ProbLog program.
  3. Probabilistic Logic Programming: A predefined ProbLog program uses these facts to deduce both the final label $y$ and a "possible world" $\omega_F$ (e.g., the actual discrete values of the two digits in the image).
  4. The Decoder: Finally, the decoder takes the subsymbolic style $z$ and the discrete logical world $\omega_F$ to reconstruct the image.

Unlike baselines like CCVAE that force a one-to-one mapping between the latent space and the label, VAEL uses the logic program as an intermediary. The neural network only learns to map raw pixels to basic symbols, while the logic program handles complex reasoning. The entire system is trained jointly using an adapted Evidence Lower Bound (ELBO) objective.

Datasets and Zero-Shot Generalization

The authors evaluated VAEL on two compositionally complex datasets:

  • 2digit MNIST: 64,400 images containing two MNIST digits, labeled during training with their mathematical sum.
  • 2level Mario: 6,720 images showing two consecutive states of a 3x3 grid world, labeled with the agent's move (up, down, left, right).
VAEL demonstrating zero-shot task generalization on the 2level Mario dataset.

A standout feature of VAEL is its Zero-Shot Task Generalization. Because the generative process is factorized, the logic program can be swapped out at test time. For example, after training VAEL on the Mario dataset to predict single moves, the underlying program can be replaced with a "shortest path" program. The model instantly generates correct sequential images conditioned on this new, complex logical rule without updating a single neural weight.

Furthermore, explicitly encoding the rules of the environment into the ProbLog program acts as a massive inductive bias. The neural network is relieved of learning these rules from scratch and only has to learn perception. This architecture also forces a natural disentanglement, separating "what" is in the image (logic branch) from "how" it looks (subsymbolic branch).

Key Results

VAEL demonstrates significant improvements over CCVAE, a state-of-the-art conditional VAE baseline:

  • Generative Accuracy: On 2digit MNIST, VAEL achieved 79.2% generative accuracy compared to CCVAE's 51.4%. On the 2level Mario dataset, VAEL achieved 81.3%, while CCVAE completely failed (0.0%) because it could not learn the complex mapping required to draw the agent, only managing to reconstruct the background.
  • Data Efficiency: Thanks to the logical prior, VAEL trained on just 10 images per digit pair outperformed CCVAE trained on 100 images per pair (VAEL generative accuracy ~0.41 vs CCVAE ~0.28).

Limitations and Future Work

While powerful, ProbLog inference can become computationally expensive as the number of probabilistic facts and rule complexity grow. To address this scalability issue, future work will investigate alternative semantics, such as Stochastic Logic Programs. The authors also plan to explore how VAEL can achieve out-of-distribution generalization and apply the framework to structured object generation, such as 3D scene composition.

Summary updated

Abstract

We present VAEL, a neuro-symbolic generative model integrating variational autoencoders (VAE) with the reasoning capabilities of probabilistic logic (L) programming. Besides standard latent subsymbolic variables, our model exploits a probabilistic logic program to define a further structured representation, which is used for logical reasoning. The entire process is end-to-end differentiable. Once trained, VAEL can solve new unseen generation tasks by (i) leveraging the previously acquired knowledge encoded in the neural component and (ii) exploiting new logical programs on the structured latent space. Our experiments provide support on the benefits of this neuro-symbolic integration both in terms of task generalization and data efficiency. To the best of our knowledge, this work is the first to propose a general-purpose end-to-end framework integrating probabilistic logic programming into a deep generative model.
BibTeX
@inproceedings{misino2023vael,
  author = {Eleonora Misino and Giuseppe Marra and Emanuele Sansone},
  title = {{VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming}},
  booktitle = {NeSy},
  year = {2023},
  url = {https://openreview.net/forum?id=ifMNmAFlR7},
}
Figure