Introduction
Standard conditional generative models, such as conditional Variational Autoencoders (VAEs), tightly couple their latent representations to the specific labels provided during training. For instance, if a model is trained on images of two handwritten digits labeled with their mathematical sum, its latent space learns to represent "the sum" rather than the individual digits themselves. This creates a rigid system: if tasked with generating an image of two digits conditioned on their multiplication or subtraction, standard models fail. They must be completely retrained on a newly labeled dataset because they haven't learned the underlying symbolic structure.
To address this, researchers introduce VAEL, a novel neuro-symbolic generative model that bridges VAEs with Probabilistic Logic Programming. By using a logic program to decouple the latent representation from the specific training task, VAEL can perform entirely new generative tasks zero-shot, while also drastically improving data efficiency.
The VAEL Architecture
VAEL features an end-to-end differentiable architecture that factorizes the generative process into perceptual and logical components:
- The Split Latent Space: An encoder maps an input image $x$ to two distinct latent vectors: a subsymbolic vector $z$ (capturing continuous features like style and background) and a symbolic vector $z_{sym}$.
- The Neuro-Symbolic Bridge: Instead of mapping $z_{sym}$ directly to a label, a Multi-Layer Perceptron (MLP) maps $z_{sym}$ to the probabilities of logical facts within a ProbLog program.
- Probabilistic Logic Programming: A predefined ProbLog program uses these facts to deduce both the final label $y$ and a "possible world" $\omega_F$ (e.g., the actual discrete values of the two digits in the image).
- The Decoder: Finally, the decoder takes the subsymbolic style $z$ and the discrete logical world $\omega_F$ to reconstruct the image.
Unlike baselines like CCVAE that force a one-to-one mapping between the latent space and the label, VAEL uses the logic program as an intermediary. The neural network only learns to map raw pixels to basic symbols, while the logic program handles complex reasoning. The entire system is trained jointly using an adapted Evidence Lower Bound (ELBO) objective.
Datasets and Zero-Shot Generalization
The authors evaluated VAEL on two compositionally complex datasets:
- 2digit MNIST: 64,400 images containing two MNIST digits, labeled during training with their mathematical sum.
- 2level Mario: 6,720 images showing two consecutive states of a 3x3 grid world, labeled with the agent's move (up, down, left, right).
A standout feature of VAEL is its Zero-Shot Task Generalization. Because the generative process is factorized, the logic program can be swapped out at test time. For example, after training VAEL on the Mario dataset to predict single moves, the underlying program can be replaced with a "shortest path" program. The model instantly generates correct sequential images conditioned on this new, complex logical rule without updating a single neural weight.
Furthermore, explicitly encoding the rules of the environment into the ProbLog program acts as a massive inductive bias. The neural network is relieved of learning these rules from scratch and only has to learn perception. This architecture also forces a natural disentanglement, separating "what" is in the image (logic branch) from "how" it looks (subsymbolic branch).
Key Results
VAEL demonstrates significant improvements over CCVAE, a state-of-the-art conditional VAE baseline:
- Generative Accuracy: On 2digit MNIST, VAEL achieved 79.2% generative accuracy compared to CCVAE's 51.4%. On the 2level Mario dataset, VAEL achieved 81.3%, while CCVAE completely failed (0.0%) because it could not learn the complex mapping required to draw the agent, only managing to reconstruct the background.
- Data Efficiency: Thanks to the logical prior, VAEL trained on just 10 images per digit pair outperformed CCVAE trained on 100 images per pair (VAEL generative accuracy ~0.41 vs CCVAE ~0.28).
Limitations and Future Work
While powerful, ProbLog inference can become computationally expensive as the number of probabilistic facts and rule complexity grow. To address this scalability issue, future work will investigate alternative semantics, such as Stochastic Logic Programs. The authors also plan to explore how VAEL can achieve out-of-distribution generalization and apply the framework to structured object generation, such as 3D scene composition.
Summary updated