Abstract
In this paper, we introduce Neural Probabilistic Soft Logic (NeuPSL), a novel neuro-symbolic (NeSy) framework that unites state-of-the-art symbolic reasoning with the low-level perception of deep neural networks. To model the boundary between neural and symbolic representations, we propose a family of energy-based models, NeSy Energy-Based Models, and show that they are general enough to include NeuPSL and many other NeSy approaches. Using this framework, we show how to seamlessly integrate neural and symbolic parameter learning and inference in NeuPSL. Through an extensive empirical evaluation, we demonstrate the benefits of using NeSy methods, achieving upwards of 30% improvement over independent neural network models. On a well-established NeSy task, MNIST-Addition, NeuPSL demonstrates its joint reasoning capabilities by outperforming existing NeSy approaches by up to 10% in low-data settings. Furthermore, NeuPSL achieves a 5% boost in performance over state-of-the-art NeSy methods in a canonical citation network task with up to a 40 times speed up.
Introduction
Artificial Intelligence has historically been divided between neural methods, which excel at pattern recognition in raw data but struggle with logic, and symbolic methods, which reason effectively using domain knowledge but fail to process sensory input. Neuro-Symbolic (NeSy) AI aims to bridge this gap. However, a primary bottleneck remains joint inference: the ability to reason across multiple related examples simultaneously. As the complexity of relational data grows, the solution space expands exponentially, causing existing NeSy methods to struggle with scalability or data efficiency.
Methods: The NeSy-EBM Framework and NeuPSL
To address these challenges, we propose the Neuro-Symbolic Energy-Based Model (NeSy-EBM) framework. This mathematical abstraction provides a unified language for NeSy systems by defining "energy functions" where lower energy states represent higher compatibility between raw data and logical constraints.
NeuPSL is our specific implementation of this framework. It extends Probabilistic Soft Logic (PSL) by incorporating "neural predicates"—logical atoms whose truth values are predicted by deep neural networks.
Key Technical Innovations
- Convex Relaxation: Unlike discrete logic systems (0 or 1), NeuPSL utilizes "soft" logic (continuous values between 0 and 1). This transforms hard combinatorial problems into continuous, convex optimization tasks solvable via the Alternating Direction Method of Multipliers (ADMM).
- End-to-End Learning: We derive a method for backpropagating gradients from the symbolic logic layer through to the neural network, enabling joint end-to-end training.
- Regularization: To prevent "degenerate" solutions where the model ignores logic to minimize loss, we introduce a negative log regularizer on symbolic weights.
Evaluation and Datasets
We evaluated NeuPSL across three distinct task categories:
- MNIST-Addition (with Overlap): Testing joint reasoning by reusing images across multiple equations to "triangulate" labels in low-data scenarios.
- Visual Sudoku Classification: Identifying valid 4x4 MNIST grids, requiring the model to satisfy complex global constraints (e.g., no duplicates in rows/columns).
- Citation Networks (Citeseer and Cora): Standard benchmarks for node classification in large relational graphs.
Results and Discussion
NeuPSL demonstrates significant advantages in both accuracy and computational efficiency. In low-data settings on MNIST-Addition, NeuPSL outperformed independent neural models by over 30% and surpassed existing NeSy methods by up to 10%. This highlights the "data multiplier" effect of symbolic logic: constraints allow the model to narrow down identities much faster than isolated perception.
In citation network benchmarks, NeuPSL achieved a 5% performance boost over state-of-the-art NeSy methods. Most notably, it achieved this with a massive reduction in training time, running up to 40 times faster than competitors like DeepStochLog.
Conclusion
NeuPSL successfully integrates the perceptual power of deep learning with the structured reasoning of PSL. By relaxing logic to continuous values, it avoids the #P-complete complexity of exact probabilistic inference, enabling it to scale to real-world relational tasks. While the symbolic optimization step adds a slight overhead compared to pure neural networks, the gains in accuracy and data efficiency represent a significant step forward for Neuro-Symbolic AI. Future work will focus on batched symbolic inference to further enhance performance on massive datasets.
Summary updated