From Statistical Relational to Neuro-Symbolic Artificial Intelligence

Giuseppe MarraAAAI 2024 · OpenReview

Abstract

The integration of learning and reasoning is one of the key challenges in artificial intelligence and machine learning today. The area of Neuro-Symbolic AI (NeSy) tackles this challenge by integrating symbolic reasoning with neural networks. In our recent work, we provided an introduction to NeSy by drawing several parallels to another field that has a rich tradition in integrating learning and reasoning, namely Statistical Relational Artificial Intelligence (StarAI).

Introduction

Modern Artificial Intelligence is currently characterized by a divide between two powerful paradigms: Deep Learning, which excels at pattern recognition and perception from raw data, and Symbolic Reasoning, which excels at high-level logic and structured knowledge. While Neuro-Symbolic AI (NeSy) has emerged as a promising field to fuse these approaches, it has often been criticized for appearing as a collection of disparate techniques—a "bag of tricks"—rather than a unified discipline.

This paper argues that the path forward for NeSy lies in its history. By examining the evolution of Statistical Relational Artificial Intelligence (StarAI)—a field dedicated to combining logic with probabilistic graphical models—we can identify a roadmap for categorizing and building modern NeSy systems. This work bridges the gap between these two communities by proposing a shared taxonomy and a unified methodology for system construction.

The Seven Dimensions of Neuro-Symbolic AI

To provide a common language for researchers, the paper identifies seven axes along which any NeSy or StarAI system can be categorized. These dimensions define how learning and reasoning are integrated:

  1. Logic Inference: Distinguishes between model-based approaches (where logic acts as a regularization loss) and proof-based approaches (where logic serves as a template for the neural network architecture).
  2. Logic Syntax: Ranges from simple propositional logic to complex relational or first-order logic, the latter of which allows for variables and reusable templates.
  3. Logic Semantics: Addresses how discrete logic is "softened" to allow for gradient-based optimization, often utilizing fuzzy logic or probabilistic frameworks.
  4. Learning: Focuses on whether the system learns only the parameters (weights) or if it can also induce the underlying logical rules (structure learning).
  5. Symbols vs. Sub-symbols: Determines if elements are treated as discrete atoms (symbols) or mapped to dense vector spaces (embeddings).
  6. Degree of Integration: Measures how much of the original logical rigor is preserved versus how much is sacrificed for computational efficiency.
  7. Tasks: Categorizes the end goal, such as knowledge graph completion, collective classification, or rule induction.

The NeSy "Recipe": From Logic to Graphs

A core contribution of this work is the distillation of NeSy system design into a repeatable two-step "recipe." This process allows researchers to transform high-level knowledge into a trainable deep learning model:

1. Logical Construction

The first step involves creating a "ground" theory of the problem, typically represented as an AND/OR tree. This tree captures the pure logical structure and dependencies of the domain, independent of the specific numerical values or weights.

2. Differentiable Compilation

The logical tree is then compiled into a differentiable computational graph. In this stage, logical operators (AND/OR) are replaced with continuous mathematical functions—such as t-norms in fuzzy logic—and the leaves of the tree are linked to neural network outputs. The result is a hybrid deep computational graph that can be trained using standard backpropagation while remaining "correct by design" according to the initial logical constraints.

Key Insights and Theoretical Trade-offs

The integration of logic and neural networks is not without cost. The paper highlights several fundamental tensions in the field:

  • The Purity-Tractability Trade-off: There is an inherent conflict between logical rigor and computational scalability. Systems that maintain perfect logical reasoning often face NP-hard complexity, while scalable systems usually rely on "relaxations" that may deviate from pure logical behavior.
  • Logic as Architecture: One of the most potent ideas in NeSy is using logic as a template. By allowing logical rules to define the structure of the neural network itself, researchers can ensure that the resulting model inherently respects domain knowledge.
  • Leveraging StarAI: Many challenges currently facing NeSy—such as dealing with uncertainty and relational data—have been studied for decades within StarAI. Recognizing NeSy as an evolution of StarAI allows the field to leverage existing theories on probabilistic graphical models.

Limitations and Future Directions

Despite the progress in categorization, several hurdles remain. The Relaxation Gap is a primary concern; the mathematical relaxations required for gradients (like fuzzy logic) often change the fundamental behavior of the logic, meaning the system may not truly "reason" in the classical sense. Furthermore, while parameter learning is well-understood, Structure Learning—the ability for a model to automatically discover logical rules from raw data—remains a difficult and relatively unexplored frontier in the NeSy context.

By providing this unified framework, the paper moves the community toward a more principled approach to hybrid AI, where the strengths of both symbolic reasoning and deep learning can be systematically combined.

Summary updated

Abstract

The integration of learning and reasoning is one of the key challenges in artificial intelligence and machine learning today. The area of Neuro-Symbolic AI (NeSy) tackles this challenge by integrating symbolic reasoning with neural networks. In our recent work, we provided an introduction to NeSy by drawing several parallels to another field that has a rich tradition in integrating learning and reasoning, namely Statistical Relational Artificial Intelligence (StarAI).
BibTeX
@inproceedings{marra2024from,
  author = {Giuseppe Marra},
  title = {{From Statistical Relational to Neuro-Symbolic Artificial Intelligence}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=vbm3cIBUYN},
}
Figure