Deep Learning with Relational Logic Representations

Gustav SourekIJCAI 2019 · DOI

TL;DR

This paper introduces Lifted Relational Neural Networks (LRNNs), a framework that bridges the gap between deep learning's pattern recognition power and symbolic AI's reasoning capabilities. By leveraging relational logic templates, LRNNs dynamically construct neural network architectures tailored to specific data structures, enabling more expressive and interpretable learning for complex, relational data.

What Problem Does This Paper Address?

Deep learning has achieved remarkable success, but current architectures often struggle with dynamically sized and structured data, such as those found in knowledge graphs or biological networks. These limitations stem from their reliance on static computational graphs and fixed tensor representations. Furthermore, integrating the vast amount of existing symbolic knowledge into neural networks remains a significant challenge. This paper addresses the need for neural networks that can effectively handle relational structures and seamlessly incorporate symbolic reasoning, moving beyond simple pattern matching to more complex generalization and representation learning.

Methods

The core contribution is the framework of Lifted Relational Neural Networks (LRNNs). This approach follows the "lifted modeling" strategy, where instead of defining a fixed model architecture, the authors propose a template from which specific architectures are derived during the learning process. This template is based on weighted relational (Horn) clause logic, extended with differentiable activations to translate logical rules into neural network components. The translation into neural architectures is semantically grounded in the least Herbrand models of the template and example structures, with calculations performed either top-down or bottom-up depending on the domain structure. This allows the models to respect data symmetries and learn compressed representations by parameterizing isomorphic patterns jointly.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Dynamic Architecture Generation: The key innovation is the ability to dynamically construct neural network architectures based on a relational logic template and the input data. This contrasts with traditional fixed neural network designs.
  • Bridging Deep Learning and Symbolic AI: LRNNs offer a principled way to combine the strengths of deep learning (pattern recognition) with symbolic AI (reasoning, knowledge representation).
  • Expressive Representation Learning: By using relational logic, LRNNs can learn more expressive representations, including embeddings of entire latent relational structures, not just individual constants.
  • Handling Data Symmetries: The lifted modeling approach inherently respects symmetries in the data, leading to more compressed and efficient representations. This is likened to the principle behind convolutional filters.
  • Generalization of Existing Approaches: The framework is presented as a general approach that can subsume and unify previous neural-symbolic methods, including Convolutional Graph Networks and Neural Theorem Provers, through template declaration.

Key Results

The paper highlights that their framework, LRNNs, has demonstrated state-of-the-art (SotA) results on classic Statistical Relational Learning (SRL) benchmarks using relatively simple templates. They also claim that LRNNs are unique in supporting proper symbolic structure learning with neural predicate invention, a capability previously limited to inductive logic settings and difficult to achieve with standard optimization.

Limitations and Future Work

The authors acknowledge that future enhancements are needed for targeted optimization to scale LRNNs to more realistic Natural Language Processing (NLP) and Knowledge Base Completion (KBC) applications. They expect that the integration with symbolic knowledge abstractions will be crucial for future progress in neural learning within these domains.

Summary updated

BibTeX
@inproceedings{sourek2019deep,
  author = {Gustav Sourek},
  title = {{Deep Learning with Relational Logic Representations}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/920},
}
Figure