Formalizing Consistency and Coherence of Representation Learning

Harald Strömfelt, Luke Dickens, Artur S. d'Avila Garcez, Alessandra RussoNeurIPS 2022 · OpenReview

Introduction

Humans possess a remarkable ability to learn an abstract concept in one context (e.g., "3 is greater than 2") and apply its underlying logic to a completely different domain (e.g., "this tower of blocks is taller than that one"). In artificial intelligence, this capability is known as concept transfer.

While symbolic AI systems can formally define and verify logical consistency, neural networks typically lack these strict definitions. Success in deep learning is often measured by accuracy on a specific dataset, which fails to indicate whether a model has truly "understood" the underlying rules or merely memorized statistical patterns. Without formal definitions of consistency (how well a model follows logical rules) and coherence (how well those rules hold up across different domains), evaluating and improving a neural network's ability to perform robust transfer learning remains challenging.

This paper bridges the gap between neural networks and symbolic logic by providing formal definitions for consistency and coherence in representation learning. Furthermore, the authors introduce a novel Dynamic Comparator model designed to maintain logical consistency across its entire latent space, demonstrating significantly better performance when transferring abstract concepts between domains.

Formalizing Consistency and Coherence

Drawing on Model Theory, the authors propose a framework to define these critical terms for sub-symbolic (neural) systems:

  • Consistency: The degree to which a model's predictions for relations (such as isGreater) satisfy a predefined set of logical axioms (e.g., transitivity: if $A > B$ and $B > C$, then $A > C$).
  • Coherence: The ability of a model to maintain this logical consistency when transferring representations from a source domain to a target domain.

The Neuro-symbolic Autoencoder

To operationalize these concepts, the authors develop a neuro-symbolic autoencoder architecture.

The neuro-symbolic autoencoder architecture, illustrating how images from domains like MNIST or BlockStacks are encoded into latent vectors for relation decoding.

The architecture consists of three primary components:

  1. Domain Encoders ($\psi^{enc}$): These map raw inputs, such as images, into a continuous latent space $Z$.
  2. Relation Decoders ($\phi$): Modular components that take pairs of latent representations and predict the logical relationship between them.
  3. Dynamic Comparator (DC): A novel relation-decoder module introduced in this paper. Unlike standard feed-forward layers, the DC utilizes a distance-based measure and a step function to model relationships. This structural prior encourages the model to learn a consistent logical structure across the entire latent space, rather than just in the immediate vicinity of the training data points.

Experimental Setup: Partial Relation Transfer

The models are evaluated using a Partial Relation Transfer (PRT) task across two distinct visual domains:

  • MNIST: The classic dataset of handwritten digits, serving as the source domain where the model learns the concept of ordinality (e.g., $5 > 3$).
  • BlockStacks: A novel, CLEVR-style dataset introduced in this paper featuring 3D renders of colored blocks stacked at varying heights.

The core challenge is to determine if a model that learns "greater than" using digits can transfer that logic to understand "higher than" in block stacks, especially when provided with only a minimal sliver of relational information (such as the isSuccessor relation) in the new target domain.

Key Results and Insights

The authors compared their Dynamic Comparator (DC) module against standard Neural Networks (NN) and specialized Knowledge Graph Embedding models like NTN, HolE, and TransR.

Performance comparison showing the Dynamic Comparator (DC) model maintaining high accuracy in the target domain and demonstrating robustness across varying disentanglement pressures.
  • Consistency Enables Transfer: A critical insight from the study is that high accuracy in the source domain is insufficient for successful transfer. Models that maintain logical consistency even in "empty" or unobserved regions of the latent space are significantly better at transferring concepts. By measuring consistency on interpolated points in the source domain, researchers can effectively anticipate transfer performance.
  • Zero-Shot Transfer: In the BlockStacks domain, the DC model achieved near-perfect accuracy (~98%+) on held-out relations like isGreater and isLess, despite only being trained on the logic of handwritten digits.
  • Robustness to Disentanglement: The DC model's performance remained highly stable regardless of the disentanglement pressure ($\beta$) applied during training. In contrast, baseline models experienced significant performance degradation under similar conditions.
  • Superior Coherence: The DC model achieved the lowest consistency and coherence losses across all tests, proving that its internal representation of ordinality is fundamentally more logically sound than competing architectures.

Limitations and Future Work

While the Dynamic Comparator marks a significant step forward for neuro-symbolic transfer learning, the authors note several avenues for future research. The current framework primarily focuses on binary relations (relationships between two objects); extending this to n-ary relations is a logical next step. Additionally, the model currently learns a fixed set of relations simultaneously. Future iterations could explore curriculum learning, allowing the model to discover and refine concepts gradually. Finally, while the proxy loss function is effective for the tested theories, scaling it to extremely complex logical theories may present computational challenges that require further optimization.

Summary updated

Abstract

In the study of reasoning in neural networks, recent efforts have sought to improve consistency and coherence of sequence models, leading to important developments in the area of neuro-symbolic AI. In symbolic AI, the concepts of consistency and coherence can be defined and verified formally, but for neural networks these definitions are lacking. The provision of such formal definitions is crucial to offer a common basis for the quantitative evaluation and systematic comparison of connectionist, neuro-symbolic and transfer learning approaches. In this paper, we introduce formal definitions of consistency and coherence for neural systems. To illustrate the usefulness of our definitions, we propose a new dynamic relation-decoder model built around the principles of consistency and coherence. We compare our results with several existing relation-decoders using a partial transfer learning task based on a novel data set introduced in this paper. Our experiments show that relation-decoders that maintain consistency over unobserved regions of representation space retaincoherence across domains, whilst achieving better transfer learning performance.
BibTeX
@inproceedings{stromfelt2022formalizing,
  author = {Harald Strömfelt and Luke Dickens and Artur S. d'Avila Garcez and Alessandra Russo},
  title = {{Formalizing Consistency and Coherence of Representation Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=Hh3cklC5Qf},
}
Figure