Towards Learning Abductive Reasoning Using VSA Distributed Representations

Giacomo Camposampiero, Michael Hersche, Aleksandar Terzic, Roger Wattenhofer, Abu Sebastian, Abbas Rahimi

Abstract

Abstract reasoning, the ability to induce rules from limited experience and apply them to novel situations, remains a significant challenge in artificial intelligence. This paper introduces the Abductive Rule Learner with Context-awareness (ARLC), a neuro-symbolic model designed to solve Raven’s Progressive Matrices (RPM). By leveraging Vector Symbolic Architectures (VSA), ARLC achieves state-of-the-art performance on the I-RAVEN benchmark with only 480 parameters. The model demonstrates exceptional interpretability and zero-shot transfer capabilities, highlighting the efficiency of structured algebraic representations over massive-scale neural networks.

Introduction

The gold standard for testing abstract reasoning is Raven’s Progressive Matrices (RPM)—visual puzzles where a missing tile in a 3x3 grid must be identified based on underlying patterns. While Large Language Models (LLMs) exhibit some emergent reasoning, they are often brittle and require billions of parameters. Existing neuro-symbolic models frequently rely on hard-coded rules or struggle with multi-object generalization. ARLC addresses these limitations by learning rules from data within a compact, efficient framework.

The ARLC Architecture

ARLC builds upon Vector Symbolic Architecture (VSA) to map visual attributes like color, shape, and size into high-dimensional vectors. Unlike traditional neural embeddings, these vectors are manipulated through explicit algebraic operations.

The ARLC architecture overview, showing the projection of visual attributes into VSA space followed by rule computation and candidate selection.

The model employs a "context-aware" approach. Rather than tying rules to absolute grid positions, ARLC defines rules based on the relationship between a panel and its context (the other panels in its row or column). This allows the model to apply the same learned rule across different parts of the puzzle and even across different puzzle types.

Mathematical Foundations: VSA Operations

The core of ARLC's efficiency lies in its use of VSA operations to represent and manipulate logical rules. These operations allow for binding (associating values with keys) and bundling (merging information) in a way that is both computationally efficient and logically sound.

Table of VSA operations and their mathematical equivalents in binary GSBCs and real-valued spaces.

By expanding the mathematical template for rules from 6 to 12 terms, the researchers introduced "validation" terms. These terms enable the model to distinguish between competing hypotheses that might appear identical in a partial puzzle, such as distinguishing between "addition" and "shifting" when numerical values happen to align.

Results and Performance

Evaluated on the I-RAVEN benchmark, ARLC achieved an average accuracy of 92.6%, surpassing both the previous state-of-the-art neuro-symbolic model (NVSA, 88.1%) and GPT-3 (86.5%).

Key highlights include:

  • Extreme Parameter Efficiency: Achieving these results with only 480 trainable parameters.
  • Zero-Shot Transfer: The model, trained on 2x2 puzzles, successfully solved 3x3 and other complex constellations without additional training.
  • Interpretability: The algebraic nature of the rules allows researchers to inspect exactly which rule is being applied to any given attribute.

Conclusion

ARLC demonstrates that for structured logic tasks, massive scale is not a prerequisite for high performance. By combining neural perception with symbolic VSA representations, the model achieves superior accuracy, efficiency, and transparency. Future work will focus on extending these representations to capture object-level arithmetic and progression rules more effectively.

Summary updated

BibTeX
@inproceedings{camposampiero2024towards,
  author = {Giacomo Camposampiero and Michael Hersche and Aleksandar Terzic and Roger Wattenhofer and Abu Sebastian and Abbas Rahimi},
  title = {{Towards Learning Abductive Reasoning Using VSA Distributed Representations}},
  booktitle = {NeSy},
  year = {2024},
  doi = {10.1007/978-3-031-71167-1\_20},
}
Figure