TL;DR
This paper introduces a novel subsymbolic approach to solving Raven's Progressive Matrices (RPM) by framing it as a similarity-based reasoning problem. Instead of complex structure mapping, their model uses a convolutional neural network to extract basic and relational features, then computes similarity using relative entropy, achieving performance comparable to existing symbolic models.
What Problem Does This Paper Address?
The paper tackles the challenge of achieving "generality" in artificial intelligence – the ability of a system to handle a wide variety of tasks and environments. Traditionally, this has been approached through symbolic AI, often emphasizing analogical reasoning. The authors focus on Raven's Progressive Matrices (RPM) as a benchmark for this general intelligence, as these tests are designed to measure eductive ability, which involves generalizing knowledge and dealing with novel situations. Existing AI approaches to RPM often rely on complex symbolic rule induction or structure mapping for analogical reasoning. This paper proposes a subsymbolic, similarity-based approach that bypasses the need for explicit structure mapping, aiming for a more flexible and potentially more generalizable solution.
Methods
The core of the proposed method is a subsymbolic, similarity-based approach to analogical reasoning. Instead of formal structure mapping between source and target representations, the model relies on:
- Feature Extraction: A convolutional neural network (CNN) is used to process visual figures and extract subjective probabilities for a set of predefined features (e.g., shape distribution, size increments, orientation). These features can be basic or relational.
- Similarity Computation: The similarity between alternative figures and the matrix sequences is measured using relative entropy (also known as cross-entropy). This metric quantifies how well the probability distribution of features in an alternative sequence matches the distribution of features in the established matrix sequences (rows and columns).
- Selection Procedure: The model uses a Boltzmann distribution to stochastically select an answer based on the computed similarity scores. Alternatives that minimize the relative entropy (i.e., are more similar) are favored.
This approach contrasts with traditional symbolic methods that often require explicit representation alignment.
Datasets
The model was trained and evaluated on a dataset of matrix problems generated using the Sandia Matrix Generation Tool [Matzen et al., 2010]. These generated matrices are designed to mimic the structure of original Raven's matrices and have demonstrated comparable psychometric characteristics. The authors used 1480 input-output pairs (from 74 matrices, 20 examples each) to train the CNN.
Key Insights and Ideas
- Similarity as a Foundation for Analogy: The paper argues that analogical reasoning can be viewed as a special case of similarity-based reasoning, allowing for subsymbolic processing.
- Feature-Based vs. Structure Mapping: By focusing on feature-based processing (both relational and non-relational), the model bypasses the computationally intensive and often brittle structure mapping required by many symbolic analogical reasoning systems.
- Relative Entropy as a Similarity Measure: The use of relative entropy for similarity computation is highlighted for its interesting formal properties, some of which align with observations of human similarity judgments. It also connects to principles of inductive inference and Occam's razor.
- Convolutional Networks for Feature Learning: The CNN plays a crucial role in learning relevant features from visual input, demonstrating the power of subsymbolic methods for perceptual tasks relevant to general intelligence.
- Generality Through Feature Learning: The authors suggest that learning and adaptive detection of features are critical aspects of achieving generality in AI.
Key Results
- The model achieved a maximum performance of 85.01% on the Sandia matrices.
- This performance is comparable to existing symbolic analogy-based models.
- Analysis of the CNN's hidden layers revealed that the convolutional architecture significantly contributes to the model's feature detection ability. Hierarchical clustering of activations showed strong agreement between clusters of matrix sequences and answer sequences, and clusters of distractor sequences.
Limitations and Future Work
- Performance Limitation: The authors acknowledge that the model's current performance is a limitation and plan to address this through continued development.
- Feature Set Dependencies: Further analysis suggests that some dependencies within the feature set contribute to errors. An enlarged, more independent feature set is expected to improve performance.
- Perceptual Errors: Some errors are also attributed to perceptual issues within the neural network.
- Testing on Original RPMs: The authors plan to test the model on original Raven's Progressive Matrices items.
- Exploring Other Analogical Problems: The work suggests exploring the model's potential for explaining other forms of analogy.
- Feature Learning Integration: Future work could integrate feature learning more deeply with the similarity-based reasoning framework.
- Broader Computational Theory: The authors propose that their findings could contribute to a broader, integrative computational theory of general intelligence, linking inductive inference, simplicity, similarity, and analogy.
Summary updated