Introduction
While deep learning has achieved remarkable success in perceptual tasks such as image classification and object detection, it continues to struggle with abstract reasoning—the ability to identify underlying logical rules and apply them to novel scenarios. A primary benchmark for evaluating this capability is Raven’s Progressive Matrices (RPM). These puzzles consist of a 3x3 grid of geometric patterns where the final cell is missing. To solve them, an agent must deduce the latent logical rules (e.g., rotation, arithmetic operations on shapes, or progression) governing the rows and columns.
Traditional AI models typically employ single-stream architectures, using either Convolutional Neural Networks (CNNs) for feature extraction or Vision Transformers (ViTs) for global context. However, these often fail to simultaneously capture fine-grained object details and complex spatial relationships. This paper introduces the Dual-stream Reasoning Network (DRNet), a bio-inspired approach that mimics the human brain's dual-pathway visual processing system to overcome these limitations.
Methodology
DRNet is grounded in the "two-stream hypothesis" from neuroscience, which posits that the brain processes visual information through a ventral stream (the "what" pathway for object identity) and a dorsal stream (the "where" pathway for spatial location).
Model Architecture
As illustrated in the architecture diagram below, DRNet consists of four primary components:
- Dual Encoder Module: This module processes input images through two parallel branches. The CNN branch utilizes ResBlocks to extract local, high-level features (identifying shapes and attributes), while the ViT branch employs a Vision Transformer to attend to global spatial context and relative positioning.
- Integration Module: The features from both streams are fused. While several methods were evaluated, a simple Linear (LIN) projection—concatenating the features and projecting them into a unified space—yielded the highest performance.
- Rule Extractor: This component serves as the reasoning engine. It processes the fused features of the eight context images alongside each of the eight candidate answers. Using 1D convolutions, it infers relational patterns across the grid.
- Classifier: A final Multi-Layer Perceptron (MLP) assigns a score to each candidate, selecting the one that most closely aligns with the deduced logic.
Experimental Results
DRNet was evaluated on the most rigorous RPM benchmarks, including Procedurally Generated Matrices (PGM) and various RAVEN-style datasets. These benchmarks test both standard performance and Out-of-Distribution (OOD) generalization, where the model must apply learned rules to unseen attribute combinations.
Performance Benchmarks
DRNet established a new state-of-the-art (SOTA) across multiple metrics:
- PGM Neutral: Achieved an accuracy of 99.06%.
- Generalization: On the PGM "Held-Out Attribute Pairs" OOD test, DRNet reached 93.74% accuracy, outperforming earlier specialized models by nearly 30%.
- Average Accuracy: The model maintained a consistent average of 97.79% across diverse benchmarks, surpassing previous leaders like PredRNet.
Discussion and Insights
The Complementary Nature of Dual Streams
Ablation studies confirmed that the CNN and ViT branches are not redundant but complementary. The CNN effectively captures the identity of geometric primitives, while the ViT excels at capturing the logic of their placement. Removing either branch resulted in a significant performance decline, particularly in challenging OOD scenarios.
Emergent Rule Discovery
One of the most significant findings was the model's ability to develop internal representations of logical rules without explicit supervision. Through t-SNE visualization of the model's latent space, researchers observed that DRNet naturally clusters problems by their underlying logic (e.g., XOR, OR, Progression), as shown in the figure below. This suggests that the dual-stream approach allows the model to "discover" abstract concepts autonomously.
Conclusion
DRNet demonstrates that bio-inspired dual-stream architectures can significantly enhance the abstract reasoning capabilities of neural networks. By effectively balancing "what" and "where" information, the model achieves superior generalization and SOTA performance on complex visual puzzles.
Future work will address current limitations, such as lower recognition rates on highly complex sub-grid configurations, by exploring adaptive kernel and patch sizes. Additionally, the authors propose integrating multimodal components to investigate whether "verbalizing" logical rules can further improve reasoning robustness, moving closer to human-level cognitive flexibility.
Summary updated