Deep learning has mastered the art of identifying what is present within an image or text. Modern convolutional networks and vision-language models can easily segment a circuit component, detect a person in a crowd, or label an isolated object. However, as benchmarks like EEE-Bench: A Comprehensive Multimodal Electrical And Electronics Engineering Benchmark demonstrate, these same models frequently fail when asked to reason about how those components interact or depend on one another.
This failure stems from a fundamental architectural limitation: standard deep learning models treat scenes as collections of isolated entities or dense, unstructured feature maps. They lack an explicit representation of the "connective tissue"—the spatial, logical, and functional relationships that define a system's global state. To bridge this relational reasoning gap, researchers are designing architectures that treat relations as first-class citizens.
Why Standard Architectures Fail to Reason
To understand why standard models struggle, consider how a typical convolutional neural network (CNN) or vision-language model processes an image. The network extracts localized features (textures, shapes, categories) and aggregates them via global pooling or dense layers.
As John Mern demonstrates in Exchangeable Input Representations for Reinforcement Learning, representing objects through independent feature projections and global pooling fails to capture relational dependencies. When a model pools features, the precise spatial and functional links between discrete entities are washed away. Monika Wysoczanska et al. confirmed this empirically in Toward Unsupervised Visual Reasoning: Do Off-the-Shelf Features Know How to Reason?, showing that even self-supervised visual features that match human performance on classification tasks fail to encode the spatial and relational data required for multi-step logical tasks.
Without a dedicated structural mechanism, a model cannot distinguish between "the cat is on the mat" and "the mat is on the cat"—it merely registers the presence of both entities.
Baking in Relational Priors via Permutation Invariance
How do we force a neural network to learn relationships instead of just accumulating attributes? The key intuition is to bake relational inductive biases directly into the network's architecture.
In Discovering objects and their relations from entangled scene representations, Adam Santoro et al. introduced Relation Networks (RNs). The core mechanism of an RN is deceptively simple: instead of processing the entire scene representation at once, the network explicitly computes a function over all pairs of objects:
$$O_{rel} = f_\phi \left( \sum_{i,j} g_\theta(o_i, o_j) \right)$$
Here, $g_\theta$ is a neural network that evaluates the relationship between object $o_i$ and object $o_j$, and $f_\phi$ aggregates these pairwise insights. Because the summation is symmetric, the architecture is inherently permutation invariant—the order in which objects are listed does not alter the scene's meaning. This structural constraint forces the network to focus entirely on how distinct entities interact.
Similarly, Peter Battaglia et al. proposed Interaction Networks in Interaction Networks for Learning about Objects, Relations and Physics. By explicitly decomposing a physical system into distinct objects and specific relations, their framework acts as a learnable physics engine, predicting complex multi-body dynamics by evaluating the causal effects of interactions step-by-step.
From Pixels to Propositions: Neuro-Symbolic Reasoning
While pairwise networks capture statistical interactions, abstract reasoning often requires identifying underlying logical structures that generalize across entirely different object distributions. For instance, the concept of "symmetry" or "functional dependency" remains identical whether applied to simple geometric shapes or complex engineering diagrams.
To achieve this level of abstraction, researchers have turned to neuro-symbolic architectures that map continuous perceptual features to discrete, logic-like propositions. In An Explicitly Relational Neural Network Architecture, Murray Shanahan et al. introduced PrediNet. PrediNet uses a multi-head attention mechanism to extract pairs of objects and explicitly output relations as soft propositional formulas, enabling the model to generalize abstract spatial rules to entirely unseen distributions.
Taking this further, Hikaru Shindo et al. developed αILP: thinking visual scenes as differentiable logic programs, which combines object-centric perception with a differentiable inductive logic programming engine. By representing visual scenes as differentiable logic programs, the model can learn human-readable, symbolic rules directly from raw images using gradient descent. This marriage of perception and explicit forward logic—also explored by Devendra Singh Dhami et al. in Neuro-Symbolic Forward Reasoning—ensures that the model reasons about dependencies rather than relying on black-box feature matching.
Graph-Structured Context and Explainable Interactions
When scenes scale in complexity, evaluating every possible pair of objects becomes computationally intractable and contextually noisy. Modern approaches handle this by structuring scenes as dynamic graphs, where nodes represent entities and edges represent their relationships.
For example, in group activity recognition, Zhiwei Deng et al. implemented a differentiable message-passing neural network in Deep Structured Models For Group Activity Recognition. This allows individual action predictions to refine, and be refined by, the global context of the group's collective behavior. In the domain of Visual Question Answering, Qingbao Huang et al. developed a Dual Channel Graph Convolutional Network in Aligned Dual Channel Graph Convolutional Network for Visual Question Answering to align the spatial graph of an image with the syntactic dependency graph of a question.
Crucially, shifting to a relational paradigm also changes how models explain their decisions. Traditional interpretable models, like Concept Bottleneck Models, assume that concepts are properties of single, isolated entities (e.g., "is red"). To fix this, Pietro Barbiero et al. introduced Relational Concept Bottleneck Models. By defining the bottleneck layer over emergent properties of relationships between entities (e.g., "Component A is connected in parallel to Component B"), they allow deep networks to provide human-readable explanations for tasks where the core logic depends entirely on object interactions.