Learning to Make Analogies by Contrasting Abstract Relational Structure

Felix Hill, Adam Santoro, David G. T. Barrett, Ari S. Morcos, Timothy P. Lillicrap

Analogical reasoning—the ability to map abstract relations from one domain of experience to another—is a cornerstone of human intelligence. It allows us to understand acoustics by comparing sound to water waves, or to solve a puzzle by recognizing that an "increase in size" in one set of objects should map to an "increase in quantity" in another.

For artificial intelligence, this capacity has historically been notoriously difficult to achieve. Symbolic models are often too rigid to handle diverse, noisy domains, while modern neural networks tend to be "lazy," relying on surface-level perceptual shortcuts rather than true relational reasoning.

In a recent study, researchers from DeepMind demonstrated that neural networks can indeed master complex analogical reasoning without needing highly specialized architectures. The secret lies not in the model's design, but in a novel training method called Learning Analogies by Contrasting (LABC).

The "Source-Blind" Trap

To understand why neural networks struggle with analogies, we must look at how they are typically trained. In a standard visual analogy multiple-choice setup, a model is given a source sequence (e.g., shapes increasing in number) and must complete a target sequence (e.g., lines increasing in darkness).

Usually, the incorrect multiple-choice options (distractors) are generated randomly. The researchers discovered a critical flaw in this standard approach: models can achieve high accuracy without ever looking at the source problem. Because the distractors are random, the network simply learns to pick the only candidate that looks like a "valid" sequence in the target domain. It effectively "cheats" the reasoning task by relying on a back-door pattern-matching solution.

The Solution: Learning Analogies by Contrasting (LABC)

To force models to learn true abstract relations, the authors introduced LABC. Under this training regime, the incorrect options are designed to be semantically plausible.

A visual analogy target sequence requiring the model to identify the progression of line darkness.

This means every distractor follows some valid rule within the target domain. For example, if the correct answer completes a sequence of lines getting darker, one distractor might show lines staying the same color, while another shows them decreasing in darkness.

Because all options are locally valid sequences, the model can no longer rely on the "source-blind" trap. To pick the right answer, it must look at the source domain, identify the specific abstract relation (such as Progression, XOR, or AND), and map that relation across to the target domain.

Experimental Setup

The researchers tested LABC across several architectures, ranging from a standard CNN-RNN pipeline and ResNet-50 to Relation Networks (WReN), which are specifically designed to compare pairs of objects.

They utilized two primary benchmarks:

  1. Visual Analogy Problems: A synthetic dataset of greyscale panels involving seven domains (shape size, color, quantity, position, etc.) and four logical relations (XOR, OR, AND, Progression).
  2. Symbolic Analogy Problems: A controlled environment using multi-dimensional vectors of integer features, where relations are defined by mathematical functions like MIN, MAX, SUM, and RANGE.

Key Findings: Data Over Architecture

The most surprising finding of the study is that the training regime is far more important than the model's architectural complexity. Even simple architectures outperformed sophisticated ones if they were trained using LABC.

  • Novel Domain Transfer: The true test of analogical reasoning is applying a known relation to a completely new domain mapping (e.g., seeing a relation move from "shape size" to "line color" for the first time). LABC-trained models achieved 83-90% accuracy on these novel transfers, compared to a mere 58% for standard training. When combined with the Parallel Relation Network (WReN), LABC reached an impressive 95% accuracy.
  • Emergent Relational Clusters: When analyzing the hidden states of LABC-trained models, researchers found that the networks naturally began to cluster data by relation type (e.g., grouping all "XOR" problems together) rather than by domain (e.g., grouping all "shape" problems together). This strongly suggests the models developed a truly abstract representation of logic.
  • Symbolic Success: On the symbolic task, standard training performed at chance levels (25%). Implementing LABC pushed performance to 89%.
  • Extrapolation: LABC also improved the models' ability to handle values they had never seen before, such as darker lines or higher quantities of shapes than were present in the training set.

Automating the Process

While LABC is highly effective, creating "semantically plausible" distractors traditionally requires some knowledge of the underlying data-generating rules. To address this, the authors explored an adversarial approach (LABC-adversarial). By using a "generator" network to create maximally confusing distractors—similar to how a Generative Adversarial Network (GAN) operates—they could induce analogical reasoning automatically. While this automated approach lagged behind hand-crafted LABC (62% vs 89% accuracy), it proves that the contrasting principle can be learned without explicit human labeling.

Conclusion

This research demonstrates that the critical factor for inducing analogical reasoning in AI is careful attention to the choice of data and how it is presented. By forcing models to contrast abstract relational structures, we can move neural networks beyond simple pattern matching and closer to the flexible, domain-general reasoning characteristic of human intelligence. Future work may explore applying these contrasting principles to more naturalistic, unstructured data like video or complex language tasks.

Summary updated

Abstract

Analogical reasoning has been a principal focus of various waves of AI research. Analogy is particularly challenging for machines because it requires relational structures to be represented such that they can be flexibly applied across diverse domains of experience. Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data. We find that the critical factor for inducing such a capacity is not an elaborate architecture, but rather, careful attention to the choice of data and the manner in which it is presented to the model. The most robust capacity for analogical reasoning is induced when networks learn analogies by contrasting abstract relational structures in their input domains, a training method that uses only the input data to force models to learn about important abstract features. Using this technique we demonstrate capacities for complex, visual and symbolic analogy making and generalisation in even the simplest neural network architectures.
BibTeX
@inproceedings{hill2019learning,
  author = {Felix Hill and Adam Santoro and David G. T. Barrett and Ari S. Morcos and Timothy P. Lillicrap},
  title = {{Learning to Make Analogies by Contrasting Abstract Relational Structure}},
  booktitle = {International Conference on Learning Representations},
  year = {2019},
  url = {https://openreview.net/forum?id=MExQXWTKUr2},
}
Figure