Few-shot Visual Reasoning with Meta-analogical Contrastive Learning

Youngsung Kim, Jinwoo Shin, Eunho Yang, Sung Ju Hwang

TL;DR

This paper introduces a novel "meta-analogical contrastive learning" approach to tackle few-shot visual reasoning. By leveraging human-like analogical reasoning to identify structural similarities between problem pairs, their method significantly outperforms state-of-the-art models, especially when training data is scarce, and demonstrates strong generalization to unseen attributes.

What Problem Does This Paper Address?

Deep learning models for visual reasoning, like those used for Raven's Progressive Matrices (RPM), typically require massive datasets to achieve good performance. This is in stark contrast to humans, who can often solve these logical puzzles with just a few examples. The core challenge is bridging this gap in sample efficiency and enabling models to generalize better from limited data. The authors hypothesize that this sample efficiency in humans stems from their ability to perform analogical reasoning – identifying underlying relational structures and applying them across different contexts. Existing deep learning models lack this capability, leading to their data-hungry nature.

Methods

The proposed method, Meta-analogical Contrastive Learning, centers around two key ideas:

  1. Analogical Learning: The core innovation is to treat visual reasoning problems as pairs of "support" and "query" sets. By creating slightly perturbed versions of the original problem (e.g., replacing a panel with noise), the model learns to identify and enforce the relational similarity between the original and perturbed sets. This is achieved by making their learned representations as similar as possible using contrastive learning. This process is repeated with various perturbations, allowing the model to learn robust relational embeddings even from a single pair of examples.
  2. Meta-analogical Contrastive Learning: To further improve generalization, especially to unseen attributes or problem configurations, the authors extend this by applying contrastive learning across different problem domains (e.g., different visual attributes or configurations within the RAVEN dataset). This meta-learning approach trains the model to learn a general analogical reasoning strategy that can adapt to new, unseen problem types.

The framework involves a contextual episode module to generate these support-query pairs and an analogy learning module to minimize the gap between their relational characteristics. They explore three types of analogy kernels: task inference scores, variational context latents, and generative contextual images.

Datasets

The primary dataset used is the RAVEN dataset. This dataset is designed for visual reasoning tasks and consists of problems with varying rules and attributes (shape, size, color). The authors use it for both training and evaluating their model, particularly focusing on scenarios with limited training data and cross-domain generalization.

Key Insights and Ideas

  • Analogy as a Key to Sample Efficiency: The paper strongly suggests that mimicking human analogical reasoning is crucial for achieving sample-efficient visual reasoning. By focusing on relational similarity rather than just raw pattern matching, models can learn more abstract and generalizable representations.
  • Perturbation-based Analogies: Creating "noisy" or perturbed versions of a problem and forcing the model to see the same underlying relation is a clever way to generate training signals for analogical reasoning without explicit rule supervision.
  • Meta-Learning for Cross-Domain Generalization: Applying contrastive learning across different problem configurations (domains) is shown to be effective in making the analogical reasoning strategy robust to unseen attributes and variations.

Key Results

  • Outperforms State-of-the-Art: The proposed method significantly outperforms existing state-of-the-art models, including CoPINet, on the RAVEN dataset.
  • Dramatic Gains in Few-Shot Settings: The performance improvements are particularly pronounced in few-shot (limited training data) scenarios, where the proposed method shows much larger gains compared to baselines. For instance, with very limited data (0.049% of the training set), their model achieves over 40% accuracy, while baselines are much lower.
  • Strong Generalization to Unseen Attributes: The model demonstrates impressive generalization capabilities on unseen visual attributes (shapes), achieving significant improvements (over 50% relative improvement) compared to baselines.
  • Improved Performance with Meta-Contrast: Combining analogical learning with meta-contrastive learning across domains further boosts performance, especially in cross-domain generalization tasks.

Limitations and Future Work

  • Computational Cost: The paper notes that generative analogy learning, while effective, requires more memory and thus smaller minibatch sizes, potentially impacting training speed.
  • Generalizability to Other Domains: While the authors believe their method has potential for other applications, the current validation is primarily on visual reasoning tasks.
  • Future Work: The authors suggest that their analogical learning strategy could be useful in other areas where similar relational structures are present, hinting at broader applicability beyond visual reasoning. They also explored an ablation study on meta-analogical contrastive learning using pairwise samples on cross-domains.

Summary updated

Abstract

While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task. In this work, we propose to solve such a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning, which is a unique human ability to identify structural or relational similarity between two sets. Specifically, given training and test sets that contain the same type of visual reasoning problems, we extract the structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We repeatedly apply this process with slightly modified queries of the same problem under the assumption that it does not affect the relationship between a training and a test sample. This allows to learn the relational similarity between the two samples in an effective manner even with a single pair of samples. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce. We further meta-learn our analogical contrastive learning model over the same tasks with diverse attributes, and show that it generalizes to the same visual reasoning problem with unseen attributes.
BibTeX
@inproceedings{kim2020few,
  author = {Youngsung Kim and Jinwoo Shin and Eunho Yang and Sung Ju Hwang},
  title = {{Few-shot Visual Reasoning with Meta-analogical Contrastive Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=UmU08pv7eig8},
}
Figure