On the generalization capacity of neural networks during generic multimodal reasoning

Takuya Ito, Soham Dan, Mattia Rigotti, James Kozloski, Murray Campbell

Large Language Models (LLMs) often appear to possess human-like reasoning, but it remains notoriously difficult to determine if they are truly "reasoning" or merely performing sophisticated data interpolation. This ambiguity is compounded by the massive, opaque datasets used for pretraining.

To isolate how specific architectural choices influence a model's ability to generalize, researchers evaluated several base neural network architectures in multimodal settings—where a model must combine visual information with logical instructions.

The gCOG Benchmark and Generalization Types

To ensure full experimental control, the authors introduced Generic COG (gCOG), a configurable benchmark evolved from the original COG dataset. Instead of raw pixels, gCOG uses abstracted categorical tokens (e.g., "orange 't'"), allowing researchers to test reasoning without the confounding variable of computer vision performance. It can also generate task trees of arbitrary depth.

Examples of the gCOG task, showing a simple depth-1 task and a more complex depth-3 task involving conditional logic.

Using gCOG, the study focused on three specific types of out-of-distribution (OOD) generalization:

  1. Distractor Generalization: Can the model ignore irrelevant "noise" in the input?
  2. Systematic Compositional Generalization: Can the model handle new combinations of known components (e.g., understanding a "red square" after only seeing "red circles" and "blue squares")?
  3. Productive Compositional Generalization: Can the model scale its logic to more complex, deeper task structures than it saw during training?

Evaluated Architectures

The researchers trained six base architectures from scratch on the same number of samples:

  • RNNs & GRUs: Classic recurrent models.
  • Single Stream Transformer (SSTfmr): Concatenates task and stimulus into one sequence.
  • Dual Stream Transformer (DSTfmr): Processes modalities separately before merging.
  • Transformers with Cross Attention (CrossAttn): Uses one modality to "query" the other.
  • Perceiver: A cross-attention-heavy architecture designed specifically for multimodal inputs.

Key Findings

Cross-Modal Attention is Essential for Robustness Models that explicitly leverage cross-attention mechanisms (CrossAttn and Perceiver) were significantly more robust. For Distractor Generalization, while most models performed well on in-distribution data, only the cross-attention models maintained high accuracy (>99%) when the number of distractors scaled from 5 up to 40. Standard Transformers saw sharp performance drops as noise increased.

The Perceiver also emerged as the clear winner in Systematic Generalization, achieving 65.7% OOD accuracy on depth-3 tasks, significantly outperforming RNNs (54.1%) and Single Stream Transformers (35.9%). Representational Similarity Analysis (RSA) revealed that these high-performing models maintained internal representations that were better aligned with the actual task features.

The "Productivity" Wall Despite successes in handling distractors and new combinations, every single architecture failed the Productive Generalization test. Even if a model perfectly learned a depth-3 logic tree during training, its accuracy dropped to chance levels when asked to solve a depth-5 or depth-7 tree.

Notably, increasing the number of layers and attention heads improved systematic generalization but did absolutely nothing to overcome this productivity failure. This suggests a hard ceiling on current architectures: Transformers and RNNs fundamentally struggle with the recursive, hierarchical nature of complex, unseen reasoning depths.

Conclusion and Future Directions

While cross-attention is a powerful mechanism for integrating multiple sources of information and handling distractors, current neural architectures lack the ability to piece together syntactic structures on-the-fly for deeper logical tasks. Future research may need to look beyond standard Transformers toward architectures that can handle recursion more naturally, such as neuro-symbolic hybrids or novel memory mechanisms. The gCOG benchmark remains an open-source tool for the community to tackle these fundamental missing links in AI reasoning.

Summary updated

Abstract

The advent of the Transformer has led to the development of large language models (LLM), which appear to demonstrate human-like capabilities. To assess the generality of this class of models and a variety of other base neural network architectures to multimodal domains, we evaluated and compared their capacity for multimodal generalization. We introduce a multimodal question-answer benchmark to evaluate three specific types of out-of-distribution (OOD) generalization performance: distractor generalization (generalization in the presence of distractors), systematic compositional generalization (generalization to new task permutations), and productive compositional generalization (generalization to more complex tasks with deeper dependencies). While we found that most architectures faired poorly on most forms of generalization (e.g., RNNs and standard Transformers), models that leveraged cross-attention mechanisms between input domains, such as the Perceiver, fared better. Our positive results demonstrate that for multimodal distractor and systematic generalization, cross-attention is an important mechanism to integrate multiple sources of information. On the other hand, all architectures failed in productive generalization, suggesting fundamental limitations of existing architectures for specific types of multimodal OOD generalization. These results demonstrate the strengths and limitations of specific architectural components underlying modern neural models for multimodal reasoning. Finally, we provide *Generic COG* (gCOG), a configurable benchmark with several multimodal generalization splits, for future studies to explore.
BibTeX
@inproceedings{ito2024generalization,
  author = {Takuya Ito and Soham Dan and Mattia Rigotti and James Kozloski and Murray Campbell},
  title = {{On the generalization capacity of neural networks during generic multimodal reasoning}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=zyBJodMrn5},
}
Figure