Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses

Tal Golan, JohnMark Taylor, Heiko Schütt, Benjamin Peters, Rowan P. Sommers, Katja Seeliger, Adrien Doerig, Paul Linton, Talia Konkle, Marcel van Gerven, Konrad Kording, Blake Richards, Tim C. Kietzmann, Grace W. Lindsay, Nikolaus Kriegeskorte

Abstract

An ideal vision model must account for behavior and neurophysiology across both naturalistic conditions and controlled laboratory experiments. While traditional psychological theories often rely on verbal descriptions, artificial neural networks (ANNs) provide a framework that is image-computable—meaning they can perform visual tasks and generate testable predictions for any arbitrary input. This paper argues that ANNs should not be viewed as a single, falsifiable hypothesis, but rather as a powerful computational language for expressing and testing a diverse range of hypotheses. By embracing the failures of specific models as "bugs" to be fixed rather than refutations of the entire framework, the "Neuroconnectionist" research program offers the most progressive path toward understanding the human visual system.

Introduction: Beyond the False Dichotomy

In contemporary vision science, a tension exists between traditional psychological theories and modern AI-driven models. Critics often point to the ways current ANNs diverge from human behavior—such as their susceptibility to adversarial noise or their reliance on texture over shape—as evidence that the entire ANN approach is fundamentally flawed.

However, this perspective creates a false dichotomy. This commentary, authored by leading computational neuroscientists, reframes the debate. Instead of asking "Are ANNs the right theory?", we should recognize that ANNs are the language we use to build better theories. They provide the mathematical and computational tools necessary to bridge the gap between two traditionally separate worlds: controlled laboratory experiments and large-scale naturalistic benchmarks.

The Neuroconnectionist Framework

The authors advocate for a research program termed "Neuroconnectionism." Drawing on Imre Lakatos’s philosophy of science, they argue that Neuroconnectionism is a "progressive" research program. In this view, the core commitment is that brain function emerges from the parallel processing of many simple units.

A key technical advantage of this framework is image-computability. Unlike a verbal theory (e.g., "humans prioritize global structure"), an image-computable model takes raw pixels as input and produces a quantitative, measurable output. This allows researchers to subject models to "severe tests" using any visual stimuli imaginable, from ImageNet photos to abstract psychophysical patterns.

ANNs as a Language, Not a Single Hypothesis

The central insight of the paper is that you do not "falsify" a language; you use it to write more accurate descriptions of reality. Because ANNs are universal approximators, they can theoretically implement any computational mechanism. If a specific model fails a test—such as showing a "texture bias" where humans show a "shape bias"—it represents a failure of that specific architecture or training regime, not the underlying framework.

These shortcomings are actually the engine of progress. For example, the discovery of texture bias in standard ImageNet-trained models led researchers to develop new training methods (like data augmentation with stylized images) that forced models to focus on shape. This iterative process brings models closer to human-like vision, demonstrating how the "language" of ANNs allows for the continuous refinement of computational hypotheses.

Synthesis of Evidence

While current ANNs are not perfect replicas of the human brain, they remain the "current best" models for several reasons:

  • Task Performance: They are the only existing models capable of recognizing objects or parsing complex scenes at human-level performance.
  • Neural Alignment: The intermediate layers of deep ANNs currently provide the most accurate match for the representations found in the human ventral visual stream, as evidenced by benchmarks like Brain-Score.
  • Predictive Power: They offer image-specific predictions for human errors and reaction times that, while imperfect, far exceed the predictive capabilities of alternative non-computational models.

The Path Forward

The authors acknowledge that a significant gap still exists between machine and human vision. Future work within the Neuroconnectionist program should focus on:

  1. Model Comparison: Rather than attempting to refute the entire framework, researchers should compare different architectures, objective functions, and training environments to see which best align with human data.
  2. Integrating Mechanisms: Standard feedforward models may need to be augmented with recurrent connections, feedback loops, and attention mechanisms to better capture the complexities of biological vision.
  3. Unifying Benchmarks: Developing models that can simultaneously satisfy the constraints of high-dimensional naturalistic data and low-dimensional, controlled psychological experiments.

By treating ANNs as a flexible language for expressing computational hypotheses, vision science can move past unproductive debates and focus on the iterative work of building a complete, computable theory of the human mind.

Summary updated

Abstract

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.
BibTeX
@inproceedings{golan2023deep,
  author = {Tal Golan and JohnMark Taylor and Heiko Schütt and Benjamin Peters and Rowan P. Sommers and Katja Seeliger and Adrien Doerig and Paul Linton and Talia Konkle and Marcel van Gerven and Konrad Kording and Blake Richards and Tim C. Kietzmann and Grace W. Lindsay and Nikolaus Kriegeskorte},
  title = {{Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses}},
  booktitle = {Behavioral and Brain Sciences},
  year = {2023},
  url = {https://openreview.net/forum?id=UIRdLh7sGf},
}
Figure