This editorial introduces a special issue of Frontiers in Computational Neuroscience that explores the growing synergy between Artificial Neural Networks (ANNs) and computational neuroscience. It highlights how ANNs, from simple rate-based models to complex spiking neural networks, are becoming powerful tools for understanding how biological brains process information, and vice-versa.
What Problem Does This Paper Address?
The paper addresses the increasing convergence of Artificial Intelligence (AI) and neuroscience. While ANNs have achieved remarkable success in solving complex tasks, their utility as models for understanding biological neural computation is a subject of ongoing research. The core question is whether ANNs are merely convenient computational tools or if they can offer genuine insights into the mechanisms of the brain. This special issue aims to bridge this gap by showcasing research that uses ANNs to model neural information processing and explores the biological plausibility of ANN architectures and learning rules.
Methods
This editorial doesn't present new methods but rather summarizes the diverse approaches discussed in the accompanying research papers. These include:
Rate-based ANNs: Standard AI models used for complex problem-solving.
Spiking Neural Networks (SNNs): More biologically plausible models that mimic the temporal dynamics of neuronal firing.
Equilibrium Propagation: A novel learning framework for energy-based models that avoids separate error propagation circuits.
Neuromodulator Models: Neural network models incorporating the physiological release schedules of neuromodulators like acetylcholine (ACh) and dopamine (DA).
Representational Distance Learning (RDL): A method to align the representational spaces of different neural network models.
Recurrent Neural Networks (RNNs): Used to model how neural representations of sensory input are functionally organized.
Deep Neural Networks: Applied to decode brain activity, such as dream content from fMRI data.
Datasets
Not discussed in the paper.
Key Insights and Ideas
ANNs as Theoretical Vehicles: ANNs are not just for AI tasks; they serve as valuable theoretical frameworks for understanding neural information processing in biological systems.
Biological Plausibility is Key: A central theme is evaluating whether ANNs reflect how real brains learn and compute. This includes questioning the biological plausibility of learning algorithms like backpropagation.
Bridging the Gap: The research presented in the special issue demonstrates how ANNs can be used to tackle fundamental neuroscience questions, such as how sensory stimuli relate to neural activity and how the brain forms sensory maps.
Neuromodulation Matters: Understanding the role of neuromodulators like ACh and DA is crucial for building more biologically realistic neural models.
Learning and Optimization: There's a growing insight that biological learning might be driven by cost function optimization, similar to how ANNs are trained.
Dream Decoding: Advanced ANNs are showing promise in decoding complex cognitive states like dreams from brain imaging data.
Key Results
This editorial summarizes the themes and approaches of the papers in the special issue, rather than presenting specific quantitative results. The key "results" are the insights and advancements discussed within those papers, such as:
New learning frameworks (e.g., Equilibrium Propagation) that offer greater biological plausibility.
Models that better capture the relationship between sensory input and neural activity.
Demonstrations of how ANNs can be used to decode complex brain states like dreams.
Evidence that ANNs can learn to represent information in ways analogous to biological sensory maps.
Limitations and Future Work
The editorial highlights the ongoing challenge of establishing the full biological plausibility of ANNs, particularly concerning learning mechanisms like backpropagation. Future work is expected to involve:
Continued development of more biologically realistic ANN architectures and learning rules.
Deeper integration of neuromodulatory effects into neural models.
Further exploration of how ANNs can explain the hierarchical organization of sensory processing in the brain.
Increased interaction between the AI and neuroscience communities to foster theoretical and practical breakthroughs.
Summary updated
Abstract
IntroductionIn artificial intelligence (AI), new advances make it possible that artificial neural networks (ANNs) learn to solve complex problems in a reasonable amount of time (LeCun, Bengio, & Hinton, 2015). To the computational neuroscientist, ANNs are theoretical vehicles that aid in the understanding of neural information processing (van Gerven, 2017). These networks can take the form of the rate-based models that are used in AI or more biologically plausible models that make use of spiking neurons (Brette, 2015). The objective of this special issue is to explore the use of ANNs in the context of computational neuroscience from various perspectives.Biological plausibilityBiological plausibility is an important topic in neural networks research. That is, are ANNs simply convenient computational models or do they also inform about the computations that take place in our own brains? Marblestone et al. (2016) carefully lay out the rapid advances in deep learning and contrast these developments with current practice and views in neuroscience. Their main insight is that biological learning may be driven by the optimization of cost functions using successive neural network layers. A classic question that has haunted ANNs for years is whether backpropagation is biologically plausible (Crick, 1989). Scellier and Bengio (2017) introduce Equilibrium Propagation as a new learning framework for energy-based models. The algorithm computes the gradient of an objective function without relying on separate circuits for error propagation that integrate non-local signals.While acetylcholine (Ach) and dopamine (DA) are neuromodulators that are known to have profound and lasting effects on the neural responses to stimuli, it is unknown what their respective functional roles are. Holca-lamarre et al. (2017) develop a neural network model that is combined with the physiological release schedules of ACh and DA.Improving performanceSeveral papers propose new mechanisms to improve the perfomance of ANNs.Li et al. (2016) investigate chunking, which is a phenomenon referring to the grouping of items when performing a memory task, leading to improvements in task performance. The authors show that chunking can have computational benefits as it allows the use of synapses with narrow dynamic range and low precision when performing a memory task. An important limitation of Hopfield networks is their limited storage capacity. Folli et al. (2017) show that by allowing non-zero diagonal elements on the weight matrix, maximal storage capacity is obtained when the number of stored memory patterns exceeds the network size. McClure and Kriegeskorte (2016) introduce representational distance learning (RDL) as a stochastic gradient descent method that drives the representational space of a student model to approximate the representational space of a teacher model. Spiking neural networksAn important endeavor in computational neuroscience is to further our understanding of biological and artificial spiking neural networks. How sensory stimuli relate to the activity of neurons is one of the big open questions in neuroscience, and determining this relationship between the input a neuron receives and the outgoing spike-train has remained a challenge. Zeldenrust et al. (2017) propose a new ANN-based method to measure in vitro how much information a neuron transfers in this process. The rate with which spikes are emitted is often mapped to the analog activation values of artificial neurons, but it is well known that this relationship captures only part of the information processing in real neurons. Carrillo-medina and Latorre (2016) develop networks of spiking neurons that operate based on the principles developed for so-called signature neural networks.How does the central nervous system develop the hierarchy of sensory maps that reflect different internal or external patterns and/or states? Chen (2017) shows how simple recurrent and reentrant neuronal networks can discriminate different inputs and generate sensory maps. Understanding brain functionANNs have also been embraced as a new tool for understanding neural information processing in the brain. In this special issue, a number of advances in this area are put forward. One question is whether supervised or unsupervised neural networks provide better explanations of neural information processing. Testolin et al. (2017) taught neural networks to learn an explicit mapping between different spatial reference frames. They show that both network architecture and the employed learning paradigm affect neural coding properties. An elusive property of our own brains is that we engage in dreaming during sleep. Horikawa and Kamitani (2017) used deep neural networks in an effort to decode what people dream about. They found that decoded features from dream fMRI data positively correlated with those associated with the object categories that related to the dream content. An important question in neuroscience is how neural representations to sensory input are functionally organized. Güçlü and van Gerven (2017) show that neural responses to sensory input can be modelled using recurrent neural networks that can be trained end-to-end.ConclusionNeural networks are experiencing a revival that not only transforms AI but also provides new insights about neural computation in biological systems. The contributions in this special issue describe new advances in neural networks that increase their efficacy or plausibility from a biological point of view. A closer interaction between the AI and neuroscience communities is expected to lead to various other theoretical and practical breakthroughs in the years to come.
BibTeX
@article{gerven2017editorial,
author = {Marcel van Gerven and Sander M. Bohté},
title = {{Editorial: Artificial Neural Networks as Models of Neural Information Processing}},
journal = {Frontiers Comput. Neurosci.},
year = {2017},
doi = {10.3389/fncom.2017.00114},
}