Fast amortized inference of neural activity from calcium imaging data with variational autoencoders

Artur Speiser, Jinyao Yan, Evan W. Archer, Lars Buesing, Srinivas C. Turaga, Jakob H. Macke

Overview

The authors introduce DeepSpike, a framework that uses Variational Autoencoders (VAEs) to "learn" how to infer neural spikes from calcium imaging data. By training a deep neural network to invert biophysical models, they achieve state-of-the-art accuracy with massive speedups, enabling real-time spike inference for thousands of neurons simultaneously.

The Challenge of Spike Inference

In neuroscience, calcium imaging is a vital tool for watching brain activity. However, it’s an indirect measure: neurons fire electrical "spikes," which cause a surge in calcium, which then makes fluorescent indicators glow. To understand what the brain is actually doing, researchers must work backward from the "glow" (fluorescence traces) to the original spikes.

Existing solutions fall into two camps:

  1. Generative/Bayesian Methods: These use physics-based models of how calcium behaves. They are accurate and don't need much training data, but they are painfully slow because they require complex math (MCMC sampling) or iterative optimization for every single neuron.
  2. Supervised Deep Learning: These are lightning-fast but require massive amounts of "ground truth" data (simultaneous electrical and optical recordings), which are incredibly difficult and expensive to collect.

The gap is a method that is fast, accurate, and unsupervised (doesn't require ground truth labels).

The DeepSpike Framework

The authors propose DeepSpike (DS), which treats spike inference as a variational autoencoder (VAE) problem.

DeepSpike VAE framework and recognition model architecture
  • The Generative Model (The Decoder): This is a biophysical model that describes how spikes produce fluorescence. The authors show their framework works with simple linear models and complex nonlinear models.
  • The Recognition Model (The Encoder): This is a deep neural network trained to "invert" the generative model. It takes the fluorescence trace and predicts the probability of spikes.
  • Amortized Inference: Instead of solving a hard optimization problem for every new piece of data, the network learns a general strategy for solving the problem during training. At test time, inference is just a single "forward pass" through the network.

The authors tested two encoder architectures:

  • DS-F (Factorized): A 1D Convolutional Neural Network (CNN) that treats each time point independently.
  • DS-NF (Non-Factorized): A Recurrent Neural Network (RNN) that captures temporal correlations (e.g., the fact that a neuron is unlikely to fire two spikes in a tiny fraction of a millisecond).

Datasets and Evaluation

The models were evaluated on:

  • Simulated Data: Generated using linear and nonlinear models to provide a controlled environment for testing accuracy and the impact of temporal correlations.
  • Real Experimental Data: Two-photon imaging data from mouse visual cortex (GCaMP6s and GCaMP6f indicators) with simultaneous cell-attached recordings (ground truth) from the GENIE project.
  • Dendritic Simulation: A multi-dimensional dataset designed to test the model's ability to separate local synaptic inputs from global backpropagating action potentials in dendrites.

Key Insights

  • Unsupervised Learning: By using the VAE framework, the model "teaches itself" to find spikes by trying to reconstruct the fluorescence traces. It doesn't need human-labeled spikes to learn.
  • The Power of Amortization: The computational "heavy lifting" is done once during training. This allows the model to process 500 neurons' worth of data in less than 90 seconds—orders of magnitude faster than traditional Bayesian methods.
  • Temporal Correlations Matter: The RNN-based model (DS-NF) outperformed the CNN by better accounting for the "refractory period" of neurons, assigning higher posterior probability to the true spike trains.
  • Black-Box Flexibility: Because the framework is "black-box," you can swap out the underlying biophysical model (the decoder) as better ones are developed without changing the core inference algorithm.

Results and Performance

DeepSpike demonstrates significant improvements in speed while maintaining high accuracy.

Reconstruction performance and quantitative comparison of DeepSpike models
  • Speed: On a GPU, the factorized model (DS-F) can process a 10,000-sample trace in 0.001 seconds. Even the more complex RNN (DS-NF) takes only 1.5 seconds, comparing favorably to the 9.2 seconds required by previous state-of-the-art methods on a CPU.
  • Accuracy: DeepSpike achieved competitive or superior correlation scores compared to established methods like MCMC and CDEC on both simulated and real GCaMP6 data.
  • Dendritic Demixing: The model successfully separated synaptic inputs from backpropagating action potentials in simulated dendrites, achieving a correlation of 0.84 (soma) and 0.78 (spine), significantly outperforming baseline methods.

Limitations and Future Work

  • Generalization: Like all neural networks, a model trained on one type of calcium indicator might not perform perfectly on another without retraining or fine-tuning.
  • Model Mismatch: If the underlying biophysical model (the decoder) is inaccurate, the inference will be biased. Developing more accurate generative models is a key next step.
  • Real-Time Applications: The speed of DeepSpike opens the door for "closed-loop" experiments where a computer could trigger a stimulus based on inferred neural activity in real-time.

Summary updated

Abstract

Calcium imaging permits optical measurement of neural activity. Since intracellular calcium concentration is an indirect measurement of neural activity, computational tools are necessary to infer the true underlying spiking activity from fluorescence measurements. Bayesian model inversion can be used to solve this problem, but typically requires either computationally expensive MCMC sampling, or faster but approximate maximum-a-posteriori optimization. Here, we introduce a flexible algorithmic framework for fast, efficient and accurate extraction of neural spikes from imaging data. Using the framework of variational autoencoders, we propose to amortize inference by training a deep neural network to perform model inversion efficiently. The recognition network is trained to produce samples from the posterior distribution over spike trains. Once trained, performing inference amounts to a fast single forward pass through the network, without the need for iterative optimization or sampling. We show that amortization can be applied flexibly to a wide range of nonlinear generative models and significantly improves upon the state of the art in computation time, while achieving competitive accuracy. Our framework is also able to represent posterior distributions over spike-trains. We demonstrate the generality of our method by proposing the first probabilistic approach for separating backpropagating action potentials from putative synaptic inputs in calcium imaging of dendritic spines.
BibTeX
@inproceedings{speiser2017fast,
  author = {Artur Speiser and Jinyao Yan and Evan W. Archer and Lars Buesing and Srinivas C. Turaga and Jakob H. Macke},
  title = {{Fast amortized inference of neural activity from calcium imaging data with variational autoencoders}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2017},
  url = {https://openreview.net/forum?id=rk-TtU-_WS},
}
Figure