Model of a Biological Neuron as a Temporal Neural Network

Sean D. Murphy, Edward W. KairissNeurIPS 1994

Researchers at Yale have developed the Spatio-Temporal Event Mapping (STEM) cell, a neural network architecture designed to mimic the complex behavior of biological neurons. By training an artificial network to learn the "transfer function" of a biophysically realistic neuron, they achieved a 10x speedup and a 90% reduction in memory usage compared to traditional simulation methods.

The Computational Bottleneck of Biological Realism

To understand how the brain processes information, neuroscientists often build "compartmental models." These models treat a single neuron as a series of connected segments, each governed by complex, non-linear differential equations representing ion channels and membrane physics.

While highly realistic, these models are a computational nightmare. Simulating even a small network of a few thousand neurons requires solving thousands of "stiff" equations with tiny time steps. This computational burden makes it nearly impossible to study large-scale brain functions like learning and memory. The authors argue that while molecular details are important for some studies, network-level simulations primarily require the neuron's dynamical transfer function—the rule that maps incoming pulses (synaptic activations) to outgoing pulses (action potentials).

The STEM Architecture

The authors introduce the STEM architecture, which treats a neuron as a device that maps a multidimensional temporal signal into a unidimensional one.

Detailed schematic of the STEM architecture

The architecture consists of four main components:

  1. The Tau Layer: Acting as a "preprocessor," this layer consists of nodes that calculate second-order rise-and-decay functions. It essentially translates a sequence of spikes over time into a high-dimensional vector representing the current "state" of the neuron.
  2. Feedback Tau Node: A specialized node that tracks the cell’s own recent firing history, allowing the model to account for refractory periods and adaptation.
  3. Multilayer Perceptron (MLP): A standard feedforward neural network that takes the vector from the Tau layer and classifies it as either "FIRE" or "DON'T FIRE."
  4. Output Filter: A final stage that enforces a threshold and a refractory period, ensuring the output looks like discrete biological spikes rather than a continuous value.

Training via Teacher-Student Simulation

The training process employs a "teacher-student" approach. The researchers first run a high-fidelity biological simulation (the teacher) and then train the STEM cell (the student) to replicate its firing patterns.

The "Teacher" model used was Roger Traub’s (1991) compartmental model of a hippocampal CA1 cell. This is a sophisticated model with 19 compartments and approximately 100 voltage- and calcium-sensitive channel mechanisms. For input data, 94 synapses were placed on the model, each receiving random spike trains at an average frequency of 10Hz. The response of the Traub model was recorded and used to create "FIRE/NO-FIRE" labels for the STEM cell's MLP.

Key Insights and Results

A core philosophy of this work is abstraction over implementation: the functional behavior of a neuron can be separated from its biological implementation. We don't need to simulate every ion channel to understand how a neuron contributes to a circuit. By using temporal-to-spatial projection in the Tau layer, the architecture converts a history of events into a single spatial vector that a standard MLP can process. Furthermore, because the STEM cell is a structured neural network, it is highly scalable and much easier to implement in specialized hardware (VLSI chips) than a system of differential equations.

When tested against both passive and active versions of the CA1 hippocampal cell, the STEM cell demonstrated remarkable efficiency and accuracy:

  • Speed: The STEM cell was approximately 10 times faster than the compartmental model.
  • Efficiency: It required only 10% of the memory of the original biophysical simulation.
  • Accuracy: The STEM cell's output closely tracked the firing patterns of the complex biophysical model, capturing the timing of spikes with high fidelity.
Comparison of spike trains between the compartmental model and the STEM cell

Limitations and Future Work

Despite its advantages, the STEM cell requires a pre-existing, computationally expensive model to act as a teacher. This training overhead means researchers must "pay" the computational cost upfront before reaping the speed benefits.

The authors also acknowledge that the current MLP implementation may not be optimal and suggest exploring modular recurrent networks to better capture long-term dependencies. Future work aims to integrate Hebbian learning rules and delay lines into the STEM architecture to investigate real-time dynamic pattern tracking in larger networks.

Summary updated

BibTeX
@inproceedings{murphy1994model,
  author = {Sean D. Murphy and Edward W. Kairiss},
  title = {{Model of a Biological Neuron as a Temporal Neural Network}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1994},
}
Figure