Neural Computation with Non-negative Firing Rate Constraints

This direction is split into other directions: non-negative firing rate constraint. Its last version stays readable and citable here.

Version 1 of 129 papers39 conceptsfrozen 2026-10-04

In standard artificial neural networks, numbers are just numbers. A weight can be -0.5, and an activation can be -1.2. However, biological neurons communicate through action potentials—discrete spikes that result in a "firing rate." Because you cannot fire fewer than zero spikes per second, biological signals are fundamentally non-negative.

While this might seem like a trivial implementation detail, recent research suggests that this non-negativity constraint is a primary driver of how the brain represents information, learns from errors, and even organizes its spatial maps.

The Challenge of Negative Information

Most machine learning algorithms rely on signed values to function. For example, backpropagation requires calculating an error signal that tells a neuron to either increase or decrease its activity. If an error is negative, the gradient is negative.

As Roy Abel and Shimon Ullman point out in Biologically-Motivated Learning Model for Instructed Visual Processing, this creates a "sign problem." If a neuron can only signal "more," how does it signal "less"? Researchers like Changze Lv have proposed mechanisms like Bidirectional Spike-based Distillation (BSD) to solve this. In Biologically Plausible Learning via Bidirectional Spike-Based Distillation, they demonstrate that by using separate feedforward and feedback pathways, networks can learn using only unsigned, discrete spikes, achieving the efficiency of biological systems without needing negative gradients.

Breaking Symmetry to Find Unique Features

In abstract mathematics, many ways to represent data are equivalent. If you use Principal Component Analysis (PCA) to compress an image, you can rotate the resulting "basis vectors" without losing information. This rotational symmetry makes it hard to say that any one neuron represents a specific, meaningful feature.

However, Will Dorrell, Peter E. Latham, and James C. R. Whittington argue in Convex Efficient Coding that non-negativity breaks this symmetry. Because neurons cannot go below zero, the "optimal" way to represent a signal becomes unique. This mathematical "convexity trick" explains why we can find individual neurons in the brain that respond to very specific things, like a particular orientation of a line or a specific person's face.

This is why techniques like Nonnegative Matrix Factorization (NMF) often outperform PCA in biological modeling. As Sung-Phil Kim and José C. Príncipe showed in Determining Patterns in Neural Activity for Reaching Movements Using Nonnegative Matrix Factorization, NMF extracts "parts-based" representations that map cleanly onto functional groups of neurons, whereas PCA produces mathematical abstractions that have no biological counterpart.

Forcing Structure in Spatial Maps

The non-negativity constraint also explains the strange, geometric patterns found in the brain's navigation system. Grid cells in the entorhinal cortex fire in a hexagonal pattern as an animal moves through space. Why hexagons?

According to the authors of Actionable Neural Representations: Grid Cells from Minimal Constraints, if you try to represent a location using periodic waves (like Fourier transforms) while staying above zero, you are forced to combine multiple harmonic frequencies. To keep the "floor" of the signal at zero while creating a sharp "peak" of activity, the math naturally settles into a hexagonal grid.

Similarly, Minglu Zhao and Ying Nian Wu suggest in Place Cells as Multi-Scale Position Embeddings that hippocampal place cells are non-negative embeddings of random walks. While the underlying math of these walks often involves negative values, the brain's requirement for non-negative firing rates forces these representations to become sparse and localized, creating the "place fields" that allow for path planning.

Rectification as a Computational Tool

If the brain needs to perform a subtraction—for example, to filter out background noise in the auditory system—it must handle the resulting negative values. In Incorporating Midbrain Adaptation to Mean Sound Level Improves Models of Auditory Cortical Processing, Ben D. B. Willmore and colleagues show that the brain uses half-wave rectification. It simply clips any value below zero to zero.

This "kink" in the response curve isn't just a loss of information; it's a source of non-linearity that the network can use. David G. T. Barrett, Christian K. Machens, and Sophie Denève demonstrated in Firing rate predictions in optimal balanced networks that when a neuron hits this "zero-floor," the rest of the network must adjust to compensate. This interaction creates the complex, non-linear tuning curves seen in real biological data, allowing the brain to perform sophisticated computations using simple, non-negative parts.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_computation_with_non_negative_fir_54ce7193,
  title = {Neural Computation with Non-negative Firing Rate Constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-computation-with-non-negative-firing-rate-constraints/rdr_54ce71930a29426995085a40e5bfc2e7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_computation_with_non_negative_fir_54ce7193_v1,
  title = {Neural Computation with Non-negative Firing Rate Constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-computation-with-non-negative-firing-rate-constraints/rdr_54ce71930a29426995085a40e5bfc2e7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}