A lattice filter model of the visual pathway

Karol Gregor, Dmitri B. ChklovskiiNeurIPS 2012 · OpenReview

Introduction

Our eyes are bombarded with a massive amount of information. Natural scenes are highly redundant; what happens at one moment is usually a good predictor of what happens a millisecond later. To transmit this information efficiently, the visual system must "whiten" or decorrelate the signal—essentially stripping away the predictable parts and only sending the "surprises" (prediction errors) to the brain. This is known as predictive coding.

Despite this understanding, two major puzzles remain in visual neuroscience:

  1. The Time Constant Paradox: Biological neurons and synapses have very short time constants (less than 100ms), yet they manage to decorrelate signals over much longer timescales. How do they build "long memories" out of "short-lived" components?
  2. Cell Diversity: Why does the lateral geniculate nucleus (LGN) contain two distinct populations of cells—lagged and non-lagged—that seem to respond to the same stimuli with different timings?

This paper proposes that the visual pathway—from the retina to the primary visual cortex—functions like a "lattice filter," a classic tool from signal processing used to decorrelate signals. By modeling the brain this way, the authors explain how neurons with very short internal timers can process visual information over long durations and why the brain contains specific, seemingly redundant cell types.

The Lattice Filter Approach

The authors bridge the gap between engineering and biology by applying Lattice Filters to the visual pathway.

In signal processing, a lattice filter is a cascade of stages. Each stage takes an input and tries to predict its current value based on the immediate past. Crucially, it calculates two things at every step:

  • Forward Error: The difference between the current signal and a prediction based on the past.
  • Backward Error: The difference between a past signal and a prediction based on the "future" (more recent samples).
Continuous-time lattice filter architecture

To make this model biologically plausible, the authors adapted the standard discrete-time filter into a continuous-time version using "all-pass filters" (specifically Laguerre polynomials). This allows the model to simulate neurons that process continuous streams of light rather than discrete digital samples. Finally, they show that the connection weights in this filter can be learned using Hebbian rules—the classic "cells that fire together, wire together" mechanism of biological learning.

Biological Validation and Key Insights

As a theoretical modeling paper, this work validates its model against decades of established electrophysiological recordings from cats, primates (RGCs, LGN neurons, and V1 simple cells), and insects (fruit fly LMCs).

  • The Cascade is the Key: The visual system’s physical structure (Retina $\rightarrow$ LGN $\rightarrow$ V1) mirrors the mathematical structure of a lattice filter. Each anatomical stage adds another layer of decorrelation, naturally extending the "memory" of the system without requiring individual neurons to have long time constants.
  • Lagged Cells are "Backward Errors": One of the most elegant findings is that "non-lagged" neurons correspond to the forward error branch of the filter, while "lagged" neurons correspond to the backward error branch. They aren't just redundant; they are two sides of the same mathematical coin required for optimal signal processing.
  • Developmental Efficiency: Because lattice filters can be learned stage-by-stage, the retina can "finish" its development and wiring before the LGN or V1 even start. The earlier stages don't need to be retrained when a new, higher-level stage is added to the cascade.
  • Cross-Species Unity: The same lattice filter logic explains why fruit flies have a pair of neurons (L1 and L2) for every "pixel" in their eye. These cells likely represent the forward and backward error branches, just like in mammals.

Key Results

The model's predictions align closely with observed biological phenomena:

  • Receptive Field Growth: The model correctly predicts that temporal receptive fields (the window of time a neuron cares about) get progressively longer as you move from the retina to the LGN and then to the V1.
Temporal filters for RGC and LGN
  • Biphasic Responses: The model produces the characteristic "peak and rebound" (biphasic) response seen in real neurons. In "forward" (non-lagged) cells, the initial peak is stronger than the rebound; in "backward" (lagged) cells, the rebound is more prominent.
  • Step-Response Matching: When simulated with a "step" stimulus (suddenly turning a light on and keeping it on), the model’s output qualitatively matches the transient-then-sustained firing patterns recorded in real cat LGN cells.

Limitations and Future Directions

The current model is effectively linear. While it handles temporal processing beautifully, it doesn't fully account for the non-linearities of the "ON" and "OFF" pathways (neurons that respond specifically to light increasing vs. decreasing).

The authors suggest that future work could integrate these lattice filters with more complex spatial processing models. Additionally, they point toward connectomics—the high-resolution mapping of actual neural wiring—as the ultimate way to prove if the physical connections in the brain truly match the cross-link structure of a lattice filter.

Summary updated

Abstract

Early stages of visual processing are thought to decorrelate, or whiten, the incoming temporally varying signals. Because the typical correlation time of natural stimuli, as well as the extent of temporal receptive fields of lateral geniculate nucleus (LGN) neurons, is much greater than neuronal time constants, such decorrelation must be done in stages combining contributions of multiple neurons. We propose to model temporal decorrelation in the visual pathway with the lattice filter, a signal processing device for stage-wise decorrelation of temporal signals. The stage-wise architecture of the lattice filter maps naturally onto the visual pathway (photoreceptors -> bipolar cells -> retinal ganglion cells -> LGN) and its filter weights can be learned using Hebbian rules in a stage-wise sequential manner. Moreover, predictions of neural activity from the lattice filter model are consistent with physiological measurements in LGN neurons and fruit fly second-order visual neurons. Therefore, the lattice filter model is a useful abstraction that may help unravel visual system function.
BibTeX
@inproceedings{gregor2012lattice,
  author = {Karol Gregor and Dmitri B. Chklovskii},
  title = {{A lattice filter model of the visual pathway}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2012},
  url = {https://openreview.net/forum?id=ryl4KuZO-B},
}
Figure