A New Approach towards Vision Suggested by Biologically Realistic Neural Microcircuit Models

Wolfgang Maass, Robert Legenstein, Henry MarkramBiologically Motivated Computer Vision 2002 · DOI

TL;DR

This paper proposes a novel paradigm for visual processing: instead of explicitly extracting features from discrete frames, it injects continuous visual input directly into a complex, biologically realistic neural microcircuit. The circuit's internal dynamics then encode this information, allowing simple "readout" neurons to extract relevant features in real-time, even with partially unknown or faulty hardware. This approach demonstrates the potential of generic neural circuits as powerful, flexible "wetware" for complex temporal processing tasks.

What Problem Does This Paper Address?

Traditional approaches to processing dynamic visual information, like video, often involve storing discrete frames and then applying computationally intensive algorithms to extract features such as motion. This is akin to meticulously analyzing each frame of a movie to understand the plot. The authors argue that this approach is biologically implausible and inefficient, especially for complex, time-varying inputs. They aim to bridge this gap by proposing a method inspired by the hypothesized function of biological cortical microcircuits, which are known for their complex, recurrent, and sparsely connected nature. The challenge is to leverage these intricate biological structures for efficient and robust real-time visual processing without needing a precise understanding of every connection and neuron.

Methods

The core of the proposed method is the use of a generic neural microcircuit model. This model is a large (768 neurons), sparsely but highly recurrently connected network of leaky integrate-and-fire neurons, with a biologically realistic mix of excitatory and inhibitory neurons and dynamic synapses.

The key innovation lies in how visual input is processed:

  • Direct Input Injection: Instead of frame-by-frame analysis, continuous time-varying visual stimuli are directly fed into the microcircuit.
  • Dynamical System Encoding: The complex internal dynamics of the microcircuit transform the input stream into a high-dimensional, nonlinear "code" within the network's state. This transformation is hypothesized to act like a kernel, facilitating subsequent processing.
  • Simple Readout Devices: Trained "readout" neurons (or pools of neurons) are then used to extract specific information from the current state of the microcircuit. Crucially, these readouts are trained using standard static pattern classification or regression techniques, as the temporal dynamics have already been encoded by the microcircuit.
  • Unsupervised Learning for Readouts: The readout neurons are trained in an unsupervised manner, meaning they learn to predict future sensor activations based on past circuit states, without explicit supervision for every step.

This approach bypasses the need for explicit feature extraction algorithms and allows for parallel processing of diverse tasks by training different readouts on the same microcircuit.

Datasets

The paper uses simulated visual input generated from an 8x8 sensor array. The "datasets" consist of simulated movements of two object types: a ball and a bar. Trajectories are generated randomly, covering a range of speeds and directions. The paper does not introduce a new benchmark dataset but rather uses these simulated visual scenarios to test the capabilities of their proposed model.

Key Insights and Ideas

  • Cortical Microcircuits as Temporal Kernels: The paper suggests that generic neural microcircuits can function as powerful "temporal kernels," transforming raw, time-varying input into a high-dimensional representation that simplifies subsequent information extraction by linear readouts.
  • Unsupervised Learning for Prediction: The success of training readout neurons in an unsupervised manner to predict future sensor states highlights the inherent predictive capabilities embedded within the microcircuit's dynamics.
  • Genericity and Flexibility: A single, generic microcircuit model can support a wide array of visual processing tasks (e.g., movement prediction, novelty detection, object classification) by simply training different readout units. This contrasts with specialized circuits designed for specific tasks.
  • Robustness to Imperfection: The approach is theoretically robust to partially unknown or faulty analog hardware, as the complex encoding is handled by the microcircuit's dynamics, and only simple readouts need to be trained.
  • "Imagined" Movement: The ability of the circuit to continue generating plausible future states even when real input is removed (by feeding back its own predictions) demonstrates a form of internal simulation or "imagination."

Key Results

The simulations demonstrate the feasibility of the proposed approach across several tasks:

  • Movement Prediction: Readout pools were trained to predict sensor activations 25 ms and 50 ms into the future. For 25 ms prediction, errors were low (8.5% false alarms, 4.8% missed predictions, 10.1 ms average error in activation time). For 50 ms prediction, performance was slightly lower but still significant (16.5% false alarms, 4.6% missed predictions, 14.5 ms average error).
  • Movement Direction Estimation: 28 readout pools were trained to predict which perimeter sensors would be activated when an object left the field. This task, requiring disambiguation of movement, resulted in an average error of 4.9 degrees in estimating the movement direction.
  • Object Classification: One readout pool trained in a supervised manner to classify objects (ball vs. bar) achieved 0% error on 300 test examples.
  • Speed Estimation: Another supervised readout pool estimated object speed with an average error of 1.48 units per second.
  • Novelty Detection: The system demonstrated an ability to detect surprising events (e.g., a change in object direction) by observing a significant increase in prediction errors, even though it was never explicitly trained for novelty detection.
  • "Imagined" Movement: When real input was replaced by predictions, the circuit autonomously generated a continuation of the movement, which proceeded faster than the initial real movement. The firing patterns of neurons during imagined movement were remarkably similar to those during real movement, albeit temporally compressed.

Limitations and Future Work

  • Scalability: The authors note that prediction errors could be further reduced by employing larger generic recurrent circuits.
  • Complexity of Tasks: While the generic circuit performed well, the paper acknowledges that explicitly constructed circuits might offer better performance for highly specialized tasks.
  • Future Work: The authors are applying this paradigm to real-time processing of actual visual input and infrared sensors for a mobile robot. They also plan to investigate the combination of top-down processing of expectations with bottom-up visual information processing.

Summary updated

BibTeX
@inproceedings{maass2002new,
  author = {Wolfgang Maass and Robert Legenstein and Henry Markram},
  title = {{A New Approach towards Vision Suggested by Biologically Realistic Neural Microcircuit Models}},
  booktitle = {Biologically Motivated Computer Vision},
  year = {2002},
  doi = {10.1007/3-540-36181-2\_28},
}

Loading related content...

Figure