Brain-inspired algorithms for processing of visual data

Nicola Strisciuglio

TL;DR

This paper reviews how insights from the brain's visual system, particularly concerning neuron function and hierarchical organization, have inspired algorithms for image processing and computer vision. It highlights the crucial role of neural inhibition mechanisms in enhancing stability and robustness, and how these are being integrated into modern Convolutional Neural Networks (ConvNets).

What Problem Does This Paper Address?

The paper addresses the challenge of developing more effective and robust algorithms for processing visual data. It stems from the observation that the human visual system, honed by millions of years of evolution, is remarkably adept at handling complex visual information, even under challenging conditions like noise or varying illumination. Traditional computer vision algorithms often struggle with these complexities. The paper explores how understanding the biological underpinnings of visual processing, from individual neuron responses to the overall hierarchical structure of the visual cortex, can provide a blueprint for creating superior artificial vision systems. Specifically, it focuses on how the brain's mechanisms for detecting features like edges and lines, and its sophisticated use of inhibition to stabilize and refine these detections, can be translated into computational models.

Methods

The paper reviews a range of brain-inspired approaches for visual data processing. Key methods discussed include:

  • Neocognitron: An early hierarchical network inspired by the simple and complex cells of the visual cortex, demonstrating unsupervised learning of feature hierarchies.
  • Gabor Filters and Difference-of-Gaussians (DoG): These are computational models used to represent the receptive fields of neurons in the early visual system (e.g., LGN and simple cells) for tasks like edge and line detection.
  • COSFIRE (Combination of Shifted Filter Responses) and its variants (e.g., B-COSFIRE): These models are designed to detect features by combining responses of simpler filters (like Gabor filters) in a spatially organized manner, mimicking how simple cells integrate inputs from LGN neurons. They are extended to handle curved structures and object parts.
  • Inhibition Mechanisms: The paper emphasizes the implementation of two key inhibitory phenomena:
  • Push-Pull Inhibition: Mimics the antiphase excitation and inhibition observed in simple cells, improving contour detection in noisy or textured scenes.
  • Surround Suppression: Models the phenomenon where a neuron's response is suppressed by stimuli in its surrounding receptive field, enhancing selectivity and robustness.
  • Convolutional Neural Networks (ConvNets): The paper discusses how the hierarchical structure of ConvNets mirrors the brain's visual hierarchy. It highlights findings where filters learned in the first layers of ConvNets resemble biologically plausible receptive fields (like Gabor filters), and how inhibitory mechanisms are being integrated into ConvNet architectures (e.g., Local Response Normalization, push-pull inhibition layers) to improve performance and stability.

Datasets

Not explicitly discussed in the paper. The paper focuses on reviewing existing algorithms and their inspiration from neuroscience, rather than presenting new experimental results on specific datasets.

Key Insights and Ideas

  • The brain as a powerful design blueprint: The visual system's evolved mechanisms for feature detection, hierarchical processing, and robust handling of noise offer a rich source of inspiration for computer vision.
  • Inhibition is key to robustness: Mechanisms like push-pull inhibition and surround suppression are not just biological curiosities; they are critical for stabilizing neural responses, improving selectivity, and making visual processing resilient to noise and texture.
  • ConvNets echo brain hierarchy: The layered, hierarchical structure of ConvNets naturally aligns with the brain's visual processing pathway, suggesting that further bio-inspiration could lead to more powerful ConvNet architectures.
  • Early visual cortex features are fundamental: Models of simple cells (e.g., Gabor filters, DoG) and their integration (COSFIRE) are effective for low-level feature extraction, forming the basis for more complex vision tasks.
  • Bridging the gap: There's a clear trend towards explicitly incorporating biological inhibitory mechanisms into artificial neural networks to achieve performance gains, particularly in challenging image conditions.

Key Results

The paper doesn't present new quantitative results but reviews the effectiveness of the discussed brain-inspired algorithms:

  • Improved Contour and Line Detection: Algorithms incorporating push-pull inhibition (like RUSTICO) have shown superior performance in detecting lines and contours in noisy and textured images compared to their non-inhibitory counterparts.
  • Enhanced Object Detection: COSFIRE-based models have been successfully applied to object detection and recognition tasks, demonstrating the ability to learn and represent parts of objects.
  • ConvNet Regularization: The integration of inhibitory mechanisms like Local Response Normalization (LRN) in ConvNets has been shown to improve recognition performance by enhancing relevant features and regularizing the network.
  • Biologically Plausible Filters: Studies have shown that filters learned by ConvNets in their early layers can resemble biologically observed receptive fields, reinforcing the link between artificial and biological vision.

Limitations and Future Work

  • Approximation of Biological Complexity: While bio-inspired models capture certain aspects of neural function, they are still simplifications of the incredibly complex biological visual system. The paper acknowledges that current models don't fully replicate all properties of neurons (e.g., response saturation, cross-orientation suppression).
  • Computational Cost: Some bio-inspired approaches, like quadratic convolutions (Volterra kernels), offer better biological approximation but come with a significant increase in computational cost and parameters, limiting their practical application to specific layers.
  • Future Directions: The paper suggests continued exploration of inhibitory mechanisms and their integration into ConvNets for improved robustness. Further research into how higher-level visual processing in the brain can inspire more sophisticated ConvNet architectures is also implied. The potential for these bio-inspired networks to perform well with limited training data is also highlighted as a promising avenue.

Summary updated

Abstract

The study of the visual system of the brain has attracted the attention and interest of many neuro-scientists, that derived computational models of some types of neuron that compose it. These findings inspired researchers in image processing and computer vision to deploy such models to solve problems of visual data processing. In this paper, we review approaches for image processing and computer vision, the design of which is based on neuro-scientific findings about the functions of some neurons in the visual cortex. Furthermore, we analyze the connection between the hierarchical organization of the visual system of the brain and the structure of Convolutional Networks (ConvNets). We pay particular attention to the mechanisms of inhibition of the responses of some neurons, which provide the visual system with improved stability to changing input stimuli, and discuss their implementation in image processing operators and in ConvNets.
BibTeX
@inproceedings{strisciuglio2019brain,
  author = {Nicola Strisciuglio},
  title = {{Brain-inspired algorithms for processing of visual data}},
  booktitle = {BrainComp},
  year = {2019},
  url = {https://openreview.net/forum?id=DEdSKAtVc4},
}
Figure