Learning Dendritic-Neuron-Based Motion Detection for RGB Images: A Biomimetic Approach

Tianqi Chen, Yuki Todo, Zhiyu Qiu, Yuxiao Hua, Delai Qiu, Xugang Wang, Zheng TangBiomimetics 2024 · DOI

TL;DR

This paper introduces a novel biomimetic Artificial Visual System (AVS) that mimics the human visual pathway to detect motion direction in RGB images. By integrating principles from photoreceptor cells, bipolar/horizontal cells, ganglion cells, and the LGN, the AVS achieves high accuracy and robustness, particularly in low-data scenarios, outperforming traditional CNNs.

What Problem Does This Paper Address?

The human visual system is incredibly adept at processing complex visual information, including motion detection, across multiple color channels. Traditional deep learning models, while powerful, often struggle with the nuances of biological vision, requiring vast amounts of data and computational resources. Specifically, detecting motion direction in multi-channel (RGB) images presents a challenge due to the need to integrate information across different color spectra. Existing bio-inspired models have shown promise but often focus on single-channel (grayscale) inputs. This paper aims to bridge this gap by developing a biomimetic system that can effectively process RGB images for motion detection, offering a more biologically plausible and computationally efficient alternative to current deep learning approaches.

Methods

The core of this work is the development of a biomimetic Artificial Visual System (AVS) that emulates several stages of the biological visual processing pipeline.

  • Photoreceptor Simulation: The system begins by mimicking cone cells, which are responsible for detecting different wavelengths of light, to simulate the initial input processing for each RGB channel.
  • Retinal Processing: It then simulates bipolar and horizontal cells to process signals within each channel, detecting changes in pixel intensity and identifying motion and object boundaries through "On-Off" response mechanisms.
  • Ganglion Cell Emulation: A learnable dendritic neuron model is employed to replicate ganglion cells, which integrate these retinal signals to detect motion direction within each channel. This model leverages the dendritic structure's capacity for nonlinear signal integration.
  • LGN Integration: A nonlearnable dendritic neuron model simulates the Lateral Geniculate Nucleus (LGN), responsible for consolidating information across the RGB channels to form a final, integrated motion detection output. This stage ensures that all color channels contribute to the final decision, mirroring biological LGN function.
  • Learning Algorithm: The model is trained using a cross-entropy loss function and optimized via gradient descent, adjusting synaptic weights and biases to minimize prediction errors.

Datasets

The authors created a custom dataset for training and evaluating their AVS. This dataset consists of multi-channel (32x32x3) square images simulating object motion under various lighting and background conditions. The dataset includes eight distinct object-background combinations (e.g., dark object on light background, random object on random background) and varies object sizes. Motion is simulated by moving objects by one pixel between consecutive frames.

Key Insights and Ideas

  • Dendritic Neuron Power: The paper strongly advocates for the utility of dendritic neuron models in visual processing, highlighting their ability to perform complex, nonlinear integration of inputs, which is crucial for tasks like motion detection.
  • Biomimetic Advantage: By closely mimicking the biological visual pathway (photoreceptors, retinal cells, LGN), the AVS demonstrates superior generalization and robustness, especially in challenging scenarios where traditional CNNs falter.
  • RGB Integration is Key: The LGN simulation is critical for effectively combining information from different color channels, a step often simplified or overlooked in purely computational models. This multi-channel integration is shown to be essential for accurate motion detection.
  • Low-Data Robustness: The AVS exhibits remarkable performance even with limited training data, suggesting its potential for real-world applications where large annotated datasets are impractical or expensive to obtain.

Key Results

  • Superior Generalization: In cross-validation experiments, the AVS consistently achieved high accuracy across various object-background configurations, significantly outperforming EfN-B0, ResNet-50, and ConvNeXt-tiny, which struggled with generalization.
  • High Accuracy with Limited Data: Even with a 5:95 training-to-testing data split, the AVS maintained over 96% test accuracy, demonstrating its exceptional learning capability from minimal data.
  • Computational Efficiency: The AVS model is significantly more parameter-efficient and requires less memory than the compared CNN models, while also achieving competitive training times. For instance, it uses only 432 parameters compared to ConvNeXt-tiny's 27.8 million.

Limitations and Future Work

  • Simplified Visual Scenes: The current AVS design is primarily tested on simplified visual scenes. Its performance in more complex environments with varying lighting, occlusions, or dynamic changes needs further investigation.
  • Expanding Visual Features: Future work could extend the model to incorporate other visual features like depth perception or object segmentation to create a more comprehensive visual processing system.
  • Real-time Processing: Incorporating additional computational strategies could further enhance the model's real-time processing capabilities.
  • Environmental Adaptation: Exploring mechanisms like reinforcement learning could enable the AVS to adapt to a wider range of environmental changes, increasing its robustness and flexibility.

Summary updated

Abstract

In this study, we designed a biomimetic artificial visual system (AVS) inspired by biological visual system that can process RGB images. Our approach begins by mimicking the photoreceptor cone cells to simulate the initial input processing followed by a learnable dendritic neuron model to replicate ganglion cells that integrate outputs from bipolar and horizontal cell simulations. To handle multi-channel integration, we utilize a nonlearnable dendritic neuron model to simulate the lateral geniculate nucleus (LGN), which consolidates outputs across color channels, an essential function in biological multi-channel processing. Cross-validation experiments show that AVS demonstrates strong generalization across varied object-background configurations, achieving accuracy where traditional models like EfN-B0, ResNet50, and ConvNeXt typically fall short. Additionally, our results across different training-to-testing data ratios reveal that AVS maintains over 96% test accuracy even with limited training data, underscoring its robustness in low-data scenarios. This demonstrates the practical advantage of the AVS model in applications where large-scale annotated datasets are unavailable or expensive to curate. This AVS model not only advances biologically inspired multi-channel processing but also provides a practical framework for efficient, integrated visual processing in computational models.
BibTeX
@inproceedings{chen2024learning,
  author = {Tianqi Chen and Yuki Todo and Zhiyu Qiu and Yuxiao Hua and Delai Qiu and Xugang Wang and Zheng Tang},
  title = {{Learning Dendritic-Neuron-Based Motion Detection for RGB Images: A Biomimetic Approach}},
  booktitle = {Biomimetics},
  year = {2024},
  doi = {10.3390/biomimetics10010011},
}
Figure