IEEE Access Special Section Editorial: Biologically Inspired Image Processing Challenges and Future Directions

Jiachen Yang, Qinggang Meng, Maurizio Murroni, Shiqi Wang, Feng ShaoIEEE Access 2020 · OpenReview

Abstract

Humans are exposed to vast amounts of visual data, with over 80% of sensory information processed through the visual system. The unique mechanisms of the human visual system (HVS) allow for fast, accurate, and efficient image processing. While modern computer vision techniques have advanced significantly in fields like medical analysis and virtual reality, a substantial gap remains between artificial systems and biological efficiency. This special section of IEEE Access explores biologically inspired image processing to bridge this gap, integrating neural biology, perception mechanisms, and mathematical modeling to create more robust and "human-like" systems.

Introduction

This editorial introduces a special section dedicated to "Biologically Inspired Image Processing Challenges and Future Directions." It synthesizes 18 peer-reviewed articles that leverage biological principles—such as neural biology, eye-movement patterns, and binocular perception—to solve complex image processing tasks. By mimicking the efficiency and accuracy of the human brain, these works advance fields ranging from medical diagnostics and 3D reconstruction to document recognition and aerial surveillance.

The Challenge of Artificial Vision

While modern computer vision has seen massive success, engineering-heavy approaches often encounter limitations when dealing with background complexity, visual inconsistency, or high computational costs. Humans process data with incredible speed and energy efficiency. The goal of this special section is to explore biological vision mechanisms to establish objective image-processing models and algorithms that are more efficient and robust.

Methodological Innovations

The 18 papers in this section present a diverse array of methodologies:

  • Deep Siamese Networks: Employed for identity-aware face synthesis, using contrastive loss to maintain pose-invariance.
  • Knowledge Graphs (IKG): Introduced to incorporate semantic and scene associations into image classification, helping resolve issues with background complexity.
  • Pulse Coupled Neural Networks (PCNN): Applied to finger-vein image enhancement to improve recognition accuracy without the need for empirical training.
  • Attention-Based Encoder-Decoders: These models mimic human eye-movement history to improve the precision of recognizing cursive and connected characters in historical documents.
  • Spike-Based Calibration: A bio-inspired mechanism for stereo vision calibration in Dynamic Vision Sensors (DVS), enabling low-power operation on FPGAs.
  • Multi-level Semantic Hashing: A framework for cross-modal (image-text) retrieval that captures complex semantic structures across different data forms.

Datasets and Benchmarks

To verify these bio-inspired models, researchers utilized several standard and novel benchmarks:

  • ImageNet: Used to validate knowledge-graph-based classification.
  • New Eye-Tracking Database: Introduced by Sun et al., this dataset contains 220 video clips designed to facilitate research in real-time video saliency prediction.
  • High-Resolution Aerial Images: Used for training the ESFNet building extraction model.
  • Medical Imaging: Brain CT and retinal images were utilized for specialized tasks like midsagittal plane extraction and retinal registration.
  • Historical Documents: Pre-modern Japanese historical documents were used to test human-inspired document reading systems.

Key Insights: Mimicking Biological Efficiency

Several core themes emerge from this collection of research:

  • Contextual Integration: Biological systems do not process objects in isolation. Integrating "context information" (Li et al.) and "knowledge graphs" (Zhang et al.) significantly improves tracking and classification in complex environments.
  • Efficiency through Sparsity: The use of "spike-based" mechanisms and "separable factorized residual blocks" (ESFNet) suggests that mimicking the sparse, efficient firing of biological neurons can lead to models with much lower computational and memory footprints.
  • Temporal Awareness: By storing the "history of eye movement," models can better predict future movements, similar to how humans scan text.
  • Binocular Simulation: Simulating binocular interaction and depth perception (creating a "cyclopean image") leads to quality predictors that align more closely with subjective human observations than standard metrics.

Results and Future Directions

The special section highlights significant quantitative improvements across various domains. The "Parallel-KNN" method for 3D reconstruction demonstrates rapid matching of large volumes of keypoints. In medical imaging, new morphological cascade CNNs showed improved detection of kidney lesions in CT scans. Retinal image registration techniques achieved higher stability by using dual constraints to overcome distortion.

Despite these advances, the field is still in the early stages of replicating the learning mechanisms of the human brain. Future research directions include deeper integration of neural biology with mathematical modeling, developing algorithms for long-term tracking in non-static environments, and expanding efficient networks to resource-constrained edge computing applications.

Summary updated

Abstract

Human kind is exposed to large amounts of data. According to statistics, more than 80% of information received by humans comes from the visual system. Therefore, image information processing is not only an important research topic but also a challenging task. The unique information processing mechanism of the human visual system provides it with fast, accurate, and efficient image processing capabilities. At present, many advanced image analysis and processing techniques have been widely used in image communication, geographic information systems, medical image analysis, and virtual reality. However, there is still a large gap between these technologies and the human visual system. Therefore, building an image system research mechanism based on the biological vision system is an attractive but difficult target. Although it is a challenge, it can also be considered as an opportunity which utilizes biologically inspired ideas. Meanwhile, through the integration of neural biology, biological perception mechanisms, and computer science and mathematical science, related research can bridge biological vision and computer vision. Finally, the biologically inspired image analysis and processing system is expected to be built on the basis of further consideration of the learning mechanism of the human brain.
BibTeX
@inproceedings{yang2020ieee,
  author = {Jiachen Yang and Qinggang Meng and Maurizio Murroni and Shiqi Wang and Feng Shao},
  title = {{IEEE Access Special Section Editorial: Biologically Inspired Image Processing Challenges and Future Directions}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=JFomDb-A23},
}
Figure