Abstract
Humans are exposed to vast amounts of visual data, with over 80% of sensory information processed through the visual system. The unique mechanisms of the human visual system (HVS) allow for fast, accurate, and efficient image processing. While modern computer vision techniques have advanced significantly in fields like medical analysis and virtual reality, a substantial gap remains between artificial systems and biological efficiency. This special section of IEEE Access explores biologically inspired image processing to bridge this gap, integrating neural biology, perception mechanisms, and mathematical modeling to create more robust and "human-like" systems.
Introduction
This editorial introduces a special section dedicated to "Biologically Inspired Image Processing Challenges and Future Directions." It synthesizes 18 peer-reviewed articles that leverage biological principles—such as neural biology, eye-movement patterns, and binocular perception—to solve complex image processing tasks. By mimicking the efficiency and accuracy of the human brain, these works advance fields ranging from medical diagnostics and 3D reconstruction to document recognition and aerial surveillance.
The Challenge of Artificial Vision
While modern computer vision has seen massive success, engineering-heavy approaches often encounter limitations when dealing with background complexity, visual inconsistency, or high computational costs. Humans process data with incredible speed and energy efficiency. The goal of this special section is to explore biological vision mechanisms to establish objective image-processing models and algorithms that are more efficient and robust.
Methodological Innovations
The 18 papers in this section present a diverse array of methodologies:
- Deep Siamese Networks: Employed for identity-aware face synthesis, using contrastive loss to maintain pose-invariance.
- Knowledge Graphs (IKG): Introduced to incorporate semantic and scene associations into image classification, helping resolve issues with background complexity.
- Pulse Coupled Neural Networks (PCNN): Applied to finger-vein image enhancement to improve recognition accuracy without the need for empirical training.
- Attention-Based Encoder-Decoders: These models mimic human eye-movement history to improve the precision of recognizing cursive and connected characters in historical documents.
- Spike-Based Calibration: A bio-inspired mechanism for stereo vision calibration in Dynamic Vision Sensors (DVS), enabling low-power operation on FPGAs.
- Multi-level Semantic Hashing: A framework for cross-modal (image-text) retrieval that captures complex semantic structures across different data forms.
Datasets and Benchmarks
To verify these bio-inspired models, researchers utilized several standard and novel benchmarks:
- ImageNet: Used to validate knowledge-graph-based classification.
- New Eye-Tracking Database: Introduced by Sun et al., this dataset contains 220 video clips designed to facilitate research in real-time video saliency prediction.
- High-Resolution Aerial Images: Used for training the ESFNet building extraction model.
- Medical Imaging: Brain CT and retinal images were utilized for specialized tasks like midsagittal plane extraction and retinal registration.
- Historical Documents: Pre-modern Japanese historical documents were used to test human-inspired document reading systems.
Key Insights: Mimicking Biological Efficiency
Several core themes emerge from this collection of research:
- Contextual Integration: Biological systems do not process objects in isolation. Integrating "context information" (Li et al.) and "knowledge graphs" (Zhang et al.) significantly improves tracking and classification in complex environments.
- Efficiency through Sparsity: The use of "spike-based" mechanisms and "separable factorized residual blocks" (ESFNet) suggests that mimicking the sparse, efficient firing of biological neurons can lead to models with much lower computational and memory footprints.
- Temporal Awareness: By storing the "history of eye movement," models can better predict future movements, similar to how humans scan text.
- Binocular Simulation: Simulating binocular interaction and depth perception (creating a "cyclopean image") leads to quality predictors that align more closely with subjective human observations than standard metrics.
Results and Future Directions
The special section highlights significant quantitative improvements across various domains. The "Parallel-KNN" method for 3D reconstruction demonstrates rapid matching of large volumes of keypoints. In medical imaging, new morphological cascade CNNs showed improved detection of kidney lesions in CT scans. Retinal image registration techniques achieved higher stability by using dual constraints to overcome distortion.
Despite these advances, the field is still in the early stages of replicating the learning mechanisms of the human brain. Future research directions include deeper integration of neural biology with mathematical modeling, developing algorithms for long-term tracking in non-static environments, and expanding efficient networks to resource-constrained edge computing applications.
Summary updated