Abstract
Accurate segmentation of retinal blood vessels is essential for diagnosing various systemic and ocular diseases. While Neural Architecture Search (NAS) has advanced the automation of network design, it typically overlooks neuronal diversity by assuming uniform neuron types throughout a model. This paper introduces Neuron Programming (NP), a novel framework that searches for optimal neuron types at specific positions within a network. By integrating quadratic neurons into a U-Net architecture and utilizing a genetic algorithm for joint optimization, the authors achieve state-of-the-art results on multiple benchmarks. Notably, the resulting models are highly efficient, requiring up to 7x fewer parameters than existing high-performance alternatives.
Introduction
Segmenting blood vessels in retinal fundus images is a critical task for diagnosing conditions like diabetic retinopathy and glaucoma. However, it is notoriously difficult because vessels are microscopic, intertwined, and often obscured by low contrast or lesions.
While standard deep learning approaches and NAS have improved performance, they generally rely on standard "point neurons" that perform simple linear operations. The human brain, by contrast, utilizes a diverse array of neuronal types to process information of varying complexity. The authors argue that artificial neural networks should similarly leverage neuronal diversity. The core challenge addressed is the automated determination of which specific neuron type belongs at each location in a deep network without incurring prohibitive computational costs.
Methodology: Neuron Programming
The proposed Neuron Programming (NP) framework introduces two primary innovations: the use of quadratic neurons and a specialized search strategy that treats neuron types as programmable components.
Quadratic Neurons and Search Strategy
Unlike conventional neurons that use a simple inner product, quadratic neurons incorporate higher-order terms, allowing them to model complex non-linear interactions more effectively. This is particularly beneficial for capturing the intricate morphology of retinal vessels.
The search process utilizes a Genetic Algorithm to optimize the network across two dimensions:
- Architecture (Connections): Determining the "skeleton" or connectivity of the network.
- Neuronal Type (Operations): Selecting between conventional and quadratic neurons for each node.
The framework supports both a joint search (optimizing both dimensions simultaneously) and a "plug-and-play" approach. To mitigate the high cost of searching, a hypernetwork is employed to predict optimal neuronal configurations for given architectures, drastically reducing deployment time on new datasets.
The Node Library and Gene Encoding
The search space is defined by a library of pre-activation and post-activation nodes. Each node is encoded by a gene sequence that specifies its normalization type (e.g., Instance Normalization), convolution kernel size (3x3 or 5x5), and neuron type (conventional vs. quadratic).
Experimental Validation
The researchers evaluated the NP framework on four public datasets: DRIVE, STARE, CHASE_DB1, and HRF. These datasets provide a range of challenges, from pediatric subjects to various pathological changes and high-resolution imagery.
Key Results
- Superior Performance: On the DRIVE dataset, the model achieved a Sensitivity (SE) of 0.8489, outperforming standard U-Net and other complex NAS-based competitors.
- Parameter Efficiency: The model is remarkably lean, utilizing only 0.84 million parameters. In comparison, the high-performing FR-UNet uses 5.72 million parameters, representing a nearly 7x reduction in size while maintaining superior performance.
- Robustness: Qualitative analysis indicates that the model is superior at resolving topological issues, such as identifying thin vessels in low-contrast regions where other models often fail.
Discussion and Insights
A key finding is that diversity drives efficiency. By allowing the network to select complex quadratic neurons only where needed—specifically in the initial encoder layers and final decoder layers—the model gains expressiveness without becoming excessively deep or wide. Furthermore, the study identified a synergy between Instance Normalization and quadratic neurons, which helps stabilize training by preventing feature magnitude explosions.
Conclusion and Future Work
Neuron Programming represents a significant shift from searching for network skeletons to optimizing the fundamental units of computation. While the initial generation of training data for the search process is computationally intensive, the resulting "meta-architect" can deploy optimized models almost instantly. Future work will explore expanding the neuron library to include biologically inspired types like spiking neurons and applying NP to 3D medical imaging tasks such as MRI segmentation.
Summary updated