Abstract
Direct electrical stimulation of the brain via cortical visual neuroprostheses is a promising approach to restore basic sight for the visually impaired by inducing a percept of localized light called ‘phosphenes’. Apart from the challenge of condensing complex sensory information into meaningful stimulation patterns at low temporal and spatial resolution, providing safe stimulation levels to the brain is crucial. We propose an end-to-end framework to learn optimal stimulation parameters (amplitude, pulse width and frequency) within safe biological constraints. The learned stimulation parameters are passed to a biologically plausible phosphene simulator which takes into account the size, brightness, and temporal dynamics of perceived phosphenes. Our experiments on naturalistic navigation videos demonstrate that constraining stimulation parameters to safe levels not only maintains task performance in image reconstruction from phosphenes but consistently results in more meaningful phosphene vision, while providing insights into the optimal range of stimulation parameters.
Introduction
Cortical visual neuroprostheses aim to restore sight by bypassing damaged eyes and stimulating the visual cortex directly. This stimulation creates "phosphenes"—small spots of light in the user's field of view. However, two major hurdles remain: translating complex, high-resolution video into sparse, interpretable electrical pulses, and ensuring these pulses do not damage brain tissue. Most previous research has focused on image quality while ignoring critical safety limits or the complex temporal dynamics, such as phosphene fading due to habituation. This paper addresses these issues by treating stimulation as a constrained optimization problem: maximizing visual information while strictly adhering to biological safety boundaries.
Methodology
The researchers developed a three-part end-to-end pipeline designed to optimize stimulation parameters while enforcing safety constraints.
- The Encoder: A deep neural network that processes video frames to predict three key stimulation parameters for each electrode: amplitude, pulse width, and frequency. The architecture utilizes 3D CNNs to capture temporal features.
- The Phosphene Simulator: A biologically plausible, differentiable model that converts predicted electrical parameters into "Simulated Phosphene Vision" (SPV). It accounts for phosphene size, brightness, and temporal habituation.
- The Decoder: A network that attempts to reconstruct the original video from the SPV, serving as a proxy for the user's perception.
To ensure safety, a scaled sigmoid function at the end of the encoder forces predicted parameters into safe ranges (e.g., limiting amplitude to 500 $\mu$A). The system is trained to minimize a combined loss: maximizing reconstruction accuracy while ensuring phosphene patterns are sparse and interpretable.
Experimental Setup
The framework was evaluated using the KITTI Vision Benchmark Suite, which provides naturalistic, first-person navigation videos of urban environments. Videos were split into 1-second sequences (10 frames at 10Hz) to allow the model to learn temporal dynamics and habituation in realistic scenarios.
Results and Discussion
The study yielded several key insights:
- Safety and Interpretability: Enforcing safety constraints actually improved phosphene interpretability. Unconstrained models tended to "overstimulate," creating massive, blurry blobs of light that were both dangerous and difficult to interpret.
- Parameter Optimization: Amplitude was identified as the most critical factor to constrain. Furthermore, simultaneously optimizing amplitude, pulse width, and frequency outperformed models that learned only one parameter, allowing the encoder to find energy-efficient stimulation patterns.
- Model Performance: 3D CNNs generally outperformed Recurrent (LSTM) networks for the short video clips used, providing more stable and descriptive phosphene patterns.
- Quantitative Success: The constrained model successfully kept all parameters within safe micro-scale limits while achieving reconstruction performance comparable to physically impossible unconstrained baselines.
Conclusion
This work demonstrates that end-to-end learning of stimulation parameters enables the enforcement of critical biological safety constraints without sacrificing visual quality. By incorporating a realistic phosphene simulator, the framework provides a path toward safer and more effective cortical neuroprosthetic vision. Future work will focus on integrating eye movements, multi-electrode interactions, and specific hardware limitations into the training loop.
Summary updated