End-to-end learning of safe stimulation parameters for cortical neuroprosthetic vision

Burcu Küçükoğlu, Bodo Rueckauer, Jaap de Ruyter van Steveninck, Maureen van der Grinten, Yağmur Güçlütürk, Pieter R. Roelfsema, Umut Güçlü, Marcel van Gerven

Abstract

Direct electrical stimulation of the brain via cortical visual neuroprostheses is a promising approach to restore basic sight for the visually impaired by inducing a percept of localized light called ‘phosphenes’. Apart from the challenge of condensing complex sensory information into meaningful stimulation patterns at low temporal and spatial resolution, providing safe stimulation levels to the brain is crucial. We propose an end-to-end framework to learn optimal stimulation parameters (amplitude, pulse width and frequency) within safe biological constraints. The learned stimulation parameters are passed to a biologically plausible phosphene simulator which takes into account the size, brightness, and temporal dynamics of perceived phosphenes. Our experiments on naturalistic navigation videos demonstrate that constraining stimulation parameters to safe levels not only maintains task performance in image reconstruction from phosphenes but consistently results in more meaningful phosphene vision, while providing insights into the optimal range of stimulation parameters.

Introduction

Cortical visual neuroprostheses aim to restore sight by bypassing damaged eyes and stimulating the visual cortex directly. This stimulation creates "phosphenes"—small spots of light in the user's field of view. However, two major hurdles remain: translating complex, high-resolution video into sparse, interpretable electrical pulses, and ensuring these pulses do not damage brain tissue. Most previous research has focused on image quality while ignoring critical safety limits or the complex temporal dynamics, such as phosphene fading due to habituation. This paper addresses these issues by treating stimulation as a constrained optimization problem: maximizing visual information while strictly adhering to biological safety boundaries.

Methodology

The researchers developed a three-part end-to-end pipeline designed to optimize stimulation parameters while enforcing safety constraints.

Overview of the end-to-end framework including the encoder, phosphene simulator, and decoder.
  1. The Encoder: A deep neural network that processes video frames to predict three key stimulation parameters for each electrode: amplitude, pulse width, and frequency. The architecture utilizes 3D CNNs to capture temporal features.
  2. The Phosphene Simulator: A biologically plausible, differentiable model that converts predicted electrical parameters into "Simulated Phosphene Vision" (SPV). It accounts for phosphene size, brightness, and temporal habituation.
  3. The Decoder: A network that attempts to reconstruct the original video from the SPV, serving as a proxy for the user's perception.
Detailed architecture of the 3D CNN encoder and decoder used in the framework.

To ensure safety, a scaled sigmoid function at the end of the encoder forces predicted parameters into safe ranges (e.g., limiting amplitude to 500 $\mu$A). The system is trained to minimize a combined loss: maximizing reconstruction accuracy while ensuring phosphene patterns are sparse and interpretable.

Experimental Setup

The framework was evaluated using the KITTI Vision Benchmark Suite, which provides naturalistic, first-person navigation videos of urban environments. Videos were split into 1-second sequences (10 frames at 10Hz) to allow the model to learn temporal dynamics and habituation in realistic scenarios.

Results and Discussion

The study yielded several key insights:

  • Safety and Interpretability: Enforcing safety constraints actually improved phosphene interpretability. Unconstrained models tended to "overstimulate," creating massive, blurry blobs of light that were both dangerous and difficult to interpret.
  • Parameter Optimization: Amplitude was identified as the most critical factor to constrain. Furthermore, simultaneously optimizing amplitude, pulse width, and frequency outperformed models that learned only one parameter, allowing the encoder to find energy-efficient stimulation patterns.
  • Model Performance: 3D CNNs generally outperformed Recurrent (LSTM) networks for the short video clips used, providing more stable and descriptive phosphene patterns.
  • Quantitative Success: The constrained model successfully kept all parameters within safe micro-scale limits while achieving reconstruction performance comparable to physically impossible unconstrained baselines.

Conclusion

This work demonstrates that end-to-end learning of stimulation parameters enables the enforcement of critical biological safety constraints without sacrificing visual quality. By incorporating a realistic phosphene simulator, the framework provides a path toward safer and more effective cortical neuroprosthetic vision. Future work will focus on integrating eye movements, multi-electrode interactions, and specific hardware limitations into the training loop.

Summary updated

Abstract

<i>Objective.</i>Direct electrical stimulation of the brain via cortical visual neuroprostheses is a promising approach to restore basic sight for the visually impaired by inducing a percept of localized light called 'phosphenes'. Apart from the challenge of condensing complex sensory information into meaningful stimulation patterns at low temporal and spatial resolution, providing safe stimulation levels to the brain is crucial.<i>Approach.</i>We propose an end-to-end framework to learn optimal stimulation parameters (amplitude, pulse width and frequency) within safe biological constraints. The learned stimulation parameters are passed to a biologically plausible phosphene simulator which takes into account the size, brightness, and temporal dynamics of perceived phosphenes.<i>Main results.</i>Our experiments on naturalistic navigation videos demonstrate that constraining stimulation parameters to safe levels not only maintains task performance in image reconstruction from phosphenes but consistently results in more meaningful phosphene vision, while providing insights into the optimal range of stimulation parameters.<i>Significance.</i>Our study presents a stimulus-generating encoder that learns stimulation parameters (1) satisfying safety constraints, and (2) maximizing the combined objective of image reconstruction and phosphene interpretability with a highly realistic phosphene simulator accounting for temporal dynamics of stimulation. End-to-end learning of stimulation parameters this way enables enforcement of critical biological safety constraints as well as technical limits of the hardware at hand.
BibTeX
@article{kucukoglu2025end,
  author = {Burcu Küçükoğlu and Bodo Rueckauer and Jaap de Ruyter van Steveninck and Maureen van der Grinten and Yağmur Güçlütürk and Pieter R. Roelfsema and Umut Güçlü and Marcel van Gerven},
  title = {{End-to-end learning of safe stimulation parameters for cortical neuroprosthetic vision}},
  journal = {Journal of Neural Engineering},
  year = {2025},
  doi = {10.1088/1741-2552/ade918},
}
Figure