Abstract
We describe via simulation novel optimization algorithms for a Hopfield neural network constructed using manufacturable three-terminal Silicon-Oxide-Nitride-Oxide-Silicon (SONOS) synaptic devices. We first present a computationally-light, memristor-based, highly accurate compact model for the SONOS. Using the compact model, we describe techniques of simulated annealing in Hopfield networks by exploiting imperfect problem definitions, current leakage, and the continuous tunability of the SONOS to enable transient chaotic group dynamics. We project improvements in energy consumption and latency for optimization relative to the best CPUs and GPUs by at least 4 orders of magnitude, and also exceeding the best projected memristor-based hardware; along with a 100-fold increase in error-resilient hardware size (i.e., problem size).
Introduction
Solving NP-hard optimization problems—such as resource scheduling, gene sequencing, and portfolio management—remains a significant challenge for traditional von Neumann architectures. Energy-based neural networks, specifically Hopfield networks, offer a promising alternative by mapping optimization problems to an energy minimization task. However, hardware implementations using two-terminal memristors (RRAM) face scalability issues due to high leakage and a lack of a dynamic "tuning knob" to escape local minima. This work proposes using three-terminal SONOS transistors as synapses to overcome these hurdles.
SONOS Synaptic Device and Compact Modeling
The core of the proposed architecture is the Silicon-Oxide-Nitride-Oxide-Silicon (SONOS) transistor, a standard Flash memory component. Unlike two-terminal devices, the SONOS transistor's third terminal (the gate) provides a crucial mechanism for dynamic conductance tuning.
As shown in Figure 1, a computationally-light compact model was developed to accurately capture the device's behavior, including charge trapping and current-voltage characteristics. This model enables high-fidelity large-scale simulations of Hopfield networks, demonstrating that standard Flash-type transistors possess the necessary physics for advanced neuromorphic hardware.
Optimization via Transient Chaos
The three-terminal nature of the SONOS device allows for a novel optimization strategy termed "transient chaos." By applying controlled perturbations to the gate voltages of the diagonal synaptic elements, the network can be pushed out of local energy minima.
As illustrated in Figure 5, the researchers implemented a "damped perturbation" scheme, analogous to simulated annealing. Initially, high gate voltages induce chaotic dynamics, allowing the system to explore the solution space. As the optimization cycles progress, the perturbation is damped, letting the network settle into a global optimal solution. This approach effectively turns hardware non-idealities, like current leakage and programming noise, into beneficial stochastic sources that aid the optimization process.
Key Results and Scalability
The SONOS-based Hopfield network demonstrates significant advantages over both traditional digital hardware and emerging RRAM-based neuromorphic systems:
- Energy Efficiency and Speed: The system is projected to be 4 orders of magnitude more energy-efficient and faster than state-of-the-art CPUs and GPUs, and 2x more efficient than projected RRAM hardware.
- Scalability: While RRAM arrays are limited by interconnect resistance and leakage (typically hitting a wall at 100x100 problem sizes), the high-resistance state of SONOS devices allows scaling to 1000x1000 arrays—a 100-fold increase in hardware capacity.
- Success Rate: The damped perturbation method achieves success probabilities exceeding 50% for complex Max-Cut problems, significantly outperforming static or noise-free digital approaches.
Conclusion
By leveraging the inherent physics of manufacturable SONOS transistors, this work provides a path toward scalable, high-performance neuromorphic hardware for optimization. The use of the third terminal to induce and then damp transient chaos represents a paradigm shift in how hardware non-idealities are managed—treating them as features rather than bugs. Future work will focus on physical hardware implementation and further refining the balance between diagonal and off-diagonal tuning for even better performance.
Summary updated