To build efficient neuromorphic hardware or compute-in-memory architectures, engineers must solve a fundamental physical challenge: how do you store a neural network's weights directly inside the processing elements?
In traditional digital architectures, memory and computation are separated. In analog neural networks, computation happens natively within the memory array itself by exploiting physics—such as using Ohm's law ($I = V \cdot G$) to multiply an input voltage by a synaptic conductance, and Kirchhoff's current law to sum the results along a wire. However, this elegant paradigm breaks down if the physical medium used to store the conductance ($G$) cannot hold its state reliably.
The evolution of analog weight storage is a journey from highly volatile, "leaky" capacitive storage to stable, non-volatile solid-state mechanisms that mimic biological synapses without consuming continuous power.
The Leaky Bucket Problem of Capacitive Storage
Early analog neural network chips relied on dynamic capacitors to store synaptic weights as electrical charges or voltages. Capacitors are highly compact and easy to integrate into standard silicon processes, making them an attractive choice for dense parallel hardware.
However, as Mackie et al. (1987) and Brownlow et al. (1989) observed, capacitive storage is inherently volatile. A capacitor acts like a leaky bucket: the stored charge continuously escapes due to subthreshold transistor conduction, junction leakage, and thermal dark currents (Neugebauer et al. (1991); Genov et al. (2001)). As the charge leaks, the weight values drift, rapidly degrading the computational accuracy of the network.
Capacitive Storage (Volatile) Floating-Gate Storage (Non-Volatile)
| | <- Charge Leaks Out ========================= <- Control Gate
--|--|-- ------------------------- <- Oxide Insulator
| | [][][ Trapped Charge [][ ] <- Floating Gate
|__| ------------------------- <- Oxide Insulator
Capacitor ========================= <- Silicon Channel
(Drifts over time) (Retains charge for years)
To counteract this decay, early hardware designers had to implement complex mitigation strategies:
- External Digital Refresh Loops: Hamilton et al. (1988) designed systems that periodically read the analog weights, digitized them, and reloaded them via digital-to-analog converters (DACs). This constant maintenance loop introduces massive system complexity and power overhead, defeating the low-power purpose of analog computing.
- Differential Storage: Schwartz et al. (1988) stored weights as the relative difference between two leaking capacitors ($V_+ - V_-$). While the relative difference remains stable over short training windows because both sides leak at similar rates, the absolute charge eventually vanishes, limiting long-term autonomy.
- Bistable Driving Circuits: Giacomo Indiveri proposed using comparator-based circuits that actively pushed a decaying capacitor voltage toward one of two stable voltage rails. While this successfully prevents memory loss, it collapses the continuous analog weight space into a binary state, sacrificing the high-resolution capacity required for complex learning.
Trapping Charge Permanently with Floating Gates
To achieve true non-volatility without sacrificing continuous analog resolution, researchers turned to floating-gate (FG) MOS transistors.
A floating-gate transistor modifies a standard field-effect transistor by embedding an additional polysilicon gate inside the silicon dioxide ($SiO_2$) gate insulator layer. Because this floating gate is entirely surrounded by high-quality, non-conductive oxide, any electrical charge placed on it becomes trapped in a potential well. Without an external force, the electrons cannot escape, allowing the device to retain its programmed state for years without active power (Hsu et al. (2000)).
The trapped charge creates an electric field that modulates the threshold voltage of the transistor. As shown by Hasler et al. (1994), this single device can simultaneously store a high-resolution synaptic weight (as trapped charge) and compute the product of an input signal and that weight (via its channel current).
The Physics of Bidirectional Updates
To serve as an adaptive synapse, a floating gate must be able to increase or decrease its stored charge smoothly during learning. The authors achieve this bidirectional tuning by balancing two distinct quantum-mechanical and physical phenomena (Shon et al. (2001); Basu et al. (2015)):
- Fowler-Nordheim Tunneling: By applying a high voltage across the oxide layer, the energy barrier narrows enough for electrons to quantum-tunnel off the floating gate. This reduces the negative charge, effectively increasing the weight or conductance.
- Hot-Electron Injection: By creating a high electric field within the transistor channel, electrons traveling through the channel gain significant kinetic energy. A fraction of these "hot" electrons acquire enough energy to surmount the $SiO_2$ energy barrier and inject themselves onto the floating gate, increasing the negative charge and decreasing the conductance.
By precisely controlling the duration and amplitude of these voltage pulses, floating-gate synapses achieve continuous, non-volatile analog weight adjustments with resolutions exceeding 10 bits (Basu et al. (2015)), completely eliminating the need for external refresh circuitry (Hsu et al. (2002)).
Alternative Solid-State Mechanisms
While floating gates rely on trapping free electrons in an insulated well, other modern research directions achieve non-volatile analog storage by altering the intrinsic material properties of the device itself.
Ferroelectric Polarization
Instead of moving electrons across an oxide barrier, ferroelectric field-effect transistors (FeFETs) utilize the spontaneous electrical polarization of a ferroelectric thin film within the gate stack (Jiang et al. (2022)). Applying an external voltage pulse shifts the crystalline alignment of the material, changing its internal polarization state. This state remains stable indefinitely without power, decoupling long-term weight retention from transient electrical activity or illumination history (Liu et al. (2025)).
Programmable Resistance and Memristors
Another approach replaces charge-based storage entirely with atomic-level structural changes. Holmes et al. (1994) pioneered this by sandwiching a thin layer of amorphous silicon between metal electrodes to act as a programmable resistor, avoiding capacitive drift entirely.
Modern memristive devices achieve a similar effect by controlling the physical migration of ions or oxygen vacancies through a material matrix to form conductive filaments (Gao et al. (2021)). Because the physical structure of the material is altered, the programmed resistance state is inherently non-volatile, offering a highly scalable path forward for dense, zero-standby-power neural network accelerators.
Key Foundational Work
Programmable Analog Pulse-Firing Neural Networks (1988) — contributed 3 concept elements to this direction.
Adaptive Neural Networks Using MOS Charge Storage (1988) — contributed 2 concept elements to this direction.
An Analog Self-Organizing Neural Network Chip (1988) — contributed 1 concept elements to this direction.