Compensating for Conductance Drift in Analog Neural Networks

37 concepts28 papers2 proposals

Analog In-Memory Computing (AIMC) offers a radical departure from traditional Von Neumann architectures. Instead of shuttling millions of weights between digital memory and a CPU or GPU, AIMC stores neural network weights directly as physical electrical conductances ($G$) within crossbar arrays of non-volatile memory devices, such as Phase-Change Memory (PCM) or Resistive RAM (RRAM). When input voltages ($V$) are applied to the rows, matrix-vector multiplication happens instantly and natively via Ohm’s Law ($I = V \cdot G$) and Kirchhoff’s Current Law, which sums the resulting currents along the columns.

However, this physical efficiency introduces a fundamental vulnerability: unlike digital bits, which are locked into discrete, error-corrected states, analog conductances are continuous and subject to the laws of thermodynamics. Over time, these physical states spontaneously shift. In PCM devices, for instance, the amorphous material undergoes structural relaxation, causing a predictable, power-law decay in conductance over time. As observed by Manuel Le Gallo et al. in their work on a 64-core mixed-signal chip (A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference), this conductance drift causes the programmed weights of a neural network to gradually decay, steadily eroding inference accuracy days, weeks, or months after deployment.

To make analog accelerators commercially viable, researchers have developed a multi-tiered suite of hardware-software co-design strategies that counteract this temporal instability.

Global Drift Compensation

The first line of defense relies on an elegant realization: while conductance drift is stochastic at the individual device level, its average behavior across an entire array is highly systematic. The collective conductance decay typically follows a power-law relationship over time:

$$G(t) = G(t_0) \cdot \left(\frac{t}{t_0}\right)^{-\nu}$$

where $\nu$ is the drift exponent. Because this decay acts as a uniform scaling factor on the output currents of a crossbar array, it can be corrected digitally without altering the analog weights themselves.

As described by Katie Spoon et al. (Toward Software-Equivalent Accuracy on Transformer-Based Deep Neural Networks With Analog Memory Devices) and Corey Lammie et al. (Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference), Global Drift Compensation (GDC) periodically reads a dedicated set of reference hardware weights to estimate the average signal loss over time. The system then calculates a digital scaling factor—the inverse of the estimated decay—and applies it to the analog-to-digital converter (ADC) outputs at the edge of the crossbar array.

In a single-author study, An Chen demonstrated the power of this technique by deploying a large-scale Transformer model (ALBERT) onto a 14nm analog inference chip (Demonstration of transformer-based ALBERT model on a 14nm analog AI inference chip). By dynamically adjusting these digital scaling factors, the system successfully neutralized logarithmic conductance drift, preserving near-software-equivalent accuracy over extended periods without the massive energy cost of rewriting the physical memory cells.

Absorbing Drift via Normalization Layers

Because conductance drift primarily manifests as a multiplicative scaling of a layer's outputs, neural networks already contain architectural components uniquely suited to absorb this change: normalization layers.

Consider a standard Batch Normalization (BatchNorm) layer, which normalizes an activation $x$ using a running mean ($\mu$) and variance ($\sigma^2$), followed by a learnable scale ($\gamma$) and shift ($\beta$):

$$\text{Y} = \gamma \cdot \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} + \beta$$

If every analog weight in the preceding crossbar array decays by a factor of $\alpha$ due to drift, the resulting activation becomes $\alpha x$. Consequently, the mean scales to $\alpha \mu$ and the standard deviation scales to $\alpha \sigma$. When plugged back into the BatchNorm equation, the scaling factor $\alpha$ naturally cancels out in the fraction.

Exploiting this mathematical property, Abhiroop Bhattacharjee et al. (Examining the Role and Limits of Batchnorm Optimization to Mitigate Diverse Hardware-noise in In-memory Computing) demonstrated that simply updating or optimizing these digital batchnorm parameters post-mapping can extend the functional lifespan of analog accelerators by several orders of magnitude. This "nearly training-less" calibration bypasses the need to reprogram the underlying analog weights, leveraging the digital periphery to absorb the physical drift of the analog core.

Hardware-Aware Training

While global scaling and batchnorm optimization correct for the average drift across an array, individual devices still deviate from the mean due to manufacturing variations. To survive this device-level stochasticity, the neural network must be trained to be inherently resilient to shifting weights.

This is achieved through Hardware-Aware Training (HAT). Using high-fidelity simulation toolkits developed by Malte J. Rasch et al. (A Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays), designers can inject statistical models of both initial programming noise and temporal conductance drift directly into the forward pass of the training loop.

[Software Weights] ---> [Inject Statistical Drift & Noise Models] ---> [Forward Pass (Simulated Analog)]
       ^                                                                              |
       |_______________________ [Standard Backpropagation] <__________________________|

By exposing the network to simulated weight decay during training, the optimization algorithm avoids sharp minima in the loss landscape. Instead, it converges on "flat minima"—parameter spaces where wide variations in weight values cause negligible changes in overall network performance. As a result, when Gokul Krishnan et al. (Exploring Model Stability of Deep Neural Networks for Reliable RRAM-Based In-Memory Acceleration) optimized models specifically for loss-landscape smoothness, the networks maintained high post-mapping accuracy even as the underlying RRAM hardware aged and device characteristics shifted.

Post-Training Calibration and Dynamic Zero Points

For scenarios where hardware-aware training is too computationally expensive or unavailable, Post-Training Calibration (PTC) provides an alternative. Corey Lammie et al. (Improving the Accuracy of Analog-Based In-Memory Computing Accelerators Post-Training) developed optimization methods that fine-tune input and conductance ranges after the model has been mapped to the physical crossbars. This periodic recalibration acts as a software patch, adjusting the boundaries of the digital-analog interface to match the current physical state of the drifting hardware.

Finally, conductance drift does not just threaten deployed models; it also complicates the process of training models directly on analog hardware. Because programming a massive crossbar array sequentially takes time, the very first cells programmed can begin to drift before the entire configuration is complete.

To resolve this temporal asymmetry during on-chip training, Malte J. Rasch et al. (Fast and robust analog in-memory deep neural network training) introduced a method that maintains a leaky moving average of recent analog states within digital memory. This digital moving average serves as a dynamic zero point. Rather than forcing the hardware to maintain a perfectly static physical reference, the system allows the reference baseline to drift alongside the devices, ensuring that the gradient updates remain accurate throughout the training lifecycle.

Key Foundational Work

Neuromorphic computing with multi-memristive synapses (2017) — contributed 1 concept elements to this direction.
On the Simulation of NBTI-Induced Performance Degradation Considering Arbitrary Temperature and Voltage Variations (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_conductance_drift_48403c7b,
  title = {conductance drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conductance-drift/rdr_48403c7b532f4666b7655ad181f336eb},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_conductance_drift_48403c7b_v1,
  title = {conductance drift},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conductance-drift/rdr_48403c7b532f4666b7655ad181f336eb/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake