Analog in-memory computing chips run a neural network layer by storing its weights as electrical conductances in a grid of memory cells, called a crossbar. Each input value is applied as a voltage on one row, and the currents summed down each column are the matrix-vector product, read out by an analog-to-digital converter (ADC). These chips are trained for deployment in software simulators that inject device noise into the forward pass, a method called hardware-aware training. The simulators treat that noise as fixed and independent of the input. Real crossbars behave differently. Wire resistance makes voltages sag as current flows (IR-drop), and the sag depends on how many rows are driven at once, which depends on the input. The ADC's required range is set by the largest column current any input can produce. So the part of the error that simulators model worst, and that hurts accuracy most, is controlled by how dense the activations are.
The idea is to require that at most $N$ of every $M$ consecutive rows in a crossbar tile are driven on any input. This is hardware-aligned N:M sparsity, applied to activations at run time instead of to weights. It puts a hard, input-independent ceiling on the number of rows driving each column. That ceiling bounds the worst-case column current, the worst-case IR-drop, and the range the ADC must cover. We claim that training under this constraint, together with a cheap input-dependent IR-drop term in the noise model, shrinks the gap between the fast training simulator and a high-fidelity circuit solve, and lets the ADC resolution be lowered without losing accuracy. Neither dense hardware-aware training nor unconstrained activation sparsity can do both.
Core Claim and What Is New
The mechanism is elementary. Consider a tile with $R$ rows, conductances bounded by $G_{\max}$ and input voltages bounded by $V_{\max}$. If at most $N$ of every $M$ rows are active, the column current $I_j = \sum_k G_{kj} V_k$ satisfies
$$|I_j| \le \frac{N}{M}\, R\, G_{\max} V_{\max},$$
whatever the input. Dense inputs can reach the full $R\,G_{\max}V_{\max}$. Unstructured sparsity, such as the 50–98% zeros produced by squared-ReLU in Accelerating Transformer Inference and Training with 2:4 Activation Sparsity, lowers the average current but leaves the worst case unbounded, because a rare input can still light up every row in a tile. The worst case is what sets ADC range and the tail of IR-drop error.
The new part is using a structured activation constraint as a circuit-operating-point control rather than as a FLOP-saving device. We then measure whether it makes the training-time simulator more faithful. Two lines of prior work sit on either side of this.
Crossbar-aware pruning works on weights only. Crossbar-aware neural network pruning and ReaLPrune clear whole crossbars, rows or columns so that peripheral circuits can be switched off. The closest work is Pruning for Improved ADC Efficiency in Crossbar-based Analog In-memory Accelerators, whose D.U.B. method balances the number of nonzero weights per column so that ADC bits can be removed. All three operate on static weights. None of them controls the input-dependent part of the column sum.
Structured activation sparsity works only on digital hardware. SAS: Structured Activation Sparsification and Amber Pruner impose N:M patterns on activations to use GPU sparse tensor cores. They never touch analog non-idealities.
On the training side, Hardware-aware training for large-scale and diverse deep learning inference workloads supplies the base method and the key motivating finding: output-side noise (ADC and accumulation) hurts far more than weight noise. Its experiments are fully dense, and its noise is input-independent.
Research Program
We will build on the open-source simulator from A Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays, known as aihwkit. It already models programming noise, drift, and converter quantization, and trains with a noisy forward pass while updating precise digital copies of the weights.
Activation gate. We add a gate before the digital-to-analog converters that keeps the $N$ largest-magnitude activations in each group of $M$ rows mapped to the same tile, and zeroes the rest. The groups are aligned to physical wordline groups rather than to GPU lanes. The gate is trained with a straight-through estimator.
A known training difficulty comes from SAS: under dynamic routing, each weight is updated only about $1/M$ of the time. SAS's Experience-RAdam optimizer, which rescales learning rates by update frequency, is our default fix for this. We will report whether it is still needed once hardware noise is also injected. This also answers an open question: does routing stochasticity help or hurt the flat-minimum effect of noise injection reported by Yan et al. and Krishnan et al.?
IR-drop surrogate. We add a cheap, input-dependent IR-drop term to the training noise model. It is a first-order correction in which each column's output is attenuated in proportion to the total driven current along its bitline and the wordline positions of the active rows. We fit it once, per tile size and wire resistance, against a full nodal (Kirchhoff) solve of the tile.
High-fidelity evaluator. We build the nodal solve used for fitting as a separate evaluator. It solves the resistive network of every tile exactly for each test input, with wire resistance, device variability, and ADC clipping. It is too slow to train with but feasible for evaluation on test sets.
What the evaluator is, and is not. We do not have silicon, so this evaluator is our stand-in for the chip. The "simulation-to-silicon gap" measured here is precisely the gap between the training-time model and a physically fuller circuit model. Thermal effects and layout parasitics beyond the resistive network remain out of scope. We name this limitation because it is the same kind of gap the aihwkit authors acknowledge.
Primary quantity: fidelity gap. We define
$$\Delta = \mathrm{Acc}_{\text{train-sim}} - \mathrm{Acc}_{\text{nodal}},$$
the drop in accuracy when a trained model moves from the simulator it was trained in to the nodal evaluator. Our hypothesis is that $\Delta$ shrinks under N:M activation gating, both on average and in its spread across inputs.
Secondary quantity: minimum ADC resolution. The second quantity is the smallest ADC resolution that keeps accuracy within 1% of the FP32 model. We set the ADC range from the bound above, so gating at ratio $N/M$ should free about $\log_2(M/N)$ bits of range. We test whether the accuracy cost of taking those bits is small.
Combination with weight balancing. Finally, we combine activation gating with D.U.B.-style weight-column balancing. Weight balancing caps the number of nonzero terms from the weight side and gating caps it from the input side. Where the two limits meet should set the true minimum ADC precision. We test whether their savings multiply or overlap.
Baselines
Dense hardware-aware training. The naive reference is aihwkit's hardware-aware training with dense activations and its standard input-independent noise model, as in Rasch et al. The strongest reported result of this line is five of eleven workloads reaching within 1% of FP32, and WideResNet-16 on CIFAR-10 recovering from 25% error under direct mapping to a baseline-level 20% (Hardware-aware training for large-scale and diverse deep learning inference workloads). We will retrain it with the same tile size, device preset, epoch budget and hyperparameter search as our method. We will then evaluate it on the nodal solver as well as its own simulator, since its $\Delta$ is the quantity we must beat.
D.U.B. weight pruning. The strongest existing method on the ADC question is D.U.B. It reports up to 7.13× ADC energy improvement on VGG11/CIFAR-10 (4.00× to 7.13× depending on tile size) at under 1% accuracy drop, and 1.27× on ResNet18/ImageNet (Pruning for Improved ADC Efficiency). We will reimplement it inside the same simulator on the same tile sizes. Beating it means either of two outcomes. The first is a larger ADC energy saving at matched accuracy on VGG11/CIFAR-10 and ResNet18/ImageNet from gating alone. The second, which we consider more likely, is a clear additional saving when gating is stacked on D.U.B. We will also report D.U.B.'s $\Delta$ on the nodal solver. D.U.B. was never evaluated under input-dependent IR-drop, and weight balancing alone may not control it.
Digital structured activation sparsity. For accuracy at a given sparsity ratio, the reference is SAS. On a GPU it reports 82.2% top-1 with ConvNeXt-B at 1:2 activation sparsity, against 81.5% for 1:2 weight sparsity and 80.7% for the dense model (SAS). No reported result was found on Lacuna for any activation-sparse model under analog crossbar noise. Our comparison will therefore run SAS-style gating grouped by GPU lanes rather than by wordlines, trained without hardware noise, and evaluate it on the nodal solver.
Ablations. Three ablations isolate the key ingredient. The first is unstructured activation sparsity at matched average density, using squared-ReLU as in Haziza et al.; it tests whether the worst-case bound, rather than mere sparsity, matters. The second is N:M gating without the IR-drop surrogate. The third is the IR-drop surrogate on dense activations without gating.
Why not just…?
Why not just add an input-dependent IR-drop term to dense hardware-aware training? This is the most serious alternative. The dense-plus-surrogate ablation tests it directly. We expect it to lower $\Delta$ somewhat, because the network learns to tolerate attenuation. It cannot, however, reduce the ADC range dense inputs require, and a first-order surrogate is least accurate at high currents. Those high-current cases are exactly where dense inputs spend their worst moments. If this ablation matches our method on $\Delta$ and ADC bits, the gating adds nothing and we will say so.
Why not just use the natural activation sparsity of squared-ReLU networks? Average sparsity lowers average current, but the ADC and the IR-drop tail are governed by the densest inputs. The matched-density unstructured ablation will show whether rare dense inputs dominate the error. We will do this by reporting the per-input distribution of active rows per tile and of per-input output error, not just means. If worst-case tiles under unstructured sparsity turn out to be rare enough to ignore, the structured constraint is unnecessary.
Why not just drive half the rows per cycle, in two phases? Splitting rows into two sequential phases gives the same current bound with dense activations and no accuracy risk. The cost is twice the conversions and roughly twice the latency and ADC energy per product. Gating reaches the bound in one phase and skips the zeroed rows. We will run the two-phase scheme as a baseline at iso-accuracy and compare total ADC conversions and energy. If gating's accuracy cost exceeds what the saved phase buys, two-phase execution wins.
Why not just prune weights to 2:4 and train dense-activation hardware-aware models? Static weight pruning bounds the count of nonzero weights but not the input-dependent current. SAS also reports that activation sparsity is more expressive than weight sparsity at equal FLOPs (82.2% vs 81.5% on ConvNeXt-B). We include 2:4 weight pruning with hardware-aware training as a baseline and report its $\Delta$.
Why not just quantize more aggressively? Lower-precision weights do not shrink the analog column sum the ADC must digitize. The ADC-bit sweep already treats converter precision as the quantity being traded, so quantization is covered by our metric rather than competing with it.
Proposed Experiments
Models and datasets. The vision models are VGG-8 and VGG11 on CIFAR-10, ResNet18 on CIFAR-100, and ResNet18 on ImageNet. These match the D.U.B., crossbar-pruning and aihwkit evaluations, so the numbers are directly comparable. As an extension, we will test a BERT workload from the Rasch et al. suite to check that the effect is not specific to convolutional networks.
Sweep. We vary the gating ratio (dense, 3:4, 1:2, 1:4), tile size (128 and 256 rows), and wire resistance over a range spanning mild to severe IR-drop.
Metrics. For each setting we report accuracy in the training simulator and in the nodal evaluator, and hence $\Delta$ and its spread across inputs. We also report the minimum ADC bits at 1% iso-accuracy, ADC energy using D.U.B.'s energy model, and accuracy after simulated drift. Gate overhead is reported as comparator operations per product, charged to the energy budget.
Decisive checks. Three results decide the proposal. First, $\Delta$ should fall monotonically as $N/M$ decreases, while unstructured sparsity at matched density does not show the same fall. This separates the bound from sparsity. Second, gating on top of D.U.B. should remove at least one further ADC bit at iso-accuracy on VGG11/CIFAR-10. Third, the gains should grow with wire resistance, which ties them to the IR-drop mechanism rather than to generic regularization.
If the first check fails, the central claim is wrong. If only the second holds, the contribution narrows to ADC co-design without the fidelity story.
References
- Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
- Crossbar-Aware Neural Network Pruning
- ReaLPrune: ReRAM Crossbar-aware Lottery Ticket Pruned CNNs
- Pruning for Improved ADC Efficiency in Crossbar-based Analog In-memory Accelerators
- SAS: Structured Activation Sparsification
- Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
- Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators
- A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays
- Computing-In-Memory Neural Network Accelerators for Safety-Critical Systems: Can Small Device Variations Be Disastrous?
- Exploring Model Stability of Deep Neural Networks for Reliable RRAM-Based In-Memory Acceleration