Shrinking resistive memory cells below about 30 nm packs far more weights into a chip, but each cell gets worse in a specific way. It can hold only a few stable resistance levels, and the act of writing a level becomes a gamble, because moving a handful of atoms can make the resistance jump by a large relative amount. Reading the cell back, however, stays comparatively cheap and accurate. Language models compressed to 2 bits per weight need only four levels per weight, so they look like a natural fit for these dense, few-level cells. They are also exactly the models that sit on a "performance cliff", where small extra errors destroy reasoning before they visibly hurt fluency.
This note proposes to treat the stochastic write as just another rounding error and to let the model absorb it. Modern 2-bit quantizers such as GPTQ/OPTQ and QuIP's LDLQ round a weight matrix one column at a time. Each time they round a column, they adjust the columns not yet rounded to cancel that column's error on the layer's output. We propose running that loop on the physical array. The method programs a column, reads back the conductances the device actually reached, and feeds the realized error, rather than the intended rounding error, into the update of the columns still unprogrammed. A cell that misses its target is then compensated by its neighbors in the output computation, instead of being rewritten again and again until it hits the target.
Core Claim
In the nanoscale regime the dominant weight error is programming stochasticity rather than rounding. Closed-loop error-feedback programming should therefore recover most of the accuracy lost between an ideal 2-bit model and the same model naively written into scaled cells. It should do this at a lower write budget than per-cell write-verify and with no retraining. The claim has a built-in sanity check. When programming noise goes to zero, the method reduces exactly to QuIP: 2-Bit Quantization of Large Language Models With Guarantees, so any gain must come from compensating realized device error.
The same quadratic objective QuIP uses also tells us which levels to choose before programming. Let $W$ be a layer's weights, $\hat W$ the chosen device levels, $N$ the zero-mean programming error with level-dependent variance $\sigma^2(\hat w)$ (independent across cells), and $H=\mathbb{E}[xx^\top]$ the input second-moment matrix. The expected output error is then
$$ \mathbb{E}\,\|(\hat W + N - W)x\|^2 = \operatorname{tr}\big((\hat W - W)H(\hat W - W)^\top\big) + \sum_{i,j} \sigma^2(\hat w_{ij})\,H_{jj}. $$
The second term is separable per entry. Inside LDLQ's sequential procedure, the rounding step for each entry therefore becomes a noise-penalized nearest-level choice: pick the level $q$ minimizing $(q-t)^2+\sigma^2(q)$, where $t$ is the feedback-corrected target. This step steers weights away from unreliable levels. In scaled cells these are plausibly the high-conductance states, given the 6-fold set-voltage increase and hindered filament formation reported in Ion Crowding Effect in Unilaterally Downsized Perovskite Memristors. The read-back feedback then handles whatever error remains.
Why this intersection, and what is new
The device literature motivates the error model. Ion Crowding Effect in Unilaterally Downsized Perovskite Memristors shrinks perovskite memristor electrodes from 3 µm to about 30 nm and finds ion crowding at electrode edges, a more than 2× local field enhancement, and a 6-fold rise in set voltage. Scaled-down ionic liquid-functionalized geopolymer memristors shows that downscaling can even invert the hysteresis direction. Oxygen migration during resistance switching and failure of hafnium oxide memristors ties smaller volumes to accelerated filament failure, which means extra write cycles cost endurance that scaled devices cannot spare. Sub-10 nm Ta Channel Responsible for Superior Performance of a HfO2 Memristor shows that conduction concentrates in a few-nanometer channel, which supports the picture of a small, discrete level set. Together these papers argue for a non-uniform, state-dependent, heavy-tailed write error, and against the spatially uniform Gaussian noise assumed by phenomenological simulators.
The quantization literature supplies the method and sets the bar. QuIP combines random orthogonal "incoherence" rotations, which spread outlier weights evenly across coordinates, with LDLQ error-feedback rounding. It reaches 2-bit LLaMA-2 70B where OPTQ collapses. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs shows that Hadamard rotations can be fused into weights, with cheap online transforms on activations. CRVQ: Channel-relaxed Vector Quantization for Extreme Compression of LLMs shows that protecting about 2% of salient channels carries most of the accuracy at around 1 bit. All three assume that a rounded weight is stored exactly. Our proposal removes that assumption and keeps their machinery.
The hardware literature uses the opposite philosophy. Hardware implementation of memristor-based artificial neural networks describes ex-situ programming through per-cell write-verify, which drives each cell to its own target independently. A Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays (aihwkit) makes networks robust by injecting noise during training. Neither lets one cell's realized error be cancelled by other, not-yet-written cells. In-memory hyperdimensional computing tolerates PCM variability through 10,000-dimensional redundancy, which is an approach that does not transfer to dense Transformer layers. Section 2 of the research file lists no work that applies rotation-based or error-feedback LLM quantization to analog arrays. The obvious bridge it suggests is to "rotate weights to suppress ion crowding" (its Gap 1). I set that bridge aside because ion crowding is a property of electrode geometry, not of the stored value, so the physical mechanism is doubtful. I keep rotation only as an ablation, for a reason I can defend: it decorrelates spatially structured defects from the salient input channels.
Research Program
The first component is a parametric scaled-cell model built on aihwkit's analog tile abstraction. A signed weight is a differential pair of cells. Each cell has $L\in\{2,3,4\}$ reachable levels. The program error has a standard deviation that grows with target conductance and a small probability of heavy-tailed jumps, reflecting single-atom events. Stuck cells occur at a rate that rises toward tile edges, reflecting edge field enhancement. Read noise is $\sigma_r$, and post-programming drift is modeled as a separate effect.
The research file gives no quantitative variance values for sub-30 nm cells. I therefore sweep these parameters across a grid rather than claim calibrated values. The micro-scale reference point comes from aihwkit's PCM model, which is calibrated on a 1M-device array. The output of this component is a phase map: the regions of device-parameter space where each method holds the 2-bit model off the cliff.
The second component is the algorithm. QuIP's pipeline is kept intact, including the optional incoherence rotation and LDL ordering, with three changes. The level grid is replaced by the device's reachable differential levels. The noise-penalized rounding rule above is used. The column update consumes read-back values, averaged over $k$ reads to suppress read noise. For a weight-only study, perturbing digitally stored weights through the cell model is equivalent to simulating the crossbar and much cheaper, so the method can run at 70B scale. Activations and ADC/DAC precision are held fixed at 8 bits. Activation quantization is a separate open problem and out of scope here.
The third component is accounting. Every method is charged in write pulses, read operations and cells per weight. The headline comparisons are made at matched array area and matched write energy, because shrinking cells is only worthwhile if it pays off per unit area.
Baselines
The naive reference point is open-loop programming: quantize with QuIP in software, then write each weight with a single programming attempt. This sets the floor and measures how much accuracy device stochasticity costs.
The ceiling is noise-free QuIP. On LLaMA-2 70B, QuIP reports a WikiText-2 perplexity of 6.33 at 2 bits, against 123.91 for OPTQ (QuIP). Our noise-free simulation must reproduce roughly this value before any comparison counts.
The strongest practical competitor is QuIP followed by per-cell write-verify with up to $m$ pulses per cell. This is the standard ex-situ programming flow described in Hardware implementation of memristor-based artificial neural networks. It is the right opponent because it attacks the same error with the same information, namely read-back. It will receive the same total write-pulse and read budget as our method, with $m$ tuned on C4 calibration data under the same tuning budget. Beating it means lower WikiText-2 and C4 perplexity, and higher zero-shot accuracy, at equal pulses. Alternatively, it means equal accuracy at a meaningfully smaller pulse count, a smaller fraction of the ideal-versus-open-loop gap left open, and fewer endurance-consuming writes.
A second strong competitor is saliency-aware placement in the spirit of CRVQ. The roughly 2% most Hessian-sensitive channels are mapped to redundant cell pairs or to edge-free tile interiors, and the rest are programmed open-loop. It is matched on total cell count.
Ablations isolate each ingredient. The first removes read-back feedback but keeps the noise-penalized rounding, so the gain from feedback can be read off. The second keeps feedback but removes the noise penalty. The third removes the incoherence rotation, which tests whether rotation helps against edge-clustered defects or instead hurts by flattening the Hessian diagonal that the noise penalty and placement rely on.
Why not just…?
Why not just use more write-verify iterations per cell? This is the most serious threat. It is the default in hardware. If programming error is mild and unbiased, enough iterations will match our accuracy. It has two problems in the scaled regime. Each extra pulse spends endurance in devices whose failure is accelerated by small volumes (Oxygen migration…). And with only two to four reachable levels, or with stuck cells, no number of iterations can hit an unreachable target, whereas compensation can spread the miss across other weights. The matched-pulse comparison settles this question. If write-verify ties our method at equal pulses across the noise grid, the proposal fails.
Why not just do hardware-aware training with noise injection? aihwkit-style training can make networks robust to a noise distribution. But QAT-style training at 13B is reported to need more than 7,000 GPU hours, versus 50 for calibration-based quantization (CRVQ). Training also learns robustness to the distribution of errors, not to the specific errors one chip actually realized, which read-back observes directly. Evaluating Quantized Large Language Models further reports that simple re-normalization does not recover the likelihood surface at 1–2 bits. I will test a short noise-injected fine-tune on the 7B model at a compute budget matched to our calibration cost. The two approaches are compatible, so I also report them stacked.
Why not just put the salient channels on reliable cells? Placement à la CRVQ is cheap and may capture much of the benefit, because accuracy concentrates in about 2% of channels. It needs a defect map, though, and it does nothing for errors in the other 98% of channels, which accumulate at 2 bits. Our method needs no map and is compatible with placement. The matched-cell-count comparison, and the combination of both methods, will show whether placement alone is enough.
Why not just keep larger cells? Micro-scale cells are more reliable, but the reason to scale is density. The comparison is therefore made at matched array area. If micro cells with open-loop programming beat nano cells with closed-loop programming per unit area across the plausible parameter grid, I will report scaling as unprofitable for 2-bit LLMs.
Proposed Experiments and Decisive Checks
The models are LLaMA-2 7B, 13B and 70B. Calibration uses 128 C4 sequences, and evaluation covers WikiText-2 and C4 perplexity plus the zero-shot suite (PIQA, WinoGrande, HellaSwag, ARC-Easy/Challenge, LAMBADA). The low-bit cliff hits multi-step reasoning before surface fluency (Evaluating Quantized Large Language Models). ARC-Challenge and LAMBADA are therefore reported separately as the early-warning metrics, because perplexity alone could hide damage. There is also an approximately 1-bit arm on 13B with CRVQ's codebook mapped onto cells. Its reference is CRVQ's reported 9.81 WikiText-2 perplexity at about 1.06 bits, which tests whether feedback extends beyond scalar grids.
There are four decisive checks.
The first is the sanity check. Gains must vanish as programming noise goes to zero, where we should recover QuIP's numbers. Gains should grow monotonically with noise.
The second is the main test. At matched write pulses, closed-loop programming should close a larger fraction of the gap between open-loop and ideal than write-verify does, across a substantial region of the device-parameter grid, including the regimes with few levels and stuck cells. If it does not, the proposal fails.
The third is robustness to read noise. The advantage must survive read noise large enough to be realistic. I will find the value of $\sigma_r$ at which the advantage disappears, and the number of averaged reads needed to recover it.
The fourth is drift. Feedback corrects only errors present at programming time. I will measure how much of the gain persists as post-programming drift accumulates, which bounds where the method is useful.
A negative result on any of the first three checks would be informative in its own right. It would say that for scaled memristors the bottleneck is per-cell reachability, not compensable error, and that density gains require better devices rather than better rounding.
References
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- Ion Crowding Effect in Unilaterally Downsized Perovskite Memristors
- Scaled-down ionic liquid-functionalized geopolymer memristors
- Oxygen migration during resistance switching and failure of hafnium oxide memristors
- Sub-10 nm Ta Channel Responsible for Superior Performance of a HfO2 Memristor
- QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
- CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
- Hardware implementation of memristor-based artificial neural networks
- A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays
- In-memory hyperdimensional computing
- Learning Compression from Limited Unlabeled Data
- An empirical study of LLaMA3 quantization: from LLMs to MLLMs