Most deployed language models do not run at full precision. They store weights as 4-bit integers on phones, laptops and inference servers. Most knowledge-editing methods, however, assume floating-point weights. One family of editing methods rewrites a single fact ("the Space Needle is in Seattle" becomes "...is in Paris") by adding a rank-one matrix, an outer product of two vectors, to one feed-forward layer. When such an edit is merged into a 4-bit weight matrix, it has to be rounded back onto the integer grid. The per-entry changes are usually far smaller than one quantization step. Rounding to the nearest level therefore erases most of the edit. Rounding randomly keeps the edit on average, but each device then holds different weights, and the noise builds up over thousands of edits.
This note proposes that the publisher and every deployed copy share a random seed and use it to make the rounding decisions. Shared-seed rounding is unbiased, so edits survive in expectation, and every replica ends up bit-identical. Because the publisher knows exactly how the fleet rounded, it can solve each new edit against the true quantized weights, and later edits automatically correct the rounding damage from earlier ones. Each edit can then be broadcast as two short vectors, a step counter and a seed, instead of a diff over millions of changed integers. The randomness never has to be sent, because both sides can regenerate it.
Core Claim
The claim has three parts, and each can be tested on its own. First, merging rank-one edits from ROME or MEMIT into a 4-bit model by round-to-nearest destroys most of the edit's effect on the edited fact. This is a measurable gap that the editing literature has not reported; I found no result on Lacuna for editing quantized models. Second, seeded stochastic rounding combined with closed-loop solving restores efficacy close to the floating-point editor across thousands of sequential edits, and it keeps every replica bit-identical. Closed-loop solving means each edit is computed against the publisher's exact copy of the fleet's current quantized weights. Third, the seed buys something a publisher-side rounding scheme cannot buy cheaply: the update can be sent at the cost of the rank-one factors rather than a sparse integer diff. Within each row, the rounding decisions can also be coupled so that their errors cancel along the edited key direction, as correlated quantization does across clients. If the third part fails, because sparse diffs turn out cheap or coupling adds nothing, the first two still stand as a practical recipe. The note says below how each would be settled.
Where This Sits Relative to Prior Work
The editing side supplies the method. Locating and Editing Factual Associations in GPT introduced ROME. It treats a middle-layer feed-forward matrix as a key–value memory and writes a new fact by adding a closed-form rank-one update. A covariance $C$ of typical keys is used to protect unrelated inputs. Mass-Editing Memory in a Transformer (MEMIT) spreads batched edits over several layers with a least-squares solve. Both assume the weights after the edit are exactly $W+\Delta W$. That is the assumption this proposal removes.
The sequential-editing literature motivates and constrains the design. Model Editing at Scale leads to Gradual and Catastrophic Forgetting shows ROME collapsing between 100 and 1,000 sequential edits. Rebuilding ROME traces part of that collapse to an implementation asymmetry and fixes it in r-ROME, but gradual degradation remains. Rounding noise is a new source of drift layered on top. Any quantized editing scheme must therefore be measured against these collapse curves, not only on single edits. CrispEdit and AlphaEdit project updates away from directions that matter for preserved knowledge. Our coupled rounding asks the same question of the rounding error: can it be pushed out of the directions that matter?
The shared-randomness side also supplies the method. How to send a real number using a single bit (and some shared randomness) shows that randomness both sides can regenerate halves worst-case variance with one shared bit. With eight shared bits it comes within 0.02% of the unlimited-randomness limit. Correlated Quantization for Distributed Mean Estimation and Optimization couples rounding across clients through a shared permutation and offset, so errors cancel when the values are summed. We reuse that construction across the entries of a weight row, where the "sum" is the dot product with the edited key, rather than across clients.
The two closest connecting works differ in kind. FedLEKE shares full-precision mediator vectors between clients so that they can reuse each other's edits, with no quantization anywhere. Just One Byte (per gradient) uses shared seeds to regenerate perturbations for zeroth-order fine-tuning, an optimizer that perturbs all weights rather than editing one localized fact. It also documents the synchronization hazard of seeded schemes across heterogeneous hardware. That hazard shapes the determinism requirements below.
Research Program
The setting has three parts. A publisher holds the same group-wise 4-bit weights as a fleet of replicas. Edits arrive as a stream. The publisher wants every replica to absorb each edit by merging it into its integers.
At step $t$, the publisher computes the dequantized weights $\hat W_t$ and runs ROME (or MEMIT for batches) against $\hat W_t$ rather than against a full-precision master copy. This closed loop matters: the term $v_* - \hat W_t k_*$ in the ROME solve already contains whatever rounding error earlier edits left on this key. The resulting factors $(a_t, b_t)$ are broadcast with the counter $t$. Each replica updates every entry with group step size $s$ as
$$ q^{t+1}_{ij} = \left\lfloor \frac{\hat w^t_{ij} + a_{t,i}\, b_{t,j}}{s} + \xi_{ij}(\text{seed}, t) \right\rfloor, $$
where $q$ is the stored integer, $\hat w^t_{ij}$ the current dequantized weight, and $\xi_{ij}$ a uniform value in $[0,1)$ from a counter-based generator keyed on the seed, the step and the entry index. This rounding is unbiased: $\mathbb{E}[s\,q^{t+1}_{ij}] = \hat w^t_{ij} + a_{t,i} b_{t,j}$.
Rank-one edits have a convenient property for reproducibility. Each entry's increment is a single floating-point multiply, which is bit-exact under IEEE arithmetic. The only cross-hardware risk is therefore the dequantize-add-divide chain, and we will pin it to a fixed order in fixed-point arithmetic. For MEMIT batches of rank $r$, the $r$ products are summed in a canonical order. We verify bit-identity by hashing weights on different GPU and CPU types, which directly tests the concern raised in Just One Byte.
The second ingredient is coupled rounding. Rather than independent $\xi_{ij}$ per entry, each row uses one shared random offset plus a shared random permutation of its entries, ordered by their contribution to $\hat W k_*$, following Correlated Quantization. Rounding decisions within a row then behave like sampling without replacement. The accumulated rounding error along the edited key stays near one step instead of growing with the square root of the number of entries.
A back-of-envelope prediction frames what we expect to see. For an entry whose fractional change is $p \approx |\delta|/s$, rounding variance is $p(1-p)s^2 \approx s|\delta|$; the approximation holds because $p$ is small. In directions that no later edit corrects, drift should therefore grow like $\sqrt{s\sum_t|\delta_t|}$. The closed loop should keep error on edited keys bounded instead. We will check both predictions empirically, and we will track weight norms and condition numbers alongside them, since Rebuilding ROME links the growth of those quantities to collapse.
Baselines
The naive reference point is round-to-nearest merging. It applies the floating-point ROME or MEMIT edit, computed against the dequantized weights, and requantizes the edited layer with the base model's quantizer. This is what a practitioner would do by default. If it already preserves edits, the problem does not exist. We will report the fraction of edit entries below half a step, alongside efficacy, so the mechanism of failure is visible.
The accuracy ceiling is the full-precision editor itself. That means sequential r-ROME and batched MEMIT running on a floating-point copy of the same model, on the same edit streams. The cited numbers for this reference are MEMIT's 96.7% efficacy and 89.7% paraphrase success on zsRE with 10,000 edits, reported in Mass-Editing Memory in a Transformer. For sequential CounterFact editing we use r-ROME's composite score of 92.22 against 89.32 for original ROME, reported in Rebuilding ROME. We rerun both ourselves on GPT-J so the comparison uses the same facts and order. Matching within a few points of our own floating-point rerun, at 4 bits, counts as success.
The strongest competitor is publisher-side careful rounding with sparse diffs. The publisher keeps a full-precision master and, after each edit, requantizes the edited rows with covariance-weighted error compensation in the style of GPTQ, using the same $C$ that ROME uses. It then ships only the changed integers. This scheme is reproducible, deterministic and possibly more accurate than any random rounding. Its costs are publisher-side memory and diff size. It gets the same calibration data and the same per-edit compute budget on the publisher. We beat it if seeded rounding matches its efficacy and preservation within noise while sending at least an order of magnitude fewer bytes per edit. If it beats us on accuracy, we will report that and present the seeded scheme as a communication–accuracy trade-off, not a dominant method.
Three ablations isolate the ingredients. Independent private stochastic rounding uses the same unbiased rounding but different randomness on each replica. It separates unbiasedness from replica agreement and is the only baseline that cannot support the closed loop. Seeded rounding without the closed loop, solving against a full-precision master instead of $\hat W_t$, isolates error correction. Seeded rounding without within-row coupling isolates the correlated-quantization ingredient.
Why not just…?
Why not just keep the edits unmerged as a floating-point low-rank side matrix? For a few hundred edits this is the right answer, and we will say so. The factors take $T(d_{in}+d_{out})$ numbers against $d_{in}d_{out}$ for the matrix. For a 4096×16384 layer the side matrix outgrows the layer itself at roughly 3,300 edits. It also adds an extra matrix multiply at inference. A floating-point master copy instead costs about four times the 4-bit weights. We include the unmerged adapter as an accuracy reference and plot memory and latency against edit count, so the crossover is measured rather than asserted.
Why not just round carefully on the publisher and ship the integer diff? This is the real threat to the communication half of the claim. Shared randomness wins only if the diff is large. We expect it to be large: unbiased rounding changes a fraction of entries proportional to $|\delta|/s$ across tens of millions of entries, whereas deterministic rounding changes few entries precisely because it discards the edit. The experiment measures bytes per edit under both schemes at matched efficacy. If careful-rounding diffs stay small and accurate, the seeded scheme's justification shrinks to replica-side simplicity, and we will report that outcome.
Why not just quantize at 8 bits, where steps are small enough that round-to-nearest keeps the edit? Possibly true, and it bounds the scope. We sweep 3, 4 and 8 bits. The claim is made only for the bit widths where round-to-nearest measurably fails. If it does not fail at 4 bits either, the first part of the core claim is refuted and the project stops there.
Proposed Experiments
We run the main experiments on GPT-J, the setting of MEMIT and the collapse studies, and repeat the key ones on LLaMA-3-8B, the setting of CrispEdit. Base weights use group-wise 4-bit quantization. Edit streams come from CounterFact and zsRE: sequential single edits up to 10,000, and MEMIT batches of 100 to 10,000. We report efficacy, paraphrase and neighborhood scores as defined by COUNTERFACT, plus retention of edits made 1,000 and 5,000 steps earlier. General capability is measured with GLUE, following the collapse studies, and with MMLU and GSM8K. For reference, CrispEdit reports under 1% average degradation in floating point. Systems metrics are bytes per edit, bit-identity across heterogeneous hardware, merge time per edit, and growth of weight norms and condition numbers.
The first decisive check runs at single edits. Does round-to-nearest merging lose a large share of ROME's efficacy at 4 bits, and does seeded stochastic rounding restore it? If round-to-nearest already works, the project ends there. The second check runs over the 10,000-edit stream. Does the closed loop keep both old-edit retention and capability within a few points of the floating-point r-ROME and MEMIT reruns, while open-loop seeded rounding and private stochastic rounding drift? Drift is expected to follow the square-root prediction above. The third check plots communication against accuracy: seeded factors plus counter against publisher-side diffs at matched efficacy, and coupled against independent seeded rounding measured by the error in $\hat W k_*$ and by neighborhood score. Together these separate what unbiasedness, the closed loop, the shared seed and the coupling each contribute.
References
- Locating and Editing Factual Associations in GPT
- Mass-Editing Memory in a Transformer
- Model Editing at Scale leads to Gradual and Catastrophic Forgetting
- Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
- CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
- How to Send a Real Number Using a Single Bit (And Some Shared Randomness)
- Correlated Quantization for Distributed Mean Estimation and Optimization
- FedLEKE: Federated Locate-then-Edit Knowledge Editing for Multi-Client Collaboration
- Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness