Language models deployed on phones and edge accelerators are usually compressed, and they also need updating: facts go stale and users want personal corrections. These two needs collide. The fastest compressed formats, such as 2:4 sparsity (a hardware pattern where exactly two of every four consecutive weights are nonzero, which NVIDIA tensor cores can skip for about a 2x speedup), are rigid. Any edit that writes a dense change into the weights breaks the pattern, so the model loses its speed or has to be recompressed. Editing methods have a separate problem. Hundreds or thousands of small weight changes in a row cause editing collapse: weight norms and condition numbers grow without bound, and the model's general abilities fall apart.
We propose putting every edit into the small, dense, high-precision "wrapper" matrices that surround a sparse core, and making each edit a block-diagonal orthogonal rotation of those wrappers. This keeps the sparse, possibly quantized core untouched, so hardware speed is preserved exactly. Because a rotation cannot change singular values, the edited layer's condition number cannot drift no matter how many edits accumulate. The open question is whether rotations confined to small blocks have enough capacity to store new facts. The wrapper block size therefore becomes a hardware-aware design variable that trades inference latency against how much the model can be edited.
Core Claim
ARMOR compresses each weight matrix as $\hat W = A\,(W'\odot M)\,B$. Here $W'\odot M$ is a strict 2:4 sparse core ($M$ is the binary mask), and $A, B$ are block-diagonal correctors with about 3–5% parameter overhead. ARMOR reports 1.57x measured layer speedup on Qwen 2.5-14B. MOSE edits dense models multiplicatively, $W_{\text{new}} = R\,W_{\text{old}}$ with $R^\top R = I$, which preserves norms and condition numbers. It reports a 12.08% sequential-editing gain and keeps 95.73% of general abilities.
We combine the two. The $t$-th edit replaces $A$ by $R_t A$, where $R_t$ is orthogonal and block-diagonal with the same blocks as $A$. Three consequences follow, each provable in a line.
First, $R_t A$ is still block-diagonal with the same block size, so the deployed kernel, memory layout, and latency are unchanged.
Second, the singular values of $R_t \hat W$ equal those of $\hat W$. This holds for the whole layer, not just for $A$, so the condition-number blow-up that PRUNE identifies as the driver of collapse is ruled out structurally rather than repaired after the fact.
Third, the edit never touches the sparse core, so the core can be stored in INT8 or INT4 without the requantization drift that pushed MobiEdit to keep its edited layer in FP32.
The cost is reduced capacity. Rotating within a block of size $d$ cannot change the norm of that block of the output. An edit must therefore be expressible as per-block redirections of the layer's output, rather than as an arbitrary rank-one write like ROME. This restriction could make the method fail, and the research program is built around measuring it.
Research Program
The editor. Each edit is learned, not solved in closed form. We parameterize each block of $R_t$ as the matrix exponential of a skew-symmetric matrix $S_{t,b}$, where $b$ indexes the blocks. We then optimize
$$\min_{\{S_{t,b}\}} \; \mathcal{L}_{\text{edit}}(R_t A) + \lambda\, \mathcal{L}_{\text{keep}}(R_t A), \qquad R_t = \mathrm{blockdiag}\big(e^{S_{t,1}},\dots,e^{S_{t,n}}\big).$$
$\mathcal{L}_{\text{edit}}$ is the standard ROME/MEMIT-style loss making the model output the new fact on the edit prompt and its paraphrases. $\mathcal{L}_{\text{keep}}$ penalizes changes to the layer's outputs on cached activations from generic text, and $\lambda$ weights the two. Because the target output is optimized rather than fixed, the model can satisfy the edit through any per-block redirection that achieves it downstream, which partly offsets the norm constraint.
Each block has only $d(d-1)/2$ free parameters, so an edit is small. This also suits MobiEdit's forward-only zeroth-order gradient estimator, whose variance grows with parameter count. We test both backpropagation and zeroth-order versions.
Protecting old edits and unrelated knowledge. Rotations preserve the spectrum but can still overwrite earlier edits and unrelated knowledge. O-Edit projects updates away from gradient directions of prior edits, and SPHERE projects updates away from principal weight directions. Both need eigendecompositions of full weight matrices, which the file flags as too expensive on edge accelerators. Within a $d\times d$ block, the analogous projection is a $d\times d$ eigendecomposition, which is cheap. We include it as an optional component and ablate it.
Hardware-aware choice of block size and editable layers. A larger wrapper block gives each rotation more freedom but makes the wrapper multiply slower. This trade-off sits on top of ARMOR's existing finding that larger blocks improve compression fidelity. We build a per-layer latency lookup table on the target device, following the MoGA practice of summing pre-measured operator costs. We then choose per-layer block sizes and the set of layers whose $A$ is editable, subject to a hard latency budget, in the spirit of ConNAS. The search space is small (a few block sizes times roughly 32 layers), so we expect a greedy or exhaustive constrained search to be enough. We do not claim a new search algorithm.
The scientific output of this part is the measured frontier of edit capacity against latency. This axis is new: the HW-NAS literature assumes, in the file's words, a "static topology" after deployment.
Baselines
The do-nothing reference is the ARMOR-compressed model with no edits. It fixes the ceiling for general ability (WikiText2 perplexity, GSM8K accuracy) and the floor for edit success.
The naive structured edit runs MEMIT on the compressed model, writes the dense update into $W'$, and keeps only entries inside the existing 2:4 mask. This is the obvious way to edit without losing speed, in the spirit of EAC's thresholding of updates onto anchor coordinates. EAC reports keeping over 70% of general abilities across 1,000 edits on dense GPT2-XL and LLaMA-3-8B. The naive structured edit isolates whether the gain comes from editing the wrappers rather than merely from staying sparse.
The additive wrapper edit is our key ablation. It applies an unconstrained additive update to $A$, of the form $A + \Delta A$ with the same block structure and the same optimizer and parameter budget. If rotations beat this at equal capacity, orthogonality is what matters. If not, the benefit comes only from where the edit is placed.
The strongest dense competitors are SPHERE and MOSE run on the uncompressed model. SPHERE reports keeping reasoning accuracy over 15,000 edits on LLaMA3-8B and Qwen2.5-7B, with a 16.41% editing-accuracy gain over baselines and 3.3–6.7% compute overhead. r-ROME reports a CounterFact composite score of 92.22 and locality of 80.62 after up to 5,000 edits on GPT-J. These are upper references, because they edit dense weights and give up compression entirely. We also run each of them followed by recompression with ARMOR.
For editing models compressed with sparse-plus-low-rank formats we found no reported result on Lacuna, so all compressed-model numbers will be produced here. We also include an additive edit into the low-rank component of 3BASiL-TM. That paper reports better initialization for LoRA, which makes this a natural rival home for edits.
All methods use the same edit streams, the same edited layers (or each method's own layer selection where that is part of the method), the same number of optimization steps per edit, and the same hyperparameter-tuning budget on a held-out stream of 200 edits.
Beating the baselines means three things. Our method must match or exceed dense MEMIT edit success and generalization at 1,000 edits. It must stay within a few points of the unedited compressed model on GSM8K and WikiText2 at 5,000 edits, where the additive and naive structured baselines should have degraded. And it must keep ARMOR's measured speedup unchanged.
Why not just…?
Why not just edit the dense model and recompress? This is the strongest quality competitor, and we run it. Recompressing with ARMOR's block coordinate descent after every edit is far too expensive for on-device use. More importantly, the greedy mask search reconstructs the layer on calibration data, so it can quietly erase the edit, which is low-energy on that data. The file notes that compression methods depend on static calibration and assume the task distribution does not change. We measure how much edit retention recompression costs. If recompression every $k$ edits preserves edits at acceptable cost, the honest conclusion is that our method wins on compute and latency but not on quality.
Why not just keep edits outside the weights, in a codebook like GRACE or a memory module like EREN? GRACE and EREN leave the compressed core untouched and avoid collapse by construction, so they are a real threat. Their costs differ. A codebook adds a lookup whose size grows with the number of edits, and it fires only on inputs near stored keys, which may limit generalization to rephrased questions. We include GRACE and compare it on paraphrase generalization and on latency at 5,000 edits. If GRACE matches us on both, the weight-space approach has no advantage, and we would refocus on settings where external memory is unavailable.
Why not just use PRUNE-style singular-value clamping on additive edits? PRUNE restores GSM8K reasoning that MEMIT drives to nearly 0% after 100 edits on LLaMA-2-7B. However, it requires SVDs of full matrices, and its additive edits still break the 2:4 pattern. Our additive-wrapper ablation, with PRUNE clamping applied inside blocks, tests whether cheap clamping already does what rotations do.
Why not just a uniform block size instead of a search? A uniform block size may well be enough, and we report it. The search is justified only if the edit-capacity-versus-latency frontier differs across layers. This is plausible given Balaskas et al.'s finding that the best compression granularity depends on the layer. If the frontier is flat across layers, we drop the search.
Proposed Experiments
We compress LLaMA-3-8B and Qwen2.5-7B with ARMOR, with the core optionally in INT8. For the on-device setting we also compress Qwen2.5-3B, matching MobiEdit's models.
Edits come from CounterFact streams of 100, 1,000, and 5,000 sequential edits. We report edit efficacy, paraphrase generalization, and locality, and we track general ability with WikiText2 perplexity and GSM8K accuracy every 100 edits. We also log the norm and condition number of each edited layer, to show directly that the rotation editor keeps them flat while the additive baselines let them grow.
Latency is measured on an A100 using ARMOR's kernels and, for the 3B model, on a phone NPU in MobiEdit's setup.
The first decisive check is the capacity curve: edit efficacy at 1,000 edits against block size, for block sizes inside ARMOR's 3–5% overhead range. If no such block size reaches dense-MEMIT efficacy, block-diagonal rotations are too weak, and the proposal fails as stated.
The second is the rotation-versus-additive ablation at equal parameter count over 5,000 edits. This isolates orthogonality as the active ingredient.
The third is speed: measured speedup must be unchanged, to within measurement noise, after 5,000 edits. By construction any gap would be a bug, and the check confirms the hardware claim end to end.
The fourth is the latency-constrained search versus a uniform block size, judged by edit capacity at a fixed latency budget.
References
- ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
- Multiplicative Orthogonal Sequential Editing for Language Models
- Perturbation-Restrained Sequential Model Editing
- MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs
- O-Edit: Orthogonal Subspace Editing for Language Model Sequential Editing
- Energy-Regularized Sequential Model Editing on Hyperspheres
- MoGA: Searching Beyond Mobilenetv3
- Constraint-guided Hardware-aware NAS through Gradient Modification
- Constraining Sequential Model Editing with Editing Anchor Compression
- Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
- 3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
- Robust and Scalable Model Editing for Large Language Models
- Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization