Key-Locked Continual Learning: Making Task Revocation Verifiable on Devices You Do Not Trust

Suppose a model that keeps learning new tasks is shipped to a hospital workstation or an edge device, and a data owner later withdraws consent for one task. The server can delete its own copy. The device can still keep the old weights indefinitely, and no classical mechanism can prove it did not. Quantum cryptography offers something classical bits cannot: a key stored as a quantum state cannot be copied, and measuring it the right way destroys it while producing a classical receipt, a "certificate of deletion", that the issuer can check. The catch is that these keys are fragile, expensive physical objects. Nobody will encode a ResNet in qubits. A practical design therefore has to put all of a task's learned knowledge behind a small key that is used only through narrow, per-input evaluations.

This note proposes such a learner. Every parameter derived from task $t$'s data is stored masked under a per-task pseudorandom-function key. Inference touches that key only to unmask the few table entries indexed by the current input, and the shared backbone never sees task data. If the key is a revocable quantum key, certified destruction of the key leaves the device holding nothing about task $t$ except a black-box transcript of the queries it made during its lease. Server-side unlearning of task $t$ is exact by construction. The open question is empirical: how much accuracy does this "evaluate only through the key" constraint cost, compared with the best continual learners that store no data?

Core Claim

Revocation of a task from an untrusted copy of a continual learner can be reduced to revocation of one short cryptographic key per task, provided two conditions hold. First, the cloneable part of the model contains no task-derived information. Second, the task-derived part is laid out so that using it requires one key evaluation per input, rather than a one-time decryption that would put plaintext weights in memory. Under these conditions, post-revocation privacy and capability are bounded by what an attacker can extract from $Q$ black-box queries, not by white-box possession of the weights. We expect a measurable but bounded accuracy cost relative to rehearsal-free methods that update a shared backbone. We expect it to grow with a quantity we call key coverage: the number of distinct key evaluations needed to reproduce the classifier on the data distribution. The goal of the project is to map that trade-off.

This differs from classical certified continual unlearning. The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning defines unlearning in a sequential stream and shows that the regularization used to prevent forgetting makes unlearning less precise. Its guarantee is statistical and lives on the server, so it says nothing about copies already deployed. Towards Certified Unlearning for Deep Neural Networks handles one-shot unlearning with a Newton step and calibrated noise, with the same limitation. On the cryptographic side, Revocable Cryptography from Learning with Errors constructs key-revocable encryption, fully homomorphic encryption and pseudorandom functions (PRFs: keyed functions whose outputs look random without the key) from lattice assumptions. It never considers learned models. To our knowledge, no prior work combines the two lines.

Research Program

The locked learner. We keep a backbone $h$ frozen after pretraining on public data disjoint from all task classes. Freezing removes the forgetting-retention dilemma at its source, because no shared parameter is ever updated with task data. Each task's classifier is a hashed multi-table sketch over backbone features. Each of $L$ hash functions $\sigma_l$ maps $h(x)$ to an $r$-bit signature. Each populated table cell stores aggregated class scores from that task's training data. Cells are stored masked, and their storage locations are also PRF-derived, so a device without the key cannot even tell which cells are populated. The class score is

$$s_c(x) = \sum_{l=1}^{L} \big[\, M_l(\sigma_l(x)) - F_{k_t}(l, \sigma_l(x)) \,\big]_c ,$$

where $M_l$ is the published masked table, $F_{k_t}$ is the PRF under task key $k_t$, and arithmetic is fixed-point modulo a large integer.

Inference needs exactly $L$ key evaluations, each at an input-dependent point. Without $k_t$ the tables are computationally indistinguishable from random. Continual learning adds new tables and never edits old ones, so there is no interference between tasks. For class-incremental evaluation, scores from all held keys are compared directly, since they are calibrated counts.

The key and its revocation. In deployment, $k_t$ is the quantum key of the revocable PRF from Revocable Cryptography from Learning with Errors, with refinements from Quantum Key-Revocable Dual-Regev Encryption, Revisited. Revocation means the device measures the key and returns a certificate. We will not build quantum hardware. All learning experiments use a classical key stub, which is legitimate because the learner touches the key only through $F_{k_t}$ calls. We will report the hardware-relevant costs: key evaluations per inference and keys per task.

We will state the cryptographic caveats plainly. The strong form of revocation relies on unproven conjectures in that paper. Security holds only for a single copy of the key. Giving the same task to several devices therefore requires separately keyed tables, or the oracle-model multi-copy constructions of Revocable Encryption, Programs, and More: The Case of Multi-Copy Security, which we treat as an extension rather than a dependency.

The analysis. Generalization requires many inputs to share table cells. Shared cells mean few distinct entries, and few entries are cheap for a leaseholder to enumerate before revocation. The signature length $r$ and the number of tables $L$ set this trade-off directly. We will measure it as accuracy versus key coverage, and as residual post-revocation capability versus query budget $Q$. We also separate two guarantees that are easily conflated. Data privacy after revocation means the transcript reveals labels only for inputs the attacker chose, so membership inference about training data degrades from white-box to black-box. Capability after revocation means an attacker who pre-labels its own unlabeled data during the lease can distill. No key-based scheme can prevent that, and we will quantify it rather than hide it.

Baselines

The naive reference point is a standard rehearsal-free continual learner shipped as ordinary weights, with revocation meaning "the device promises to delete". After revocation the attacker holds the full white-box model. Every privacy and capability metric for our method is compared against this copy.

The strongest accuracy competitor is Adam-NSCL from Training Networks in Null Space of Feature Covariance for Continual Learning. It projects updates into the null space of past tasks' feature covariance and stores no samples. It reports 73.77% accuracy with −1.6% backward transfer on 10-split CIFAR-100, and 58.28% on 25-split TinyImageNet. We rerun it on the same splits, from the same public-data pretrained initialization our frozen backbone uses, with the same number of hyperparameter trials. This isolates the cost of freezing and locking from any difference in pretraining. Beating it on accuracy is not expected. Success means staying within a pre-registered margin of five points of accuracy at matched initialization, while delivering a post-revocation guarantee that Adam-NSCL cannot offer at all.

The strongest unlearning competitor is the Hessian-based certified continual unlearning of The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning. On 30 sequential CIFAR-100 tasks with ResNet-18 it reports an empirical approximation error of about 0.009 against retraining, and utility within 1–2% of perfect retraining. The storage-free gradient variant is around 0.083. We run its 30-task protocol. Our server-side deletion error is zero by construction, so the comparison that matters is retained-task accuracy and deletion cost, especially for out-of-order deletion requests, which that paper identifies as a failure mode.

The classical key-management competitor is one expert per task with an encrypted adapter. Examples are Task-conditioned Ensemble of Expert Models for Continuous Learning and the expert routing of Expert Routing with Synthetic Data for Domain Incremental Learning, which reports 70.03% on DomainNet and 85.24% AUC on DermCL. We use the same backbone and decrypt the adapter on the device for use.

Three ablations isolate the key ingredients. A cloneable linear head on the same frozen backbone gives the accuracy ceiling of the frozen-feature restriction, and so separates "cost of freezing" from "cost of table lookup". An unmasked table has identical accuracy but leaves the attacker white-box after revocation, which shows that masking is what changes the threat model. A fine-tuned-backbone variant lets task data flow into the cloneable part and measures how much capability leaks through it.

Why not just…?

Why not just keep the model on the server and serve predictions through an interface? Revocation is then trivial: the server stops answering. This is the right design whenever the server is reachable for every query, and we concede that setting entirely. The proposal targets devices that must run offline for the length of a lease, such as air-gapped clinical systems. The per-input key evaluation is designed to approximate the server-interface threat model without a server: after revocation, the attacker holds a black-box transcript, exactly what a revoked interface customer would hold.

Why not just encrypt per-task adapters with classical keys and delete the key (crypto-shredding)? On the device, the adapter must be decrypted to run, and plaintext weights can be copied. Revocable Cryptography from Learning with Errors makes the point directly: a classical key's deletion "proof" can be faked by keeping a copy. Our expert-adapter baseline measures this. Post-revocation, the attacker's white-box membership inference on the decrypted adapter should match the naive copy.

Why not just hold the key, or the whole model, in trusted hardware? This is a real threat. If one trusts the edge device's secure hardware, the whole network can run inside it with no accuracy cost, which dominates our scheme. The quantum route matters only under a trust model that excludes hardware vendors. Our learning experiments cannot distinguish the two, and we will say so. What the experiments do establish is the accuracy price of the narrower, hardware-free interface.

Why not just use certified unlearning or differential privacy noise? These certify the server's model, not copies already shipped. They also carry approximation error that grows when deletions arrive out of order (The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning). The 30-task head-to-head will show whether exact deletion by construction buys better retained accuracy under asynchronous deletion, or whether frozen-backbone accuracy loses more than that error costs.

Proposed Experiments

The first decisive check is accuracy. We run task- and class-incremental 10-split CIFAR-100, 25-split TinyImageNet, and DermCL, sweeping $r$ and $L$ to trace accuracy against key coverage. Results are compared with Adam-NSCL at matched initialization, with the cloneable linear head on the same backbone as the ceiling. If the locked learner falls more than five points below Adam-NSCL even at high coverage, the frozen-backbone design is not viable as stated.

The second check is post-revocation privacy. Membership inference on task-$t$ training data is run in three conditions: white-box on the naive copy, black-box on transcripts of size $Q$ from the locked learner, and on a model never trained on task $t$. Success means the transcript attack is near the never-trained reference at realistic $Q$.

The third check is residual capability. An attacker holds the backbone plus $m$ labeled task-$t$ samples, with and without a lease transcript of $Q$ self-chosen queries. We plot accuracy as a function of $(Q, m)$. The gap between the "with transcript" and "without transcript" curves is the honest price of distillation during the lease.

The fourth check is the 30-task ResNet-18 CIFAR-100 unlearning protocol with in-order and out-of-order deletions. Retained accuracy and wall-clock deletion cost are compared with the Hessian and hybrid strategies, using their reported ~0.009 error and 1–2% utility gap as reference.

Throughout, we report key evaluations per inference. This is the number that decides whether a future quantum memory could serve this learner at all.