Cryptographically Verifiable Rewards for Cross-Device Side-Channel Adaptation

When a neural network is trained to break a cryptographic chip by reading its power-consumption traces, it almost never survives contact with a different physical chip, even one of identical make and model. Sub-nanometer manufacturing variation gives every silicon die its own electronic fingerprint, and models latch onto the fingerprint instead of the signal. This is the cross-device portability gap: Mind the Portability showed that a profiling attack needing fewer than 10 traces on the training chip needs more than 20 times as many, or fails outright, on a second chip of the same design. Meanwhile, an unrelated line of work has been fixing a trust problem in large-language-model training: instead of letting a learned judge score outputs, Reinforcement Learning with Verifiable Rewards (RLVR) scores them with an external deterministic oracle, such as a compiler or a solver.

This proposal connects the two by a simple observation: cryptography is the most reliable verifier available. AES is a deterministic public algorithm, so any candidate key material extracted from a target device can be checked for algebraic consistency — for example, byte candidates recovered from different rounds must satisfy the AES-128 key schedule relations — with no access to the secret key and no learned judge. We propose to train the cross-device trace-adaptation step of a side-channel attack as a reinforcement-learning policy whose reward is exactly this deterministic cryptographic check. The verifier, not a discriminator and not a reconstruction loss, decides whether the policy has succeeded in stripping a target chip's device-specific distortions.

Core Claim

The claim is that cross-device trace adaptation — currently done with adversarial training (CGAN-SCA), unsupervised denoising (Noise2Clean), or brute-force data pooling (Mind the Portability) — can be reframed as a verifiable-reward RL problem, and that grounding it in cipher invariants both outperforms those approaches in the realistic one-profiling-chip setting and sidesteps their structural weaknesses. This differs from the closest existing work in two ways. Reinforcement Learning for Hyperparameter Tuning in Deep Learning-based Side-channel Analysis does use RL in side-channel analysis, but only as an external architecture searcher that trains and tests on a single board — the policy never touches a second device and the reward is an empirical metric, not a verification. Noise2Clean does attack cross-device transfer, but its objective is mean-squared reconstruction error on paired clean/noisy traces, an objective that optimizes for visual trace similarity rather than key recovery. Neither grounds the reward in the cipher's deterministic algebra, which is the essence of RLVR as practiced in Solver-Informed RL, where staged solver checks replace learned judges.

Research Program

We will build VRSA (Verifiable-Reward Side-channel Adaptation), with three components. First, the policy: a lightweight conditioning network that, per input trace, outputs the parameters of an alignment-and-filtering operator (sample reweighting, phase resampling against clock jitter, window alignment against random delays). This is deliberately shallower than a U-Net denoiser so that most of the modeling capacity stays in a leakage model pre-trained on the profiling device and stays fixed; only the adaptation is learned. Second, the verifier: given the adapted traces, the frozen leakage model produces candidate intermediate key bytes, and the verifier checks deterministic AES-128 relations — key-schedule consistency across recovered subkey bytes, and plaintext-ciphertext consistency for full-key candidates when the ciphertext is public — returning a binary verdict. Third, the training loop: the policy is optimized with policy-gradient updates against this binary reward, collected on a handful of target-device traces whose secret key remains unknown throughout. Reward is staged in the spirit of Solver-Informed RL: partial credit for producing internally consistent subkey candidates even before the full key ranks first, which addresses the sparse-gradient problem that binary verifiers create.

A second, more scientific thread runs alongside: does verifiable-reward RL itself overfit to the profiling device? Breaking Barriers found that RLVR training on one domain degrades out-of-domain performance by 3.19% on average, and larger models degrade more. The side-channel setting offers an unusually clean physical analogue: we can watch guessing entropy on a held-out chip as a function of RLVR training steps on the profiling chip, testing whether grounding in device-independent cryptographic invariants prevents the memorization of device-specific noise that plain supervised training exhibits. This is the first controlled study of RLVR transfer fragility in a physical domain rather than a text domain.

Baselines

The naive reference point is the single-device model: a CNN trained on the profiling board's traces and applied unchanged to the target board. Mind the Portability reports this needs more than 20 times the traces or fails entirely in the Different-Device setting, versus fewer than 10 traces in the Same-Device setting, so it quantifies the gap we must close. The strongest multi-device competitor is the Multiple Device Model (MDM) from the same paper, which pools traces from several clone boards during training and improved attack efficiency by more than 10x; it will be run on the same four-board Atmega328p platform with the same per-board trace budget as VRSA, but it is handicapped by design in our headline setting where only one profiling board is permitted — we report it on both settings to show what the extra hardware buys. The strongest existing single-reference cross-device method is the U-Net denoiser of Noise2Clean, which on its own multi-microprocessor platform cut the attack from 136 traces to 25 and lowered guessing entropy by up to 5x; we rerun it under identical trace budgets, and note its paired-clean-trace requirement. Gaussian noise injection, shown by Still Making Noise to act as Tikhonov regularization and to recover the key in 3 traces on the AES RD dataset, is a near-free baseline that will be tuned with the same budget. Finally, CGAN-SCA — which on DPAv4.2 recovered the key in 1 to 3 traces where black-box deep learning fails within 2,000 — is the strongest feature-level cross-device adapter, and its hyperparameter fragility (search success rate raised from 0.74% to only just over 65%) makes tuning-cost comparison directly relevant. All baselines get the same compute cap per experiment, matching the dual-GPU, multi-day budget that the RL hyperparameter paper established as acceptable in this field. Beating a baseline means reaching guessing entropy zero on the target device using strictly fewer traces, with model size reported alongside, since the same RL paper showed compact models can matter (a 5,566-parameter CNN reaching GE=0 in 906 traces on ASCAD Fixed Key). The key ablation replaces the cryptographic verifier with a reconstruction loss while keeping the identical policy, optimizer, and data — isolating the verifier as the only changed ingredient.

Why not just…?

Why not just pool traces from several clone chips (MDM)? Because the pooling is the resource the realistic attacker or evaluation lab often lacks: Mind the Portability required 50,000 traces from each of four physically acquired boards. Our headline experiment restricts to one profiling board, where MDM is inapplicable by construction, and we report the one-board-versus-many-board frontier for both methods. If MDM with three boards beats VRSA with one board even after accounting for acquisition cost, the proposal loses its practical claim, and our experiments are designed to expose exactly that.

Why not just inject Gaussian noise into training traces? Still Making Noise shows noise injection works beautifully (3 traces to key on AES RD), and it costs nothing — we treat it as a first-class baseline. But its own authors note it treats every time sample uniformly, whereas the distortions that break cross-device transfer (clock jitter, random delays) are structured and time-localized. The decisive comparison is on the Noise2Clean platform, which includes injected random delays and clock jitter alongside Gaussian noise; if uniform noise injection matches VRSA there, the structured policy is not earning its complexity.

Why not just fine-tune a U-Net denoiser on the target device (Noise2Clean)? This is the strongest existing method and a genuine threat. Its two published weaknesses are, first, the need for paired clean/noisy traces on the profiling device — impossible when countermeasures cannot be bypassed — and second, that its objective is reconstruction similarity rather than key recovery. Reinforcement Learning for Hyperparameter Tuning already established in this domain that cross-entropy-style validation losses fail to correlate with guessing entropy, and reconstruction loss has the same defect. We will run Noise2Clean wherever its paired data can be constructed (fair fight) and wherever it cannot (VRSA-only setting), and report both.

Why not just fine-tune the last layers with cross-entropy on a few labeled target traces? Supervised fine-tuning requires known intermediate values on the target device, which requires the target key; Solver-Informed RL shows SFT mimics surface patterns and fails under structural shifts, and the cross-entropy/objective-mismatch evidence above applies unchanged. VRSA's verifier needs only public plaintexts and ciphertexts plus the cipher's public algebra, so it operates in a strictly weaker threat model that SFT cannot enter.

Proposed Experiments

The primary testbed is the four-board Atmega328p platform of Mind the Portability: VRSA is trained with verifier feedback only on board $B_1$, then attacks $B_2$ through $B_4$ under the Different-Device, Different-Key protocol, with guessing entropy and traces-to-$\text{GE}=0$ as the metrics. Success is beating MDM's realistic-setting efficiency in the one-board setting, with the MDM number (more than 10x improvement over single-device models) as the reference. The second testbed is the multi-microprocessor platform of Noise2Clean across ARM Cortex-M0/M3/M4 and AVR ATXMEGA chips with injected countermeasures, where the target to beat is its reported 25-trace attack (down from 136) without using paired clean traces, and where the countermeasures make the verifier intermittently wrong. That intermittency is a feature: we will measure the verifier's false-negative rate $\rho_1$ directly on profiling-chip data with a known key, then apply the forward-correction reweighting of PGFC, which raised Qwen2.5-7B's Math500 Pass@1 from 41.1% to 43.1% under noisy rule-based verification — the physical-signal counterpart to its text-only evaluation. Third, we study RLVR transfer fragility: train the policy on $B_1$ with verifier rewards and log GE on $B_2$ at every step, comparing the overfitting curve against the same model trained with cross-entropy, directly testing whether Breaking Barriers-style domain collapse appears in a physical domain and whether invariant grounding suppresses it. Finally, the verifier ablation (reconstruction loss in place of cryptographic checks, everything else fixed) and a countermeasure-ablation on ASCAD and CHES CTF 2018, where the RL-NAS paper provides the 906-trace, 5,566-parameter reference point on ASCAD Fixed Key. No reported result was found on Lacuna for cross-device transfer numbers on CHES CTF 2018 or ESHARD-AES128, so those datasets serve as secondary robustness checks rather than headline comparisons.

The decisive outcome is simple: if a policy trained against pure algebra, on one chip, with no clean paired data and no knowledge of the target key, reaches $\text{GE}=0$ on a second chip in fewer traces than every baseline under matched budgets, then verifiable rewards are the right objective for physical-domain adaptation — and RLVR gains a new, fully verifiable application far from text.