Does Making Networks Stable Make Them Easier to Steal?

Two research communities are pushing ReLU networks in opposite directions without noticing each other. Verification researchers train networks so that most neurons never switch on or off inside small input neighborhoods, because "stable" neurons of this kind make proofs about the network far cheaper to compute. Model-stealing research shows the other side of the coin. Attackers clone a deployed model by querying it and fitting a copy, and the most precise attacks recover exact weights by locating the input points where individual neurons switch. A network whose neurons rarely switch computes a simpler function in each neighborhood: fewer linear pieces and fewer switching boundaries. A simpler function may need fewer queries to copy. If that holds, the verification community's best training recipes quietly raise theft risk. The architecture choice, which sets how many linear pieces a network can form, would then be a security decision as much as an accuracy decision.

This note proposes to measure that coupling directly and then make it cheap to act on. The core claim is that an architecture's susceptibility to functional extraction, at matched accuracy, is predicted by how stable its neurons are on the kind of generic inputs an attacker sends. The second claim is that a training-free score, computed on an untrained network in seconds, can rank architectures by this susceptibility well enough to support a two-objective search for accuracy and resistance to cloning at near-zero cost. Either claim could fail. The program is built so that each failure is informative.

Core Claim

The precise hypothesis is this. Across a large population of trained architectures from NAS-Bench-201, a public table of 15,625 small convolutional networks with known accuracies (NAS-Bench-201), the fraction of a trained victim's neurons that stay stable around attacker-style queries correlates positively with how much of the victim's accuracy a knockoff student recovers. This correlation must survive after controlling for victim accuracy, parameter count, and FLOPs. The same holds within a fixed architecture: training with a stability-inducing loss raises extraction recovery even when clean accuracy is unchanged.

The secondary, riskier hypothesis is that this stability can be estimated before training, from activation patterns at random initialization. In that case it can be folded into zero-cost architecture search.

Here "stable neuron" means a ReLU whose pre-activation keeps one sign for every input in a small ball around a point. "Stability on the query distribution" means we measure this around inputs drawn from the attacker's generic image pool, not around test images. That choice is the main departure from verification work, which only cares about neighborhoods of task data.

Where This Sits in Prior Work

Three zero-cost architecture search methods motivate the proposal, and the proposal reuses their tools. Neural Architecture Search without Training (NASWOT) scores an untrained network by how distinct its binary ReLU on/off codes are across a batch. TE-NAS combines a trainability measure with an explicit count of linear regions and treats more regions as better. SWAP-NAS counts activation patterns per neuron rather than per sample to get finer resolution. All three read activation patterns purely as a predictor of clean accuracy. We use the same activation statistics as candidate predictors of a different target, susceptibility to cloning, and ask whether that information is already in their scores or is genuinely new.

Two training methods supply the stability intervention. OCTOPUS shifts biases during training and prunes nearly stable neurons, reporting 14x verification speedups at no accuracy loss. Neuron Behavior Consistency (NBC) penalizes changes in pre-activation direction under perturbation and reports stable-neuron ratios above 80% on CIFAR-10. Both evaluate only verification speed and accuracy. Neither asks what stability does to theft risk, and that is the measurement we add. Empirical Bounds on Linear Regions of Deep Rectifier Networks supplies the mechanism: stable neurons do not cut the input space, so stabilization reduces the number of linear pieces.

On the extraction side, Knockoff Nets shows that querying with generic out-of-distribution images recovers 81% to 96% of a victim's performance. That is why we measure stability on generic query images. Train to Defend is the paper closest in spirit. It defends small MNIST multilayer perceptrons against exact weight recovery by pushing neuron weight vectors together so their switching boundaries overlap, raising attack time from 14 minutes to 4 hours to beyond 48 hours. Its mechanism is boundary overlap, not stability, and it fixes the architecture by hand. Our prediction runs partly against it. Stabilization removes boundaries rather than merging them. We expect this to shrink the set of weights an exact attacker must solve for, which should make that attack faster, not slower. We test this head on at the MLP scale where that attack is practical.

NASPY recovers NAS-found architectures through hardware cache side channels. It bounds the threat model. An attacker who knows the architecture can choose a student of the same shape, so we evaluate students of both matched and mismatched architecture. ABanditNAS searches for robustness to adversarial input perturbations, not extraction, and uses partial adversarial training rather than training-free scoring. To our knowledge from this file, no work measures extraction susceptibility across an architecture population or relates it to activation stability.

Research Program

The first stage is a measurement study that produces a ground-truth dataset. We sample 300 architectures from NAS-Bench-201, stratified by CIFAR-10 accuracy so that elite models are well represented. We train each as a CIFAR-10 victim with the benchmark's standard recipe, and train a subset of 100 again with NBC at two strengths and with OCTOPUS bias shaping. Each victim is then attacked with Knockoff-style extraction. The query pool is CIFAR-100 and downsampled ImageNet-16-120 images, the API returns soft posteriors or top-1 labels, and query budgets are 5k, 20k, and 50k. Students are of two kinds: a fixed ResNet-18, and a copy of the victim's own cell, which covers the side-channel-informed attacker.

Susceptibility is defined as the student's test accuracy divided by the victim's, together with prediction agreement on the CIFAR-10 test set. For each trained victim we measure local stability around 2,000 query images at several ball radii. We use interval bound propagation for a sound lower bound on the stable count and dense sampling for an upper bound, which brackets the estimation error that the verification literature warns about. We also count linear regions crossed along random line segments through query space.

The second stage tests the mechanism at the scale where exact weight recovery is feasible. We use MNIST multilayer perceptrons of the sizes used in Train to Defend, trained plainly, with OCTOPUS stable pruning, and with similarity regularization. We then run the cryptanalytic attack and record time to full extraction and the number of switching boundaries found. This tells us whether stabilization and boundary overlap push extraction risk in opposite directions.

The third stage builds the training-free predictor. At random initialization we compute stability-type statistics of each architecture on a batch of generic query images: the per-neuron sign-flip rate under small perturbations, and the NASWOT and SWAP pattern scores computed on the same batch. We fit a small ranking model to the first stage's susceptibility labels with cross-validation across architectures. If the predictor holds up, we run a two-objective search over the full space that maximizes a standard accuracy proxy and minimizes predicted susceptibility. We retrain the selected Pareto architectures and attack them for real.

Baselines

The naive reference point assumes susceptibility is set by simple size and quality. It predicts each architecture's extraction recovery from victim accuracy, parameter count, and FLOPs alone, fit by the same cross-validated ranking model on the same 300 victims. Any stability-based predictor must beat this one on held-out Spearman correlation with measured recovery. Without that, the stability story reduces to "bigger models are harder to copy." Whether a size-only rule predicts extraction susceptibility is unknown: no reported result was found on Lacuna for predicting extraction susceptibility across architectures with any method, so this baseline is computed fresh.

The strongest existing training-free scores are NASWOT, SWAP, and TE-NAS's linear-region count, computed exactly as published on the same initialization batches. They are the strongest competitors because they already summarize activation patterns. If one of them already ranks susceptibility, our proxy adds nothing. For accuracy prediction their reported strength is high. SWAP reports a Spearman correlation of 0.90 with CIFAR-100 accuracy on NAS-Bench-201 (SWAP-NAS). EZNAS reports a Kendall tau of 0.65 against NASWOT's 0.57 (EZNAS). We report the partial correlation of each score with susceptibility after removing its correlation with accuracy. Our proxy beats them if its partial correlation is higher, with bootstrap confidence intervals that do not overlap.

The upper reference is an oracle: stability measured on the trained victim. It tells us how much of the predictable signal survives the move to initialization.

The key ablation isolates where stability is measured. It swaps generic query images for CIFAR-10 test images in both the trained-victim and initialization measurements. If stability around task data predicts susceptibility as well as stability around query data, the query-distribution framing is unnecessary.

For the within-architecture intervention, the baseline is plain cross-entropy training of the same cell with the same epochs and tuning budget. NBC and OCTOPUS strengths are chosen to match clean accuracy within 0.5 points. The intervention is said to raise susceptibility if recovery increases consistently across the 100 cells.

For the search stage, the baseline is selection by accuracy proxy alone, plus random search given the same number of real trainings. The output-level defense CMIM, which reports raising ResNet-50 CIFAR-100 accuracy to 78.72% from 77.81% while defeating distillation, is applied on top of both searched and baseline architectures to test whether the architecture effect compounds with it.

Why not just…?

Why not just use output-level defenses such as label truncation, CMIM, or gradient redirection, whatever the architecture? These defenses act on what the interface returns, and they are complementary rather than competing. Knockoff Nets reports that truncating outputs only marginally slows its attack. GRAD² still adds per-query computation even though it runs 3.75x to 6.33x faster than MAD. Neither touches exact weight recovery or an attacker who learned the architecture through side channels. Our experiment layers CMIM over the selected architectures. If CMIM alone erases all architecture differences in recovery, that is a real negative result, and we will report it as such.

Why not just train a standard ResNet-18 with a similarity or stability loss and skip architecture search? This answers a different question. Our second stage does exactly this within a fixed cell, and it is the cleanest test of whether stabilization raises theft risk. Architecture search becomes relevant only if susceptibility varies across architectures at matched accuracy. The first stage measures that variance directly. If the variance is small relative to the training-loss effect, we will conclude that architecture is not the right lever and drop the search stage.

Why not just reuse NASWOT or SWAP, since they already measure activation patterns? This is the most serious threat. The extraction-relevant signal may already be contained in accuracy proxies, because accurate models and complex models overlap. The partial-correlation comparison in the baselines settles it. If NASWOT's partial correlation with susceptibility matches our proxy's, the honest contribution is the finding that existing scores already encode theft risk, together with the observation that maximizing them, as TE-NAS does for linear regions, may raise or lower that risk.

Why not just predict susceptibility from model size? Larger students and victims plausibly change recovery for reasons unrelated to stability. That is why size and accuracy form the naive baseline and are controlled in every correlation we report.

Proposed Experiments and Decisive Checks

Three results decide the program. The first check is whether trained-victim stability on query images has a held-out partial Spearman correlation with Knockoff recovery that is clearly positive after controlling for accuracy and size, at both soft-label and hard-label access. If it does not, the core claim fails and the measurement study is published as a negative result about verification-friendly training.

The second check is whether NBC and OCTOPUS raise recovery at matched clean accuracy on most of the 100 cells. In the MNIST multilayer perceptrons, the same check asks whether stable pruning shortens exact-extraction time below the unprotected range of 14 minutes to 4 hours reported by Train to Defend, while similarity regularization lengthens it. Opposite signs for the two interventions would establish that "stable" and "overlapping" boundaries are distinct properties with opposite security consequences.

The third check is whether the initialization proxy retains at least half of the oracle's correlation, and whether its Pareto architectures, retrained and attacked, show lower recovery than accuracy-only selections at accuracy within one point. NAS-Bench-201's best CIFAR-10 accuracy of 94.37% serves as the reference ceiling.

The total cost is roughly 400 victim trainings plus 2,400 attacks, a few hundred GPU hours. The search itself, once the proxy exists, costs seconds per architecture.