Is Watermark Survival a Property of the Architecture?

A model watermark is a hidden behavior that an owner trains into a network so they can later prove a deployed copy is theirs. A typical example is a small set of secret "trigger" images that the model labels in a prearranged way. These marks are notoriously easy to remove. Fine-tuning, weight regularization, pruning, and stealing a model's behavior through its API all erase them, often within minutes. Every study of this fragility, however, tests the watermark on one or two hand-picked backbones such as a ResNet. That leaves an important question unmeasured: does the choice of network wiring change how long a watermark survives, and if so, does a wiring that keeps watermarks on a small dataset also keep them on a large one?

We propose to answer this by treating watermark survival the way architecture-search researchers treat accuracy. Survival becomes a measurable quantity across a whole population of architectures, here a standard benchmark space of 15,625 small networks. We then ask three things. Is survival explained by architecture beyond simple model size? Do survival rankings carry over across datasets? And under model theft, which network decides survival: the owner's or the thief's? The answer tells owners whether picking architectures for watermark robustness is worth doing at all, and against which attacks.

Core Claim

Our hypothesis has two parts, and the second is deliberately a limit on the first. First, under white-box removal attacks, meaning fine-tuning, $L_2$ regularization followed by fine-tuning, and magnitude pruning, the watermark success rate (the fraction of trigger inputs that still produce the owner's label) varies substantially across cell architectures. This variation would persist even among architectures matched for parameter count, compute, and clean accuracy, and the ranking would be stable from CIFAR-10 to CIFAR-100 to ImageNet-16-120, much as accuracy rankings are. Second, under black-box model extraction, survival is governed mainly by the architecture of the student network the thief trains, not by the owner's architecture. If both parts hold, choosing the owner's architecture is a real defense against weight-level tampering and a non-defense against theft. That distinction does not exist in the current literature, which holds the architecture fixed throughout.

How This Relates to Prior Work

The closest work is Neural Architectural Backdoors. It uses a genetic search over the NATS-Bench cell space, guided by the condition number of the neural tangent kernel (a training-free measure of how easily a network fits certain functions). The search finds architectures that pick up backdoor behavior from clean training, and the attack success rate rises from about 60% on a ResNet-18 to over 81%. This paper motivates us, because it shows architecture alone can change how readily a network learns trigger-style shortcuts. It is also the obvious idea to avoid copying. Turning its search into a defensive "architecture as watermark" fails against the most important threat: a thief who extracts a model through its API trains a student of their own choosing, so a mark that lives in the owner's topology never reaches the copy. We do not search for marked architectures. We measure whether architecture changes the survival of standard weight- and behavior-level watermarks, and we explicitly separate the owner's architecture from the thief's.

The fragility literature supplies both the watermarks and the attacks we use. Adi et al. embed 100 abstract trigger images and report 100% detection with under 1% accuracy cost. Shafieinejad et al. erase such marks to 1–10% retention through extraction or through $L_2$ regularization plus brief fine-tuning. They explain this with the "outlier hypothesis": triggers are memorized outliers that the network can discard without hurting its main task. Xiao et al. introduce a stronger removal attack, WRK, which reshapes decision boundaries with large adversarial perturbations and resets the final layer. Against it they propose class-feature watermarks (CFW), which add a whole synthetic class tied into the clean features, and these keep at least 70.15% success under extraction plus WRK. Further work shows pruning and quantization discard fine-grained weight marks (Leroux et al.) and that merging dilutes them (Waseda et al.). All of these fix the backbone. The outlier hypothesis in particular predicts that survival should depend on how much spare capacity a network has for outliers, which is exactly the variable we must control for, not mistake for an architecture effect.

The transferability literature supplies our method of analysis. On Network Design Spaces for Visual Recognition compares whole populations of architectures using empirical distribution functions while controlling for size. NAS-Bench-201 shows accuracy rankings across its 15,625 cells are stable across three datasets. NASTransfer shows that small proxies usually transfer well, with one exception: cells searched on CIFAR-100 transfer to ImageNet worse than random ones, which the authors call the CIFAR-100 trap. Each of these measures only clean accuracy. We reuse their population statistics and ranking-transfer tests for a security quantity, and we treat the CIFAR-100 trap as a warning that robustness rankings could also break on that proxy.

The search-space literature constrains feasibility. Measuring survival means embedding a watermark, training, and running an attack for every architecture, which is far more expensive than measuring accuracy. Subnet-Aware Dynamic Supernet Training improves how well a weight-sharing supernet (one large network whose sub-networks share weights, so candidates need not be trained separately) ranks architectures, with Kendall's tau on NAS-Bench-201 rising from 0.751 to 0.814. Whether such shortcuts preserve a ranking of watermark survival is part of what we test.

Research Program

Population study. We draw about 600 cells from NAS-Bench-201, stratified by parameter count and by the number of skip connections, since skip connections are the motif implicated in both the DARTS collapse failure and EVAS's shortcuts. Each cell is trained on CIFAR-10, CIFAR-100, and ImageNet-16-120 with two embedded watermarks: the Adi-style trigger set and CFW. We use three seeds per cell on CIFAR-10 so that architecture variance can be separated from seed variance.

Attack suite. Each watermarked model faces five attacks:

  1. clean fine-tuning at an elevated learning rate;
  2. $L_2$ regularization followed by fine-tuning, as in Shafieinejad et al.;
  3. global magnitude pruning swept from 30% to 90% sparsity;
  4. WRK;
  5. black-box extraction into a student.

To keep the comparison fair, every attack runs until clean accuracy falls a fixed 1 point below the original. We then record the remaining watermark success rate. Fixing the accuracy cost removes the confound that a more accurate model simply tolerates harsher attacks.

Owner-versus-thief extraction matrix. For extraction, a subset of 40 owner cells is crossed with 40 student cells, spanning the full range of white-box survival. Using a two-way variance decomposition, we estimate how much extraction survival is explained by the owner's architecture, the student's architecture, and their interaction.

Cheap estimators. We test whether survival rankings can be estimated cheaply enough to search the space. Three candidates are compared against full-schedule rankings using Kendall's tau:

  1. short training schedules, 10% of the full budget;
  2. survival measured on sub-networks inherited from a supernet trained with complexity-aware learning rates and momentum separation;
  3. the tangent-kernel condition number used by EVAS.

Search, only if warranted. If architecture matters and the rankings transfer, we run a search that maximizes accuracy plus white-box survival on CIFAR-10. We then test the selected cells on ImageNet-16-120.

Baselines

Do nothing: a standard backbone. This is the reference point. A watermarked ResNet-18 sized to match the cells, with no architectural choice, run through the same five attacks at the same 1-point accuracy-drop stopping rule. Any architecture effect must beat this to matter in practice.

The strongest existing method: CFW on a fixed backbone. CFW is the strongest published defense against extraction. Xiao et al. report a watermark success rate of at least 70.15% under extraction plus WRK on CIFAR-10. We rerun CFW with its published losses on both the ResNet backbone and our cells, using identical data, attack budgets, and hyperparameter tuning trials. Beating it means a selected cell carrying CFW raises survival under white-box attacks by a margin larger than the seed-to-seed spread, at matched accuracy and parameter count. For extraction, we expect not to beat it on the owner side. The test is whether the owner-versus-thief matrix confirms this.

Capacity-matched selection. This is the key ablation. Instead of selecting by survival, pick the cell with the most parameters, or the cell with the highest clean accuracy, from the same compute bracket. If either matches survival-based selection, the outlier hypothesis's "spare capacity" explanation wins, and there is no separate architecture effect. A zero-cost alternative, ranking cells by the EVAS tangent-kernel condition number, is run under the same budget.

Random sampling. Following NASTransfer's finding that random sampling is a strong baseline, we draw random cells with the same evaluation budget as the search. The search must beat the best random cell's survival to justify itself.

Why not just…?

Why not just use CFW on a standard architecture? This is the most serious alternative. If CFW survival barely varies across cells, architecture selection adds nothing for CFW, and we will report that plainly. There is reason to measure rather than assume. CFW has not been evaluated against pruning or $L_2$ regularization followed by fine-tuning, which Leroux et al. and Shafieinejad et al. show destroy other marks. Its feature-clustering mechanism may also depend on how the cell routes features. The variance decomposition across the 600 cells settles the question directly.

Why not just make the model bigger? The outlier hypothesis predicts that survival tracks spare capacity. If so, a larger network is the whole answer. The capacity-matched baseline and the size-controlled population comparisons test exactly this. An architecture effect counts only if it survives conditioning on parameter count, compute, and clean accuracy.

Why not just make the architecture itself the watermark, like EVAS in reverse? Under extraction, the thief picks the student architecture, so a topological mark never transfers to the stolen copy. Under white-box theft, the architecture is visible and easy to alter. The owner-versus-thief matrix quantifies the first point. If, unexpectedly, owner architecture does explain extraction survival, that would be a positive surprise reopening this direction.

Why not just sample randomly instead of searching? Random sampling may well suffice, and the random-sampling baseline is built to reveal it. We include search only if the population study shows a heavy-tailed survival distribution, where rare cells are much better than typical ones. Otherwise, random sampling is the recommendation.

Decisive Experiments

No published survival results on NAS-Bench-201 cells exist in the literature we reviewed, so every number here will be new measurement. The first check is whether architecture explains survival. We will compare variance across architectures with variance across seeds, after controlling for size and accuracy. If architecture explains less than seed noise, the program stops at a clean negative result: watermark survival is not an architectural property in this space.

The second check is whether survival rankings transfer across datasets. We will compute Kendall's tau between survival rankings on CIFAR-10, CIFAR-100, and ImageNet-16-120, and compare these against the stable accuracy rankings in NAS-Bench-201. We will specifically test whether CIFAR-100 survival rankings transfer worse, mirroring the CIFAR-100 trap.

The third check is who controls survival under theft. The owner-versus-thief extraction matrix tests the second part of our claim, that the thief's student dominates. We expect extraction survival near the 1–10% level Shafieinejad et al. report for trigger sets and near CFW's at least 70.15% for class-feature marks, with most of the variance on the student side.

Finally, we ask whether survival can be estimated cheaply. A short-schedule or supernet estimator reaching a tau around the 0.8 that supernets now achieve for accuracy would make survival-aware search affordable. Falling far below that would mean survival is too expensive to search for, whatever its scientific interest.