Steganalysis is the task of telling whether an image carries a hidden message. The best detectors are convolutional networks that hunt for extremely faint, high-frequency noise that embedding leaves behind. They are expensive, and the expense sits in an awkward place. To avoid washing out that noise, they run their first many layers at full image resolution, with no pooling (no averaging of neighboring pixels to shrink the image). On a phone-class processor this is what dominates latency and activation memory. Prior attempts to slim these detectors prune or factorize an already-trained SRNet, a widely used steganalysis network. They cut parameters dramatically, but parameters live mostly in the top layers, where the detector is redundant anyway. The layers that actually cost time on a device are left untouched. Architecture search could redesign the bottom of the network under a real device budget. Ordinary searches are too expensive, though, and the cheap "score-at-initialization" shortcuts used to make search affordable were built to reward capacity for recognizing objects. There is reason to believe that is the wrong thing to reward here.
We propose a training-free score that asks a direct question of an untrained candidate network: how much of the actual embedding change, the difference between a clean image and its message-carrying twin, survives to the network's final features, relative to how much ordinary image-to-image content variation survives? We build a ground-truth benchmark of a few hundred steganalysis architectures trained to convergence. On it we test whether this score ranks architectures better than generic shortcuts and better than short training runs at equal compute. We then use it inside a search with hard limits on measured latency and peak memory for an ARM CPU and a desktop GPU.
Core Claim
The architectural property that decides steganalysis accuracy is signal transmission, not capacity. By transmission we mean whether the network carries a sparse, high-frequency perturbation forward without averaging it away. CALPA-NET supplies the mechanism. It prunes SRNet channels bottom-to-top within an accuracy tolerance, and its Fourier analysis argues that wide upper layers aggregate toward low frequencies and suppress stego noise. It calls this the "Aggregation Paradox." EZNAS supplies the contrast. It evolves zero-cost scoring formulas for semantic benchmarks, and its best formula, EZNAS-A, behaves like a nonlinear parameter counter.
Put together, these suggest a falsifiable prediction. Among steganalysis architectures of matched latency, capacity-style proxies (Synflow, EZNAS-A, and to a lesser degree jacob_cov) will correlate weakly, or even negatively, with detection accuracy, because they reward exactly the top-layer width that the Aggregation Paradox penalizes. A paired cover–stego transmission score should not share this failure. If the prediction holds, it explains why existing cheap search cannot simply be pointed at this domain, and it gives a usable replacement. If it fails, the benchmark itself is the main contribution.
The score is defined as follows. Let $f$ be a randomly initialized candidate network truncated before its classifier. Let $(c_i, s_i)$ be cover/stego pairs, meaning the same image without and with an embedded payload. Let $c_j$ be a different cover from the same minibatch. The score is
$$ T(f) = \log \frac{\mathbb{E}_i \,\| f(s_i) - f(c_i) \|^2}{\mathbb{E}_{i \neq j}\, \| f(c_j) - f(c_i) \|^2}, $$
averaged over a handful of random initializations. The numerator measures how strongly the embedding change reaches the features. The denominator measures how strongly unrelated image content does. A network that pools early, or that mixes many channels into smooth summaries, should shrink the numerator much faster than the denominator. Computing $T$ costs one forward pass over a single minibatch of pairs. Stego twins are cheap to generate with standard embedders, so no trained model and no labels beyond the pairing are needed.
The closest unsupervised precedent is STD-NET. It decides how far to Tucker-decompose each layer of a trained SRNet by measuring feature distortion on clean images, and it found that redundancy is "top-heavy." That method measures sensitivity of a trained network to its own compression. We measure an untrained network's transmission of the stego signal, and use it to rank new topologies.
Research Program
The search space is built around the design choices that matter for steganalysis rather than around MobileNet-style blocks. Its dimensions are:
- Front end: a fixed high-pass filter bank of the kind used in Spatial Rich Models, or learned filters.
- Resolution stage: the number of full-resolution layers, from two to eight.
- First downsampling: where it happens, and whether by strided convolution or average pooling.
- Block type: plain or residual.
- Channel profile: constant, increasing (the inherited "inverted pyramid"), or tapering, so that the Aggregation Paradox is something the search can probe rather than assume.
- Final pooling: global mean or mean-plus-variance.
For hardware cost we follow MoGA and build per-layer latency lookup tables from on-device measurements. The targets are a Raspberry Pi-4 ARM CPU and a TITAN Xp-class GPU, the device classes HAT used. Each candidate's summed latency is spot-checked against end-to-end timing. Peak activation memory is computed analytically, since for full-resolution layers it is exact.
Search uses aging evolution with $T$ as fitness. Following the hardware-aware evolution of HaeNAS, which combined jacob_cov with latency and energy predictors for ImageNet, candidates are scored for accuracy and cost together. Unlike HaeNAS, we treat the budget as a hard constraint and discard any candidate over the latency or memory limit. Using hard limits instead of a weighted product sidesteps the weight-tuning problem that DUCCIO addressed with its max-margin penalty.
The expensive but necessary piece is the ground truth. We sample roughly 300 architectures, stratified across latency bins, and train each with one fixed recipe. The data are BOSSBase and BOWS2 images resized to 256×256, embedded with J-UNIWARD (JPEG, quality factor 75, 0.4 bits per non-zero AC coefficient) and S-UNIWARD (spatial domain, 0.4 bits per pixel). We estimate on the order of a few thousand GPU-hours. We will release the trained accuracies and per-device latencies as a small tabular benchmark, analogous to NAS-Bench-201 but for steganalysis. Once paid for, this cost lets every proxy, ours or anyone's, be judged without retraining.
Baselines
The do-nothing reference is full SRNet trained with the same recipe, with its latency and peak memory measured on both devices. Every slim model is compared to it in accuracy at a stated cost. CALPA-NET reports that SRNet's 92.98% on J-UNIWARD drops only to 92.86% after pruning from 47.7M to 0.69M parameters. Its FLOPs fall only about 67%. That gap between a 98% parameter cut and a 67% FLOP cut is the quantitative hint that cost lives in the unpruned bottom layers.
The strongest existing compression methods are CALPA-NET and STD-NET. STD-NET reports staying within 0.2% of uncompressed SRNet accuracy with a further 70% fewer parameters and 60% fewer FLOPs than CALPA-NET. We re-run both from their published procedures on our data and measure their real latency on the Pi-4 and GPU, which neither paper reports. We also build a latency-aware CALPA variant that prunes against our lookup tables instead of an accuracy tolerance alone. This is the fairest version of the pruning alternative, and it is the baseline we most need to beat. Beating it means equal or better detection accuracy at equal measured device latency, with a shared tuning budget and three training seeds per final model.
The strongest cheap-search competitors are two families:
- Generic zero-cost proxies:
grad_norm, SNIP, GRASP, Fisher, Synflow,jacob_cov, the Synflow–jacob_cov–SNIP vote from Zero-Cost Proxies for Lightweight NAS, and EZNAS-A. - Short-training proxies: a few epochs on a data subset, in the spirit of budgeted estimation in Rethinking Performance Estimation in Neural Architecture Search.
On NAS-Bench-201 CIFAR-10, the file reports Kendall tau 0.65 for EZNAS-A and Spearman 0.82 for the vote ensemble. There is no reported result found on Lacuna for any zero-cost proxy on a steganalysis task. All proxies are scored on the same 300-model benchmark. Short-training proxies are given exactly the wall-clock compute that our score uses across a full search, and alternatively a ten-times larger allowance, so the comparison is at matched cost.
The naive transfer baseline is MobileNetV3-style and HaeNAS-style vision backbones fine-tuned for detection.
Ablations isolate the key ingredient. First, we replace true stego twins with random ±1 changes of equal count at uniformly random positions. If this works as well, the real embedding pattern adds nothing beyond high-frequency sensitivity. Second, we drop the content denominator.
Why not just…?
Why not just prune SRNet with a latency-aware criterion? This is the most serious alternative, and we treat it as a baseline rather than dismiss it. Its limit is structural. Pruning can only narrow channels inside SRNet's fixed skeleton. It cannot move the first downsampling, shorten the full-resolution stage, or swap a learned front end for fixed high-pass filters, and those are the only knobs that significantly change latency at the bottom. It also needs a trained SRNet first, and IStego100K reports that SRNet and XuNet failed to converge on its 1024×1024 images. If latency-aware pruning matches our searched models on the Pi-4 accuracy-versus-latency curve, the search adds nothing, and we will say so.
Why not just rank candidates by training them for a few epochs? This is the second real threat. Short training uses the true objective, and its cost may be tolerable for a 300-model space. Steganalysis gives one reason for doubt. Detectors often sit near chance for a long stretch before the faint signal is picked up, and IStego100K documents outright failure to converge. Early accuracy may therefore be mostly noise. This is an empirical question, and the matched-compute comparison on the benchmark settles it. If two-epoch accuracy ranks as well as $T$ at equal cost, the practical recommendation becomes short training, and the contribution narrows to the benchmark and the diagnosis of capacity proxies.
Why not just fine-tune an off-the-shelf efficient vision network? These backbones stride and pool aggressively in their first layers. CALPA-NET argues that this, along with widening channels as resolution falls, suppresses stego noise. STD-NET also notes that compression strategies from general vision do not transfer cleanly to stego forensics. We still include MobileNetV3- and HaeNAS-style models at matched latency, because a measured gap is more convincing than an argument.
Why not just run differentiable one-shot search, as DUCCIO does? A supernet must hold every candidate operation's full-resolution activations at once, and full-resolution activations are exactly what steganalysis cannot avoid. We will report the supernet's peak memory at 256×256. If it fits, a DUCCIO-style search becomes an additional baseline under identical hard constraints.
Proposed Experiments
Proxy ranking. The first decisive check is rank quality on the benchmark, reported separately within each latency bin and within the top 10% of models. The within-bin restriction matters because any proxy can look good by tracking model size, and the top-10% restriction matters because EZNAS reports that zero-cost proxies struggle there. Success means $T$ beats the best generic proxy and the matched-cost short-training proxy on within-bin Kendall tau, for both J-UNIWARD and S-UNIWARD. The capacity-proxy prediction is confirmed if Synflow and EZNAS-A fall below 0.2 within bins, or turn negative, while correlating with parameter count.
Search under device budgets. Under budgets of 25%, 50%, and 75% of SRNet's measured latency on each device, searched models are compared with latency-aware CALPA, STD-NET, and vision backbones on accuracy at equal measured latency. We also check whether Pi-4 and GPU budgets produce different full-resolution depths and channel profiles. That would tie the per-device structural divergence reported by HaeNAS and HAT to the Aggregation Paradox.
Transfer to realistic data. The best models from each budget are retrained on ALASKA. Its organizers report a winning score of about 55% on their MD5 metric, and the paper states most entrants struggled to surpass 55–58%. They are also trained on IStego100K at native resolution. There the question is first whether the models train at all where SRNet did not, and then whether they beat DCTR's reported drop from 71.34% on same-source data to 56.95% on the mismatched test set.
Ablation. The random-perturbation ablation tells us whether any gain comes from transmitting the real embedding signal or merely from favoring high-frequency sensitivity.
References
- CALPA-NET: Channel-Pruning-Assisted Deep Residual Network for Steganalysis of Digital Images
- EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring
- STD-NET: Search of Image Steganalytic Deep-Learning Architecture via Hierarchical Tensor Decomposition
- MoGA: Searching Beyond Mobilenetv3
- HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
- HaeNAS: Hardware-Aware Efficient Neural Architecture Search via Zero-Cost Proxy
- Enhancing Neural Architecture Search With Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices
- Zero-Cost Proxies for Lightweight NAS
- Rethinking Performance Estimation in Neural Architecture Search
- IStego100K: Large-Scale Image Steganalysis Dataset