Older chips got more efficient mostly by making transistor switching cheaper. That trend has ended. At recent manufacturing nodes, transistors keep leaking current while idle, and moving data across ever-thinner wires costs more than the arithmetic itself (Myths and Legends in High-Performance Computing argues this is why the old trend cannot resume). The consequence for neural network design is simple. Once leakage is significant, a network pays energy for every microsecond it runs, not only for every multiply it performs. Latency and energy then stop being separate objectives, and their exchange rate depends on the chip generation. Neural architecture search (NAS), the automated search over network wiring diagrams, now works with target-device costs taken from lookup tables or learned predictors fit to one device. It never asks whether the architectures it picks would still be the right ones on a leakier, more wire-limited chip.
We propose a small, physically interpretable cost model that splits each architecture's measured inference energy into three parts: a switching part driven by operation count, a data-movement part driven by bytes moved between layers, and a leakage part proportional to runtime. We then pair this cost model with training-free accuracy estimates, so that re-targeting a completed search to a new chip costs seconds and a handful of measurements rather than a new search. The scientific question is falsifiable. As the leakage and wire terms grow, does the set of best accuracy-versus-energy trade-offs on standard NAS benchmarks turn over substantially? If it does, the cheap re-targeting is needed. If it does not, one search suffices across generations, and that negative result is also worth knowing.
Core Claim
We claim two things. First, three device-level constants, fitted from a few dozen measurements on a new device, predict per-architecture energy rankings about as well as a black-box predictor trained on the same few measurements, and better than the additive per-block lookup tables used today. Second, the resulting leakage parameter is the variable that controls how much the accuracy–energy Pareto front shifts. The Pareto front is the set of architectures that no other architecture beats on both accuracy and energy at once.
The model is
$$E(a) = e_{op}\,N_{op}(a) + e_{mv}\,B(a) + P_s\,T(a),$$
where $E(a)$ is the inference energy of architecture $a$, $N_{op}(a)$ its operation count, $B(a)$ the bytes of activations and weights moved between layers (including the traffic between blocks that additive lookup tables drop), $T(a)$ its measured or predicted latency, and $e_{op}, e_{mv}, P_s$ are per-device constants: energy per operation, energy per byte moved, and static leakage power. The leakage fraction $\lambda = P_s T / E$ summarizes how "post-Dennard" a device behaves. Sweeping $\lambda$ and $e_{mv}/e_{op}$ beyond the observed range gives stylized future nodes. This is an extrapolation, and we label it as one rather than as a prediction of any specific process.
Relation to Prior Work
The closest bridges between the two areas treat the chip as a black box. HW-NAS-Bench measures latency and energy for all of NAS-Bench-201 on six devices. For its large FBNet space it sums per-block costs and concedes that this may miss inter-layer effects. We reuse its measurements as ground truth and replace block summation with a decomposition that includes inter-layer traffic and a runtime-proportional leakage term.
HaeNAS combines a zero-cost accuracy proxy with a small neural-network predictor of latency and energy fit to about 1,000 board measurements. EC-NAS fits a similar predictor to training energy and reports that shorter runtime does not imply lower energy. RHNAS predicts accelerator latency and energy-delay product from high-level hardware parameters. All three predictors are device-specific and need a large training sample. None separates leakage from switching, so none can say how its chosen architectures would fare on a different node. Our work differs by asking the cross-node question and answering it with a model whose parameters mean something physically.
On the accuracy side we reuse, rather than improve, zero-cost proxies. These score an untrained network from one forward and backward pass on a single batch (NAS-Bench-Suite-Zero; EZNAS). Their known bias toward large models (NAS-Bench-Suite-Zero) is directly relevant, because large models are exactly the ones a leakage term penalizes through longer runtime. The research file's stronger claim, that current proxies "maximize physical failure modes," has no direct evidence behind it, and we treat it as a hypothesis to measure, not a premise.
The device literature motivates the leakage and wire terms, but it constrains scope as well. Computation-in-memory devices (Energy-efficient Computation-In-Memory Architecture using Emerging Technologies) have analog noise sources that our three-term model does not capture, so we leave them out rather than overreach.
Research Program
The work has three stages. In the first, we compute $N_{op}$ and $B$ analytically for every NAS-Bench-201 cell (NAS-Bench-201). We take $T$ and $E$ from HW-NAS-Bench on each device that reports both, which excludes Edge TPU because it has no energy data. We then fit the three constants per device by non-negative least squares. A fitted $P_s$ that is positive and stable across random subsamples is the first sanity check that the decomposition is physical rather than a curve-fitting artifact.
In the second stage, we test few-shot re-targeting. We fit on one device, then refit only the three constants on $k \in \{10, 25, 50, 100\}$ measured architectures from a held-out device. We report Kendall rank correlation and Pareto-front recovery on all remaining architectures.
In the third stage, we run the sensitivity study. We combine true NAS-Bench-201 accuracies, and separately zero-cost proxy scores, with energies generated under swept $(\lambda, e_{mv}/e_{op})$. We measure how much the Pareto front changes relative to the fitted present-day device, using the fraction of front members replaced and the accuracy lost when a present-day-chosen architecture is deployed on the extrapolated node. The whole pipeline requires no network training beyond the existing tables. This is the link to the computational cost of search: re-targeting becomes a re-weighting.
Baselines
The naive reference point is ranking energy by operation count alone, which is what a search blind to the target device effectively does. HW-NAS-Bench reports that the Kendall rank correlation between FLOPs and latency falls "as low as 0.18" on Edge TPU and "below 0.5" on several devices (HW-NAS-Bench). We compute the same statistic for energy on our devices, so the proposal has a concrete floor to beat.
The second baseline is the additive per-block lookup table from HW-NAS-Bench. We build it from per-operation costs on the held-out device using the same $k$ measurements. If our model does not beat it in the few-shot setting, the inter-layer and leakage terms are not paying their way.
The strongest competitor is a black-box predictor in the style of HaeNAS and EC-NAS. This is a small multilayer perceptron given the same inputs ($N_{op}$, $B$, $T$, and the operation encoding) and the same $k$ target-device samples, optionally pretrained on the source device and then fine-tuned. Its hyperparameters are tuned with an equal budget. The only reported fidelity for such a predictor in the file is EC-NAS's $R^2$ of 0.9030 for training energy on NAS-Bench-101. That figure is a different space and a different quantity. We found no reported result on Lacuna for per-architecture inference-energy prediction accuracy on HW-NAS-Bench, so we will establish that number ourselves. We also report the predictor trained on all architectures as an upper bound.
Beating the competitor means matching or exceeding its Kendall rank correlation and Pareto recovery at $k \le 50$ on at least three of four held-out devices. It also means degrading less when the test architectures are more compute-heavy than those in the calibration sample.
Two ablations isolate the key ingredients: dropping the leakage term $P_s T$, and dropping inter-layer bytes from $B$. If the leakage-free model does just as well, the physics story reduces to "energy tracks operations and memory," and the sensitivity study becomes the main contribution.
Why not just…?
Why not just fit a small predictor on a few measurements from the new chip? This is the real threat. With enough samples, a flexible predictor will match any three-parameter model on devices seen in training. Our bet is narrower: with 10 to 50 samples, and on architectures unlike the calibration set, a model with the right functional form generalizes better. The few-shot experiment and the out-of-sample split decide this directly. If the predictor wins, we report that and drop the re-targeting claim.
Why not just regress energy linearly on FLOPs and parameter count? Our model is essentially a linear regression with physically chosen features, so this alternative is close. It differs in two terms: bytes moved between layers rather than parameter count, and a runtime term that stands in for leakage. The ablation that swaps our features for FLOPs plus parameters, with the same fitting procedure, settles whether those two terms matter. If they do not, we say so.
Why not just measure the new device and rerun the search? For existing boards this is often possible, and our method is unnecessary there. It fails when measurement is unavailable: HW-NAS-Bench could not obtain Edge TPU energy because the board lacks power rails (HW-NAS-Bench). It also fails for nodes that exist only as projections. Even then, the sensitivity study tells practitioners in advance whether rerunning is worth it at all.
Proposed Experiments
The decisive checks come in order. First, on the HW-NAS-Bench devices that report energy, we confirm that fitted leakage power is positive and stable, and that the three-term model's held-out Kendall correlation exceeds both the operation-count floor and the additive lookup table. Second, in leave-one-device-out re-targeting at $k \in \{10, 25, 50, 100\}$, we compare against the equally tuned black-box predictor on CIFAR-10, CIFAR-100, and ImageNet-16-120 cells.
Third, we run the Pareto-turnover sweep twice. The first run uses true NAS-Bench-201 accuracies, whose best values are 94.37%, 73.51%, and 47.31% on the three datasets (NAS-Bench-201). The second run uses zero-cost proxies such as EZNAS-A, which reports a Kendall correlation of 0.65 against 0.57 for Synflow and NASWOT on CIFAR-10 (EZNAS). Comparing the two runs shows whether proxy bias toward large models worsens outcomes as leakage grows.
The headline result is a curve of deployment regret against leakage fraction. A flat curve means a single search can be reused across chip generations. A steep curve means search must be repeated for each node, and it makes the case for cheap, physics-parameterized re-ranking.
References
- Myths and legends in high-performance computing
- HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark
- HaeNAS: Hardware-Aware Efficient Neural Architecture Search via Zero-Cost Proxy
- EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search
- RHNAS: Realizable Hardware and Neural Architecture Search
- NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies
- EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring
- Energy-efficient Computation-In-Memory Architecture using Emerging Technologies
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search