When a neural network runs inside a trusted execution environment (a hardware-protected region of memory and code that the host operating system cannot read or tamper with), everything placed inside that region must be trusted. That trusted set is the trusted computing base, or TCB. For machine learning it usually ends up very large, because developers copy a full inference runtime, with hundreds of operator kernels and their libraries, into the protected region. The SoK: Machine Learning with Confidential Computing names this the "TCB paradox." A large part of that code exists only because of the network's design. Each distinct operator type, kernel shape, activation function and number format the network uses pulls another kernel into the trusted binary. So the architecture largely decides how much code has to be trusted.
We propose a hardware-aware neural architecture search, meaning an automated search over network designs that scores each candidate on a target-specific cost as well as accuracy, in which the cost is the size of the enclave code the network forces us to trust. Each layer also gets a placement decision. It can run inside the enclave, or, if it is linear, be outsourced to an untrusted accelerator under Slalom-style verification. The cost of a network is therefore not a sum of per-layer costs. It is the size of the union of kernel implementations the in-enclave layers need, plus a fixed cost for the verification code, plus a latency term for each crossing of the enclave boundary. The question is whether searching directly against this union-shaped cost finds networks that are more accurate and faster at a given trusted-code budget than hand-simplified models or than standard search followed by stripping unused kernels.
Core Claim
Existing hardware-aware search treats the device as a passive target whose cost adds up layer by layer. This holds for the latency lookup tables of MoGA and HAT, and for the memory and latency penalties of DUCCIO in Enhancing NAS with Multiple Hardware Constraints. Confidential-computing work treats the model as a fixed black box. SEED shrinks the operating system and runtime layer of a confidential VM from about 690 MB to 22 MB, but its workloads are unmodified TensorFlow and PyTorch containers. GuaranTEE and the CCA on-device study run stock MobileNetV1 and AlexNet inside Arm realms, which are hardware-isolated VMs under Arm's confidential computing architecture. Slalom offloads every linear layer to a GPU by a fixed rule.
Our claim is that the trusted-code footprint of ML inference is a set-valued function of the architecture and placement, and that optimizing this function directly moves the accuracy–latency–TCB frontier. We write the objective for an architecture $a$ with per-layer placement $p$ as
$$T(a,p) = C_0 + \sum_{u \in U_{\text{in}}(a,p)} c_u + \mathbb{1}[\text{any layer outsourced}]\, c_{\text{ver}},$$
where $C_0$ is the fixed monitor, loader and attestation code, $U_{\text{in}}(a,p)$ is the set of distinct kernel variants executed inside the enclave, $c_u$ is the measured code size of kernel $u$ after static linking, and $c_{\text{ver}}$ is the code for Freivalds checking and additive blinding. Freivalds checking is a cheap randomized test that an outsourced matrix product is correct, and blinding adds random noise so the accelerator never sees the real input. Latency is $L(a,p) = \sum_l \text{LUT}(l,p_l) + k(a,p)\, c_{\text{sw}}$, where $k$ is the number of enclave boundary crossings and $c_{\text{sw}}$ is the measured cost of one crossing.
Adding a layer of an operator type the network already uses costs no trusted code. Introducing a new kernel variant costs its full code size. A purely additive cost model cannot represent this, and it is the mechanism that differs from prior hardware-aware search.
Research Program
The first step is building the cost model. We will take a small C inference runtime that can be statically linked and compile it into enclave binaries containing exactly the kernels a given network needs. The targets are a SEED-style minimal SEV-SNP image and the Arm FVP simulator setup used by GuaranTEE. We then measure $c_u$ for each kernel variant in the search space, counting both lines of code and linked binary bytes, and record which shared helper routines each kernel pulls in. Because kernels share helpers, we will also check how far $T$ departs from a clean sum over kernels. Crossing cost $c_{\text{sw}}$ comes from instruction counts at the realm boundary, following the methodology behind GuaranTEE's report of 1.7x instruction inflation for inference.
The search space is a MobileNet-style supernet. Each block chooses a kernel size, an expansion ratio, an activation function (ReLU, ReLU6, hard-swish, or none), a number format (INT8 or fixed-point), and, for linear layers, whether to run inside the enclave or outsource. We encode non-ReLU activations and mixed formats as distinct kernel variants, so a network that mixes many of them pays for that variety in trusted code. Because $T$ is discrete and non-differentiable, we reuse the search method of ConNAS. ConNAS handles a non-differentiable hardware budget without a tuned penalty weight by steering the architecture gradient toward feasibility whenever a candidate breaks the budget. Here "toward feasibility" means pushing the search toward kernels already in the set. To check that results do not depend on the search algorithm, we also run a regularized evolutionary search under the same budget.
The scientific output is a set of measured trade-off curves of accuracy against trusted-code size at fixed latency, together with an analysis of what the searched networks give up. We expect them to drop activation and format diversity first, to outsource selectively rather than outsourcing every linear layer, and to merge outsourced runs so that fewer crossings occur.
Baselines
The do-nothing reference is a stock MobileNetV2 run with a full inference runtime inside the realm or confidential VM, as GuaranTEE and the CCA study do. It defines the trusted-code size we try to reduce, and its accuracy is the ceiling we hope to stay close to.
The strongest existing method for keeping enclave work small is Slalom. It applies a fixed rule: every linear layer goes to the GPU under Freivalds checking and blinding, and every non-linear layer stays in the enclave. Slalom reports 6x to 20x throughput speedup for verifiable inference on ImageNet-scale models, and a VGG16 enclave memory footprint reduced from 550 MB to 36 MB. Slalom does not report trusted-code size, and we found no result on Lacuna for TCB lines of code under Slalom-style partitioning. We will therefore re-implement it on our runtime, with the same models, quantization and hardware, and measure $T$ and $L$ ourselves.
The strongest search baseline is ConNAS with a standard additive latency and model-size budget, followed by dead-code elimination of unused kernels. ConNAS reports meeting its constraint in 100% of runs and coming within 0.14% to 0.65% of the best valid architecture on NATS-Bench. It gets the same supernet, training epochs and number of search GPU-hours as our method.
The key ablation keeps everything in our method but replaces the union cost $T$ with an additive per-layer code-size proxy. If this ablation matches our method, the set structure is not doing the work. We count our method as beating a baseline if, on at least two of the three benchmarks, it gives at least 1 point higher accuracy at equal trusted-code size and equal-or-lower latency. Equivalently, it can give a meaningfully smaller TCB at equal accuracy, which we will report against the measured spread across search seeds.
Why not just…?
Why not just hand-design a homogeneous network, for example plain 3x3 convolution and ReLU stacks with one number format? This is the most serious threat. A VGG-like or plain ResNet design already uses very few kernel variants, so its trusted code may be close to minimal with no search at all. We will include hand-homogenized ResNet and MobileNet variants as baselines at matched size. If they lie on the searched frontier, we will report that the right recommendation is a design rule rather than a search method. The search is only justified if mixing a few well-chosen variants, together with placement, buys accuracy that a strictly homogeneous network cannot reach.
Why not just run standard hardware-aware search and strip unused kernels afterwards? Stripping after the fact removes code the network does not use. It cannot stop the search from choosing networks that use many variants, because the search never sees that cost. The ConNAS-plus-elimination baseline tests this directly. If standard search happens to pick few variants anyway, the two curves will coincide, and we will report that outcome.
Why not just use Slalom's fixed split, which already moves most compute out of the enclave? Slalom's rule outsources every linear layer, which maximizes the number of boundary crossings. On Arm realms, GuaranTEE and the CCA study attribute most of the overhead to crossings and exception handling rather than to arithmetic. Slalom also keeps every non-linear kernel the model happens to use. Whether searched placement beats the fixed rule on latency at equal trusted code is a measurement in our experiments, not an assumption.
Why not just shrink the operating system and runtime layer, as SEED does, and leave the model alone? SEED's 22 MB image is complementary to this work. It reduces the fixed cost $C_0$ and does not touch the kernel term. We build on a SEED-style minimal image so that the kernel term is a visible share of the total. If the kernel term turns out to be negligible next to $C_0$, our premise fails, and the first experiment is designed to show this early.
Proposed Experiments
The first check comes before any search. We measure the share of trusted code that kernels account for in a minimal image when running MobileNetV2 and ResNet models. If kernels are a small fraction of the total, we stop.
The main experiments run search on CIFAR-10, CIFAR-100 and Visual Wake Words, and transfer the results to Tiny ImageNet. Each is reported as accuracy against trusted code (lines of code and binary bytes) and against end-to-end latency, including crossings. Latency is measured on the SEV-SNP prototype and as instruction counts on the Arm FVP. We compare against the four baselines above under matched training and search budgets, and include the additive-proxy ablation. No prior accuracy-at-TCB-budget result exists on Lacuna for these benchmarks, so all baseline numbers will be produced in our setup.
We also check that the cost model predicts reality, by correlating the predicted $T$ and $L$ with measured binary size and measured runtime over 50 sampled architectures. Finally, as a security sanity check, we count the externally reachable entry points of each enclave binary. This tests whether the smaller code size also means a smaller interface, rather than only fewer bytes.
References
- Machine Learning with Confidential Computing: A Systematization of Knowledge
- MoGA: Searching Beyond Mobilenetv3
- HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
- Enhancing Neural Architecture Search With Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices
- SEED: A Minimal‑Footprint TEE Framework for Verifiable, Confidential Microservice Deployment
- GuaranTEE: Towards Attestable and Private ML with CCA
- An Early Experience With Confidential Computing Architecture for On-Device Model Protection
- Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
- Constraint-guided Hardware-aware NAS through Gradient Modification