Phones, cameras and sensors increasingly run neural networks in pieces. The device computes the first few layers, sends the intermediate result over the radio, and a nearby server finishes the job. The best place to cut the network changes from second to second as bandwidth and battery change, so modern split-inference systems move the cut at runtime. Nothing in these systems checks that the server actually ran the layers it was handed. A server that skips a block, runs a cheaper quantized copy, or simply makes up an answer is indistinguishable from an honest one. We propose a way for a weak device to catch such a server. The device keeps tiny secret per-layer "check keys" inside its hardware-isolated secure memory. It asks the server to commit to every intermediate result, then audits one randomly chosen layer per query with a check that costs a small fraction of that layer's compute. Because the keys belong to layers rather than to a particular cut, the audit keeps working no matter where the cut moves.
In two sentences: the device stores, for each layer, a secret random compression of that layer's weights inside its secure enclave, so it can later test any server-reported layer output with one cheap dot-product-style computation. By auditing a randomly chosen, cost-weighted layer per query against commitments the server made in advance, the device obtains a tunable, provable detection rate for skipped or faked layers that holds for every possible cut point and fits in a few megabytes of secure memory.
Core Claim
A randomized check for matrix products, Freivalds' algorithm, tests whether $C = AB$ by comparing $Cs$ with $A(Bs)$ for a random vector $s$. Slalom used it to let an Intel SGX enclave verify a co-located GPU. Slalom checks every linear layer over a fast PCIe bus, fixes the division between enclave and GPU by layer type, and precomputes blinding factors for that fixed division. Our claim is that three changes turn this idea into something usable for dynamic split inference over a wireless link.
The first change is a key that is cheap enough for an Arm TrustZone enclave. For a convolution with weights $W$ that maps input $x$ to output $y$, we draw a secret random vector $s$ over output channels only. We store the single-output-channel kernel $k = \sum_o s_o W_o$, computed in the field modulo $p = 2^{61}-1$, the field SafetyNets uses. The check is that $\sum_o s_o y_{o} = k * x$ holds at every spatial position. The key is the size of one filter. The device-side cost is about $1/C_{\text{out}}$ of the layer's convolution. Any wrong output entry is caught except with probability about $1/p$. Because keys are tied to layers, moving the cut only changes which keys are in use, so nothing needs to be recomputed.
The second change is commit-then-sample auditing in place of check-everything. The server returns its prediction together with a short commitment, a Merkle root over the quantized activations of every layer it ran. Only afterwards does the device draw a layer $l$ and request that layer's input and output. This borrows the commit-sample-check pattern of VeriLLM. VeriLLM, however, audits by re-execution on peer GPUs and treats the model as one piece. Our audit is executed by the weak party, and its cost is a fraction of a single layer.
The third change is a cost-aware sampling rule. Sampled layers must be downloaded, and their sizes differ a lot. As SC2 Benchmark documents, early activations can exceed the raw input in size. Sampling uniformly therefore wastes bandwidth, and sampling only cheap layers invites cheating on expensive ones. Model a rational "lazy provider", in the sense of Hickey and Cormode: it saves $g_l$ compute by faking layer $l$ and loses a penalty $P$ when caught. Then the cheapest sampling probabilities $p_l$ that deter every shortcut, at per-audit download cost $c_l$, satisfy
$$p_l \ge \frac{g_l}{P}, \qquad \mathbb{E}[\text{audit cost}] = \sum_{l > \text{cut}} p_l\, c_l .$$
This holds because a shortcut is unprofitable exactly when its expected penalty exceeds its saving. Both $g_l$ and $c_l$ depend on where the cut currently sits, so the partitioning controller and the auditor must be optimized together. Against an arbitrary rather than rational adversary, a layer that is faked on every query survives $m$ queries with probability $(1-p_{\min})^m$.
Research Program
The system has four parts, and each one answers a concrete question.
The arithmetic substrate is integer quantized inference: INT8 weights and activations with wide integer accumulation. Freivalds checks need exact arithmetic, and floating-point drift across heterogeneous hardware makes honest workers look malicious, which is a problem VeriLLM had to engineer around. Slalom reports that fixed-point quantization costs less than 0.5% accuracy. We will measure whether that holds for the mobile networks used in dynamic partitioning, such as MobileNetV2 and the bottlenecked ResNets in sc2bench. Requantization, ReLU and pooling are cheap elementwise or local operations. For a sampled layer the device recomputes them directly from the opened tensors, which avoids the polynomial-activation restriction that limits SafetyNets.
The key store lives in the device enclave. The keys must stay secret, because the server knows $W$ and could solve for $s$ from a leaked $k$. Normal-world malware on the device is therefore part of the threat. This is the role of the trusted execution environment, the hardware-isolated region whose memory the device's own operating system cannot read. DarkneTZ and MirrorNet show that this region is roughly 10 to 16 MB on Raspberry-Pi-class TrustZone hardware. We will measure total key size per model, which we expect to be far below that limit because each key is a single filter. Where keys must spill, they will be sealed in normal-world storage under an authenticated cipher. The sealing key could be rooted in a hardware fingerprint such as TV-PUF, but this is an optional deployment detail, not a contribution. Keys are rotated by an offline pass over the weights, for instance while charging. The device aborts the session on the first failed audit, so an adversary cannot learn $s$ by probing.
The same keys also verify the device's own prefix when it runs in the untrusted normal world. This closes the gap left open by DarkneTZ and MirrorNet: both protect weights and outputs but assume normal-world execution is honest.
Audits are taken off the critical path. The prediction is accepted optimistically. The audit opening is fetched and checked afterwards within a bounded window, and results are flagged retroactively on failure. This is the same latency trade TensorTEE makes in hardware with delayed verification and poison tracing. We will quantify the server-side buffering this requires and the window it implies.
The controller extends a dynamic partitioner in the style of AMP4EC. Its cost model will include audit download bytes and device check cycles alongside compute and transmission, and the sampling rule above is re-solved at each cut change. A central empirical question is how much the optimal cut shifts once auditing is priced in. This is the bottleneck paradox of SC2 Benchmark, now with a third cost term.
Confidentiality of activations is out of scope. Slalom-style blinding needs precomputed responses for the first remote layer at each possible cut. We will report the storage this would require but not claim it.
Baselines
The naive reference point is unverified dynamic split inference: the AMP4EC-style partitioner on the same MobileNetV2 and sc2bench models with no checks. It sets the floor for latency, energy and bytes, and it has zero detection. AMP4EC reports a 78.35% latency reduction, from 1082.53 ms to 234.56 ms, on its Docker edge testbed. We will rerun that configuration and report our overheads relative to our own reproduction, not their number.
The strongest verification competitor is Slalom-style exhaustive checking ported to the network: every remote linear layer is verified with full Freivalds checks inside the device enclave. This requires downloading every remote activation. Slalom reports 6x to 20x end-to-end verifiable-throughput gains over running inside the enclave, but in a co-located PCIe setting with no wireless link. We expect the bandwidth cost to dominate over radio. Beating this baseline means reaching the same per-layer soundness guarantee, accumulated over queries, at an order of magnitude less audit traffic.
The strongest cheap heuristic is Fides, which runs a distilled verifier in a server-side SGX enclave plus a client divergence detector. Fides reports up to 98% detection of prediction-switching attacks and 4.8x to 26.4x speedups over Slalom. We will train Fides on the same models and datasets with a matched tuning budget, then test both systems on the same attack suite. Beating it means near-1 detection on subtle attacks that Fides misses, at comparable client cost.
The probabilistic re-execution reference is VeriLLM-style auditing, in which a random query is fully re-executed. Here it means re-running the whole suffix, either on the device or on a second trusted node. VeriLLM reports a 99.85% detection rate and a 1.9% false-positive rate on Qwen2.5-7B. We expect a 0% false-positive rate from exact integer checks. On the device, the comparison is audit energy per unit of detection probability.
Two ablations isolate the proposal's key ingredients. One replaces cost-aware sampling with uniform layer sampling, which isolates the value of the sampling rule. The other uses full output-by-input keys instead of channel-factored keys, which isolates whether the factorization is what makes TrustZone residency possible.
Why not just…?
Why not just re-execute a random fraction of queries on the device? For a fixed audit budget, full-suffix re-execution buys far less detection than single-layer channel checks, because one check costs roughly $1/C_{\text{out}}$ of one layer, whereas re-execution costs all remote layers. This alternative becomes a real threat when the remote suffix is only one or two small layers, as happens at deep cuts. The experiments will report the cut depth at which re-execution becomes competitive, and the controller should switch to it there.
Why not just put a TEE on the server and trust attestation? Remote attestation would indeed certify the server's code. However, commodity edge accelerators rarely offer GPU enclaves. CPU enclaves are slow for dense algebra, and Phantom and TensorTEE show that enclave I/O and metadata dominate even when the enclave is large. TensorTEE's fix also requires custom silicon. If confidential accelerators become standard on edge servers, this alternative weakens our motivation. Our scheme still differs in assuming no trusted hardware on the server, and we will state that scope explicitly.
Why not just use a statistical output validator such as Fides? Fides detects outputs that diverge from a distilled verifier. A lazy server running a slightly cheaper quantized or pruned model can produce outputs that stay close to that verifier. We will construct exactly such approximate-execution attacks and compare detection curves. If Fides catches them as reliably, the case for exact checking narrows to its zero-false-positive guarantee alone.
Why not just check every layer, as Slalom does? The algebra is the same. The cost is not, because every opened activation crosses the radio. The uniform-versus-exhaustive-versus-cost-aware comparison in the experiments measures that gap directly.
Proposed Experiments
The hardware is a Raspberry Pi 3B with OP-TEE TrustZone, matching DarkneTZ and MirrorNet, acting as the device. A GPU edge server acts as the untrusted worker, connected through emulated links driven by bandwidth traces. The models are MobileNetV2, matching AMP4EC, and the bottlenecked ResNet variants from sc2bench, evaluated on ImageNet, CIFAR-10 and CIFAR-100.
The attack suite covers skipping a residual block, substituting a lower-precision layer, replaying cached activations, faking only the final layer, and an adaptive adversary that targets the least-sampled layer. We will measure detection per query and per $m$ queries, false-positive rate, added end-to-end latency, device energy per query, audit bytes, enclave memory, and the quantization accuracy delta.
Three checks are decisive. First, channel-factored keys for a full ImageNet model must fit within the roughly 10 to 14 MB usable TrustZone memory with no paging. Second, at a matched audit-traffic budget, cost-aware sampling must deter all rational shortcuts where uniform sampling leaves expensive layers profitable to fake. Third, across a bandwidth trace that forces at least several cut changes per minute, the audit must keep its guarantee with no re-precomputation, while total overhead stays within the 3% to 10% range that DarkneTZ reports as acceptable for enclave protection. If the third check fails because audit traffic pushes the controller to systematically different cuts, that shift is itself the finding: a measured price of integrity in dynamic split inference.
References
- Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
- SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud
- VeriLLM: A Lightweight Framework for Publicly Verifiable Decentralized Inference
- SC2 Benchmark: Supervised Compression for Split Computing
- Cheap Checking for Cloud Computing: Statistical Analysis via Annotated Data Streams
- DarkneTZ: towards model privacy at the edge using trusted execution environments
- MirrorNet: A TEE-Friendly Framework for Secure On-Device DNN Inference
- TV-PUF : A Fast Lightweight Aging-Resistant Threshold Voltage PUF
- TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor Computing
- AMP4EC: Adaptive Model Partitioning Framework for Efficient Deep Learning Inference in Edge Computing Environments
- Fides: A Generative Framework for Result Validation of Outsourced Machine Learning Workloads via TEE
- Phantom: Privacy-Preserving Deep Neural Network Model Obfuscation in Heterogeneous TEE and GPU System