Detecting Silent Failures in Encrypted Inference

Homomorphic encryption lets a server run a neural network on data it cannot read. The client encrypts an input, the server computes on the ciphertext, and only the client can decrypt the answer. To make this possible, practical systems replace operations such as ReLU with polynomials that approximate them well only on a fixed input interval, for example $[-B, B]$. Inside that interval the approximation error is small and provably bounded. Outside it, a polynomial diverges quickly. In plaintext, an engineer would notice a pre-activation of 40 when the fit was done on $[-8, 8]$. Under encryption no one can see the intermediate values, not even the server. A drifted input, such as a medical image with a new staining protocol or a sensor with a different gain, can therefore produce a wrong prediction that looks plausible, and nothing indicates that the approximation guarantee no longer held.

This note proposes to measure how often such silent failures occur under realistic distribution shift, and to close the gap with a cheap encrypted range certificate. The certificate is a polynomial summary of each layer's pre-activations, computed alongside the network and decrypted by the client together with the prediction. If the certificate is below a threshold, every approximated activation provably stayed inside its fitted interval. The theory-to-practice framing is literal here. The guarantee in the approximation analysis carries an assumption, bounded inputs, that deployment silently violates. The certificate turns that assumption into a checkable runtime condition. The logged violations then feed back into where the approximation intervals are fitted, which is the bidirectional loop that Chernova et al. describe in general terms.

Core Claim

Let $x_1, \dots, x_n$ be the pre-activation values entering an approximated activation layer, and let $B$ be the interval half-width on which its polynomial was fitted. For any integer $k \ge 1$,

$$\max_i |x_i| \;\le\; \Big(\sum_{i=1}^{n} (x_i/B)^{2k}\Big)^{1/2k} \cdot B,$$

so the polynomial quantity $S_k = \sum_i (x_i/B)^{2k}$ satisfying $S_k \le 1$ certifies that every $x_i$ lies in $[-B, B]$. The bound is the standard inequality between the max-norm and the $2k$-norm. Under the homomorphic encryption scheme for approximate real arithmetic (CKKS), computing $S_k$ costs one scalar multiplication, $\log_2(2k)$ squarings, and a logarithmic number of slot rotations to sum. It runs on a side branch that ends at the output, so it never feeds back into the main computation and does not lengthen the critical path that forces bootstrapping. Bootstrapping is the expensive noise-reset operation. Larger $k$ tightens the certificate and reduces false alarms, at slightly more depth on the side branch.

We make two claims. First, polynomial-approximated encrypted networks in the style of Orion, SAFENet, and HTCNN have a non-trivial silent-failure rate under moderate, realistic corruptions, even when their clean accuracy matches plaintext. Second, a per-query certificate aggregated over layers catches most of these failures at a small fraction of inference cost. It does so more reliably than inspecting the decrypted output, which is the obvious thing a client would otherwise do.

Relation to Prior Work

The motivating evidence comes from the overhead-reduction literature, which tunes polynomial approximations for clean test accuracy. CryptoNets and HTCNN replace ReLU with $x^2$. HTCNN pays for this with 74.19% CIFAR-10 accuracy. SAFENet keeps true ReLUs in important channels and uses degree-2 or degree-3 polynomials elsewhere. Orion compiles PyTorch models to CKKS and reports that SiLU-polynomial substitutes cut runtime by 1.77x for a 2.1% cleartext accuracy drop. All of these evaluate on in-distribution test sets. None of them asks what happens when pre-activations leave the fitted range at deployment time, and none gives the client a way to know.

Sylph is the closest evidence that dynamic range matters in practice. It suppresses activation outliers in Llama-3 with token prepending and orthogonal rotations so that CKKS evaluation does not break. Sylph shrinks the range offline for a known model; it does not detect range violations per query. The two approaches are complementary.

Activation-Free Backbones (PolyNeXt) partly contradicts our premise. Its networks contain no approximated activations, so there is no fitted interval to violate. We address it directly in the experiments. Exact polynomial networks still face CKKS scale overflow, and their stability depends on Sigmoid-Scale gating, which itself must be approximated under encryption. PolyNeXt was also never run under FHE.

The compilers supply the method. EVA and Orion automate rescaling and bootstrap placement. We build the certificate branch as an Orion module so that its placement is handled by Orion's level-graph optimizer instead of by hand.

The empirical comparison by Haralampieva et al. motivates the medical setting. It used Malaria cell images and found that all frameworks preserved clean accuracy, but it tested only clean data.

Research Program

The work has three parts.

Failure audit. We take approximated networks as they would be deployed and sweep corruption severity. For each query we measure four things: the plaintext-ReLU prediction, the encrypted polynomial prediction, the maximum pre-activation relative to $B$ in each layer, and whether the two predictions disagree. Most of the sweep runs in a plaintext emulator that applies the exact polynomials and injects CKKS-scale noise. We validate the emulator against real ciphertext execution on a stratified subset, so that the cheap sweeps are trustworthy.

Certificate. We implement $S_k$ per approximated layer. We then aggregate the layers into a single scalar, the maximum of the per-layer values computed in plaintext after decryption, or alternatively their sum computed under encryption. The aggregation keeps what the client learns about model internals to one number per query. We characterize three properties: soundness up to CKKS noise, the false-alarm rate as a function of $k$, and the added cost in latency, rotations, levels, and bootstraps.

We also handle a known hazard explicitly. When $|x_i/B| \gg 1$, $S_k$ can exceed the plaintext capacity of CKKS and wrap around to a small-looking value. We treat this as a soundness threat. The mitigation is to evaluate $S_1$ alongside $S_k$, since $S_1$ has far more headroom, and to set the scale budget so that any wrap-around in $S_k$ is accompanied by a large $S_1$.

Feedback. Violations logged under shift are used to re-fit the interval or reallocate polynomial degree per layer. This means widening $B$ only where violations actually concentrate. We measure how much silent failure the re-fit removes, and at what depth cost, compared with uniformly widening every interval.

Baselines

The naive reference is deployment as published: Orion-compiled ResNet-20 on CIFAR-10 with polynomial activation substitutes, a SAFENet-style mixed network, and an HTCNN-style $x^2$ network on MNIST, all with no runtime check. Its silent-failure rate under shift is the quantity we report. No reported result found on Lacuna for silent-failure rate or prediction agreement under distribution shift for any FHE-approximated network. Orion's reported numbers, 2.38x faster than Fhelipe on ResNet-20 and 37 bootstraps against 58, set the latency reference the certificate must not materially erode.

The strongest competitor is client-side output inspection. The client decrypts the logits and flags a query when the logit norm or the softmax maximum is anomalous, with the threshold calibrated on clean validation data. This costs nothing extra under encryption, and gross polynomial blow-ups do produce huge logits. We run it on exactly the same queries and calibration split, with its threshold tuned to the same clean false-alarm rate as the certificate. The certificate beats it if, at matched clean false-alarm rate (5%), it achieves meaningfully higher recall on silent failures. Silent failures are queries where the encrypted and plaintext predictions disagree. The decisive cases are the mild overshoots that give wrong but plausibly confident outputs.

A second competitor is interval widening: fitting every polynomial on $2B$ or $4B$ at a degree that keeps clean accuracy fixed, with no runtime check. We compare at matched total latency including bootstraps. The certificate plus targeted re-fit wins if it reaches a lower silent-failure rate at equal or lower cost.

The plaintext input-side detector is a simple out-of-distribution score on the raw input, such as norm or per-channel statistics, computed by the client before encryption. It tests whether drift is visible without looking inside the network.

Two ablations isolate the key ingredient. Replacing $S_k$ with $S_1$ tests whether the higher-order certificate matters. Checking only the first approximated layer tests whether per-layer coverage matters, given that SAFENet's ranking puts early layers at highest sensitivity.

Why not just…?

Why not just look at the decrypted logits? This is the real threat to the proposal, and it is the strongest baseline above. If silent failures almost always produce extreme logits, the certificate is unnecessary and we will report that. Our expectation is that this fails in the regime that matters. When pre-activations exceed $B$ by a modest factor, a degree-2 or degree-3 polynomial distorts values without exploding. Batch normalization and later layers can then pull the distorted values back into a normal-looking range. The matched-false-alarm recall comparison on corrupted Malaria and CIFAR-10 settles this directly.

Why not just simulate the polynomial network in plaintext and audit offline? We do exactly this for the failure audit, because the divergence is deterministic and plaintext emulation reproduces it cheaply. An offline audit only covers shifts someone anticipated in advance. At serving time the server cannot see the queries, so it cannot audit the inputs that actually arrive. The certificate is the only per-query signal available under the threat model.

Why not just widen the intervals or use higher-degree polynomials? Higher degree means more multiplicative depth and more bootstraps. This is the dilemma documented by Chen et al., and it shows up concretely in Orion's 1.77x runtime gap between polynomial choices. Widening also gives no guarantee against shifts larger than the new interval. The matched-latency comparison measures whether targeted re-fitting driven by certificates dominates uniform widening.

Why not just use activation-free polynomial networks or a hardware enclave? PolyNeXt removes fitted intervals but keeps CKKS scale-overflow risk and depends on approximated gating. We run a small PolyNeXt-style network through the same audit to see whether overflow-driven silent failures remain. A trusted execution environment such as TF-Trusted ran at 0.14s against 549s for HE-Transformer in Haralampieva et al., but it changes the trust assumption to trusting the hardware vendor and accepting side-channel risk. It is out of scope for users who require cryptographic guarantees.

Proposed Experiments

The experiments use MNIST, CIFAR-10, CIFAR-100, and Malaria cell images, each with graded synthetic corruptions modeled on ImageNet-C. These are blur, noise, contrast, and brightness, plus a color and stain shift for Malaria to mimic a change in laboratory protocol. Networks are an Orion ResNet-20 with polynomial activations, a SAFENet-style ResNet-32 on CIFAR-100, an HTCNN-style $x^2$ CNN on MNIST, and a small polynomial backbone in the style of PolyNeXt. Real CKKS runs use Orion on CPU for a stratified subset of a few hundred queries per corruption type and severity. These runs validate the emulator, and we report agreement between emulated and encrypted predictions.

Metrics are as follows. The silent-failure rate is the fraction of queries where the encrypted and plaintext-ReLU predictions disagree with no flag raised. Certificate recall and false-alarm rate are reported at matched thresholds. The added cost is measured in latency, rotations, levels, and bootstraps relative to the unguarded model. We also report accuracy recovered by the feedback re-fit.

There are three decisive checks. The first is whether the silent-failure rate under moderate corruption is materially non-zero. If it is not, the problem is not real and the project stops at the audit. The second is whether the certificate beats output inspection on recall at matched false-alarm rate. The third is whether the certificate adds under 10% to end-to-end encrypted latency on ResNet-20 without adding bootstraps on the main path.