Homomorphic encryption lets a computer run a neural network on data it cannot read. The data owner encrypts the inputs, a remote machine computes on the scrambled numbers, and only the owner can decrypt the answer. This is an attractive way to put cheap, untrusted hardware to work: volunteer PCs, recycled GPUs, or a university lab's aging cluster could run encrypted jobs for groups that cannot afford frontier data centers. Encryption, however, protects only secrecy. It says nothing about whether the worker actually did the computation. A worker that is lazy, overloaded, or malicious can skip the expensive "noise-refresh" step, use a cheaper approximation of an activation function, or return an encryption of garbage. Neither the data owner nor any auditor can see the difference by inspecting the ciphertexts, because ciphertexts are designed to look random.
We propose to verify encrypted work by replaying it. The key observation is that homomorphic evaluation, unlike ordinary floating-point deep learning, is built entirely from exact modular integer arithmetic. If the software stack is pinned and the output representation is made canonical, the same encrypted job should therefore produce byte-identical output ciphertexts on any hardware. Plaintext GPU training on heterogeneous machines cannot give this guarantee. When it holds, a coordinator can re-run a small random fraction of jobs on a second worker and compare output hashes. Any deviation, however subtle, changes the bytes. A mismatch can then be traced to the first faulty operation by bisecting a committed hash chain of intermediate ciphertexts, and a cheap referee can settle the dispute by recomputing that single operation. This turns reproducible execution environments into the integrity mechanism that encrypted crowdsourced compute currently lacks.
Core Claim
Our claim has three parts. First, homomorphic evaluation under a pinned container and a canonical ciphertext encoding is bit-reproducible across CPUs and GPUs of different generations. Second, this property makes probabilistic spot-check replication a sound and cheap way to detect dishonest or silently degraded encrypted computation. Third, it holds without decrypting anything and without trusting the worker's logs or hardware. If a worker cheats on $k$ independent jobs and each job is replicated with probability $p$, the chance of catching it is
$$P_{\text{detect}} = 1 - (1-p)^k .$$
This follows because a single replay of a cheated job exposes the cheat deterministically, so the expression is simply the probability that at least one cheated job gets sampled. Ordinary floating-point workloads cannot be checked this way: their honest replays already disagree across hardware, so a mismatch proves nothing.
The research file motivates the problem from two directions. On the encryption side, Privacy-preserving Decentralized Deep Learning with Multiparty Homomorphic Encryption distributes keys so that no peer can decrypt others' data. SHE-LoRA encrypts only the most leakage-prone LoRA columns so that weak edge devices can participate. Both papers address who can read the data. Neither addresses whether the computation was done, and both assume honest-but-curious participants who follow the protocol. On the reproducibility side, the Audit-Grade Harness for Agent-driven Scientific Computation Workflows shows that computations under resource pressure silently degrade, for example by falling back from a cluster to a local node or coarsening a simulation mesh, while still hitting output targets. It catches these failures with provenance logs read by a language-model verifier. That approach depends on plaintext artifacts and on logs the auditor trusts. In our setting the worker writes its own logs and every artifact is encrypted. We keep SciAgent's insight that outcome checks miss trajectory-level failures. We replace its plaintext inspection with replay and comparison in the ciphertext domain.
Research Program
The first piece of work is an empirical determinism study. The question is simple, and as far as the file shows nobody has answered it. We will run identical encrypted programs, built on the GPU-extended OpenFHE used by EncryptedLLM, on several platforms: A100-class GPUs, older consumer GPUs, and CPUs. We will then measure byte-level agreement of intermediate and output ciphertexts. The programs are CKKS approximate-arithmetic circuits (Floating-Point Homomorphic Encryption), including bootstrapping (Bootstrapping for Approximate Homomorphic Encryption).
We expect divergence to come from implementation choices rather than mathematics. Possible sources include lazy versus full modular reduction, coefficient versus evaluation (NTT) representation at output time, the choice of key-switching decomposition, operation reordering by a compiler, and any randomness the library injects during evaluation. We will catalogue each source. We will then define a canonical form: fully reduced coefficients in a fixed residue basis and representation, with any evaluation-time randomness derived from a coordinator-supplied seed. Every backend in the pinned worker image will emit this form at each committed checkpoint.
The second piece is the worker image and protocol. Each worker runs a version-pinned container that fixes the library, the encryption parameters, and the circuit. Following the hermetic-isolation results summarized in A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests, the container also starts every job from a clean state. Low-tier workers cannot hold the gigabytes of rotation keys that KeyMemRT shows can dominate memory. The image therefore includes a key load/clear schedule in KeyMemRT's style, tuned to the container's memory cap.
This adds a requirement that existing papers do not test: key swapping must not change output bytes. The requirement also turns a known failure mode into something we can detect. A worker that hits its memory cap and quietly drops to cheaper parameters, the encrypted analogue of SciAgent's mesh coarsening, will now produce a replay mismatch rather than a plausible-looking answer.
During a job, the worker streams a hash chain over the canonical ciphertext after each layer or each bootstrapping call, and commits to the chain before learning whether the job will be replicated. When a replica disagrees, the coordinator bisects the two chains to find the first divergent checkpoint. A referee then recomputes only that segment from the agreed input ciphertext. The referee can be the data owner or any trusted node. This dispute-resolution pattern is borrowed from optimistic verification systems, and in our case its cost is bounded by one segment of homomorphic work.
The third piece applies all of this to workloads the compute-divide literature cares about. The first workload is encrypted transfer-learning heads in the style of HETAL. There a weak client extracts features locally and outsources encrypted training of the classifier head. In our version that training runs on a pool of untrusted low-tier workers rather than one trusted cloud server. The second workload is encrypted GPT-2 inference as in EncryptedLLM. The practical question is whether recycled hardware of the kind argued for in Breaking the Resource Monopoly from Industries can contribute useful encrypted throughput once verification overhead is paid. Learning@home names this exact threat in its plaintext setting: malicious volunteers returning corrupted outputs.
We scope out Sybil attacks and collusion between replica pairs. Random replica assignment reduces collusion risk but does not eliminate it, and we will report detection as a function of the colluding fraction.
Baselines
The naive reference point is unverified dispatch: jobs go to untrusted workers and their outputs are accepted as returned. It sets the throughput ceiling. It also shows how much damage the injected faults cause when nothing catches them, measured as the accuracy drop on the downstream task. It runs on the same worker pool, the same jobs, and the same fault schedule as every other condition.
The strongest existing integrity method for volunteer computing is full redundant execution: every job runs on two or three workers and the results are compared or put to a majority vote. It is the strongest competitor because, given bit-exactness, it detects every uncolluded deviation. It runs on the same pool with the same canonical encoding. Our method beats it if it reaches a matched detection rate against a fixed adversary budget at a fraction of the compute. Concretely, at a replication rate $p=0.1$ we expect roughly a 1.1x total cost instead of 2x, and we will report the exact detection-versus-overhead curve. The file reports no measured overhead for redundant FHE execution on Lacuna.
The strongest existing auditing method is a SciAgent-style trajectory verifier: append-only provenance logs written by the worker, read by a fresh-context language-model auditor. The Audit-Grade Harness reports that verifier inference costs 1.7% to 9.9% of total computational cost while catching execution-path deviations that outcome checks miss. We will give the verifier the same logs our workers emit, with ciphertext payloads replaced by hashes. We will test two adversaries: one that degrades honestly and logs it, and one that fabricates logs. Our method beats this baseline if it catches the log-fabricating adversary at comparable overhead.
The client-side baseline is decrypted-output checking. The data owner decrypts results and validation logits, as in HETAL's encrypted early stopping, and flags implausible outputs. On accuracy it must be judged against what HETAL reports for an honest server: encrypted transfer learning in 567 to 3442 seconds with at most 0.51% accuracy degradation versus plaintext. EncryptedLLM likewise reports 55.6% on SST-2 for encrypted GPT-2 versus a 55.1% baseline. Any subtle cheat that keeps accuracy within these honest-variation bands is invisible to output checking by construction. We will measure how many of our injected faults fall inside them.
Two ablations isolate the key ingredient. The first removes canonicalization: we compare raw library output bytes across hardware, which should produce false-positive mismatches on honest jobs. The second varies checkpoint granularity, from output-only to per-layer to per-bootstrap, which trades commitment bandwidth against bisection depth and referee cost.
Why not just…?
Why not just replicate every job, as volunteer computing projects do? Full replication is our upper bound on detection, so it remains a serious comparison. What it costs is the compute that weak hardware was supposed to add. Encrypted training is already roughly 19x slower than plaintext in HETAL's MNIST comparison (about 57 minutes versus about 3 minutes), so doubling it again erases much of the gain from bringing low-tier machines into the pool. Spot-checking is valid only if honest replays agree exactly. If the determinism study finds that canonical bit-exactness fails on some hardware class, full replication with tolerance-based comparison becomes the fallback. In that case we will report the result honestly as a negative finding for that class.
Why not just have the client decrypt and sanity-check the answers? This catches gross corruption but not the cheats that matter. Skipping a bootstrap until noise accumulates, or substituting a lower-degree activation approximation, can shift accuracy by less than the honest gaps the file reports: 0.51% for HETAL, and EncryptedLLM's encrypted model actually scoring above its baseline on SST-2. Output checks also give no localization, so the client cannot tell which worker or step failed. The fault-injection experiment measures exactly how many deviations stay inside these bands.
Why not just audit the worker's logs with a language-model verifier, or require hardware enclaves? Log auditing works in the Audit-Grade Harness because the harness, not the agent under test, controls the provenance trail. An untrusted worker writes its own logs and can fabricate them, and our log-fabrication adversary tests this directly. Enclaves would attest execution, but Marshaled Learning shows they demand heavy partitioning to fit models into small protected memory and rely on confidential-computing hardware that consumer and recycled machines lack. That would exclude precisely the participants this work is meant to include.
Why not just use cryptographic proofs of correct computation? Succinct proofs over homomorphic circuits would remove the need for a second worker. The file contains no system or cost measurement for them; no reported result was found on Lacuna for proof-carrying FHE evaluation. Their prover cost would land on the weakest machines. We treat them as a complementary direction, not as a baseline we can fairly run.
Proposed Experiments
The first decisive check is determinism. We will compile the KeyMemRT benchmark models (MLP, LeNet, the ResNet variants, and AlexNet) and the HETAL classification heads. These run under the pinned image on at least three hardware classes and two backends, CPU OpenFHE and its GPU extension. We report the fraction of checkpoints that are byte-identical with and without canonicalization. The project's central premise fails if honest canonical replays disagree at more than a negligible rate. We will also verify that KeyMemRT-style key loading under container memory caps leaves the output bytes unchanged.
The second check is detection. We run HETAL-style encrypted head training on MNIST, CIFAR-10, DermaMNIST, Face Mask Detection, and SNIPS, plus encrypted GPT-2 inference on SST-2, across a simulated pool in which a controlled fraction of workers cheat. The cheats are skipped bootstraps, reduced polynomial degree, fresh encryptions of zero, replayed stale outputs, and memory-triggered parameter downgrades. For each method we report detection probability against the number of cheated jobs, the false-positive rate on honest jobs, the downstream accuracy damage of undetected cheats, and total compute overhead. The prediction is that measured detection tracks $1-(1-p)^k$ for replay, while output checking misses most low-impact cheats.
The third check is the compute-divide payoff. We measure end-to-end encrypted throughput and wall-clock time when low-tier workers are added to a small high-end pool under each verification scheme, along with bisection rounds and referee cost per dispute. The proposal succeeds if verified throughput with spot-checking clearly exceeds the high-end pool alone, while full replication or log auditing either costs more or misses fabricated work.
References
- Privacy-preserving Decentralized Deep Learning with Multiparty Homomorphic Encryption
- SHE-LoRA: Selective Homomorphic Encryption for Federated Tuning with Heterogeneous LoRA
- Audit-Grade Harness for Agent-driven Scientific Computation Workflows
- EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption
- Floating-Point Homomorphic Encryption
- Bootstrapping for Approximate Homomorphic Encryption
- A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests
- KeyMemRT Compiler and Runtime: Unlocking Memory-Scalable FHE
- HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption
- Breaking the Resource Monopoly from Industries: Sustainable and Reliable LLM Serving by Recycling Outdated and Resource-Constrained GPUs
- Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
- Marshaled Learning: Bridging Large Neural Networks with Memory-Constrained Trusted Execution Environments in Federated Learning