Fully homomorphic encryption (FHE), a way of running a neural network on data that stays encrypted the whole time, lets a hospital or bank send a query to a cloud model without revealing it. The cost is steep. Encrypted inference runs orders of magnitude slower than ordinary inference, and hardware designers have answered with two competing strategies. One is to fabricate custom FHE chips with huge on-chip memories. The other is to repurpose AI accelerators that are already deployed, such as Google TPUs. Both camps compare themselves on speed and on throughput per watt. Neither counts the carbon emitted when the chip is manufactured, which is where most of a modern computer's lifetime emissions sit. This proposal asks a concrete question with a numerical answer. How many encrypted predictions must a custom FHE chip serve before its operational energy savings repay the extra manufacturing carbon, compared with running the same work on repurposed TPUs? Is that number reachable within the chip's lifetime at realistic demand for private inference?
The answer is not obvious, and it plausibly runs against the direction FHE hardware research has taken. Custom accelerators are reported to be thousands of times faster than CPUs on core kernels. Repurposed TPUs are reportedly within a small constant factor of those accelerators while requiring no new silicon. The TPU argument also has a hidden flaw: it is free only if the TPUs have spare capacity. We build a probabilistic life-cycle model that is parameterized by FHE settings, and we use it to compute the break-even inference volume as a distribution rather than a single point. We then test whether that more careful accounting changes the hardware decision relative to the simpler accounting practices already in use.
Core Claim
Life-cycle assessment (LCA) is the standard method for totaling a product's emissions from raw materials through manufacturing, use and disposal. It separates two framings. "Attributional" accounting assigns a share of existing emissions to a product. "Consequential" accounting asks what extra emissions a decision causes. Our claim is that the choice between a dedicated FHE chip and repurposed accelerators is a consequential procurement decision. It is governed by one inequality.
Let $C^{A}_{\text{emb}}$ be the manufacturing carbon of the dedicated accelerator, including its memory and peripherals. Let $e_T$ and $e_A$ be the energy per encrypted inference on the repurposed TPU and on the dedicated chip. Let $I$ be the carbon intensity of the grid during the hours the work actually runs. Let $c^{\text{marg}}_T$ be the manufacturing carbon per inference that the TPU route causes by displacing other work. That last term is zero when the TPUs have slack and approaches the amortized TPU manufacturing cost when their capacity is fully committed. The dedicated chip pays back after
$$N^{*} = \frac{C^{A}_{\text{emb}}}{I\,(e_T - e_A) + c^{\text{marg}}_T}$$
inferences. The decision favors fabrication only if $N^{*}$ is below the lifetime demand the chip will actually serve.
Two features of this inequality are specific to FHE and absent from existing computing-LCA tools. First, the FHE parameters determine both $e_A$ and, through required on-chip memory, the die area that drives $C^{A}_{\text{emb}}$. These parameters are the polynomial ring degree, the ciphertext modulus, the number of noise-resetting "bootstrapping" steps and the size of the evaluation keys. Second, encrypted inference is naturally batched and asynchronous, so $I$ is partly a choice. Deferring batches to low-carbon hours shrinks the operational savings a dedicated chip can offer, which pushes $N^{*}$ up. The novelty is the parameterized, uncertainty-aware, consequential computation of $N^{*}$ for FHE, together with the empirical question of whether it flips decisions that simpler accounting would make.
Grounding in Prior Work
Two groups of papers motivate the gap. On the hardware side, SoK: Fully Homomorphic Encryption Accelerators surveys 14 accelerators and reports that the F1 ASIC reaches "up to 20546.9x acceleration for NTT compared to a baseline CPU." The NTT, or number theoretic transform, is the polynomial-multiplication kernel at the heart of FHE. The survey also finds that the working sets demand large on-chip SRAM. CoFHE reports that such accelerators draw over 160 W, largely from their memory. It replaces part of the SRAM with spintronic NAND-SPIN memory and cuts end-to-end energy by 26%, but it counts only operational energy and omits bootstrapping.
Leveraging ASIC AI Chips for Homomorphic Encryption presents CROSS, a compiler that maps FHE onto TPU matrix units. It reports throughput-per-watt gains of 451× over CPUs running OpenFHE and 7.81× over GPUs running WarpDrive. Its implicit environmental argument, that reuse avoids fabricating new chips, is never quantified. We make that argument testable, and we also expose its dependence on spare capacity.
The accounting side supplies the method. PCAM replaces fixed emission factors with distributions over fab energy, wafer yield and hourly grid intensity. It reports embodied-carbon error within 2.67% against 23.02% for the deterministic LLMCarbon, and operational error at most 7.44% against 108.51%. We reuse its probabilistic machinery. PCAM covers only commodity CPUs, GPUs, DRAM and SSDs running plaintext models, so we must extend it to FHE-specific die and memory configurations.
The Sunk Carbon Fallacy constrains the design. It shows that charging amortized manufacturing carbon to real-time scheduling decisions raises net emissions, and that manufacturing carbon belongs only in procurement and retirement decisions. We follow that rule strictly. $N^{*}$ is a procurement quantity, and the scheduling component of the study uses only operational carbon.
A view of the sustainable computing landscape supplies the tension we quantify. That survey argues that advanced process nodes lower energy per operation but raise manufacturing carbon per unit of die area.
Research Program
The first component is a parameter-to-silicon inventory. It covers the accelerators surveyed in the SoK paper and the CoFHE design. For each one, we record die area, process node, SRAM and HBM capacity and power from the original accelerator publications. For each FHE workload, we derive the minimum on-chip memory implied by its ring degree and key sizes. This yields an embodied-carbon distribution for each design using PCAM-style yield and fab-energy distributions. The SoK finding that SRAM benefits plateau once working sets fit lets us bound how much memory a design actually needs, rather than taking the published capacity as given.
Spintronic memory has no fab data in the file. For CoFHE, we therefore treat its manufacturing intensity per area as a wide prior and report how the conclusion depends on it. We do not claim a point estimate.
The second component is measured operational energy on hardware we can run. We run CROSS through its open-source jaxite kernels on TPU v6e, OpenFHE on CPU and WarpDrive-class GPU code. For custom ASICs, which are mostly simulated designs, we take $e_A$ from reported power and throughput, and we propagate the uncertainty in those reports.
The third component estimates the displacement term $c^{\text{marg}}_T$. We sweep TPU fleet utilization from idle to fully committed and treat that sweep as the axis on which the TPU route's "free silicon" assumption holds or fails.
The fourth component is a carbon-aware batch scheduler. It defers packed encrypted batches within a latency tolerance, using hourly grid traces from PCAM's five regions. It optimizes only marginal operational carbon, as the Sunk Carbon Fallacy result requires. Its output feeds the effective $I$ into $N^{*}$.
The deliverable is a distribution of $N^{*}$ for each design, workload, region, utilization level and scheduling policy. We compare that distribution with lifetime demand scenarios, anchored at the batched throughput CryptoNets reports for its MNIST network. In CryptoNets, that figure is 58,982 predictions per hour on a single Xeon.
Baselines
The naive reference point is how the FHE literature already compares hardware: operational energy per inference, or throughput per watt, with no manufacturing carbon at all. Under this view, the dedicated chip always wins whenever $e_A < e_T$. The strongest reported number on this axis belongs to CROSS: 451× throughput per watt over CPU and 7.81× over GPU, in Leveraging ASIC AI Chips for Homomorphic Encryption. We reproduce the TPU and CPU energy measurements on the same workloads and batch sizes before building anything on them. We count our reproduction as faithful if we land within PCAM's reported operational error band of 7.44%, used here as a reasonable yardstick.
The strongest existing accounting method is deterministic attributional LCA. It multiplies die area by a fixed per-node factor and amortizes the result evenly over an assumed lifetime, as in LLMCarbon-style estimators and the SCI metric criticized by Bashir et al. We run it on exactly the same inventory, workloads and demand scenarios as our model. The comparison is fair by construction because only the accounting rule differs.
The second strong baseline is unmodified PCAM applied with its commodity CPU, GPU and DRAM parameters. We treat FHE hardware as generic silicon of the same area, without FHE-derived memory requirements.
We beat these baselines if the decision they recommend, fabricate or repurpose, differs from ours in a non-trivial share of realistic scenarios, and if the difference can be traced to a specific modeled factor. If the decisions agree almost everywhere, the extra machinery is not justified and we will report that.
Three ablations isolate the key ingredients: setting $c^{\text{marg}}_T=0$, which is the implicit CROSS assumption; fixing $I$ at the regional average instead of the scheduled value; and replacing the parameter-derived memory size with the published capacity.
Why not just…?
Why not just use a deterministic die-area calculator? This is the real threat. If uncertainty in fab energy and yield is small relative to the gap between $N^{*}$ and lifetime demand, a point estimate gives the same answer. PCAM's reported 23.02% embodied error for deterministic models suggests the uncertainty is material. But material error only matters if it straddles the decision boundary. Our decisive check measures exactly this: the fraction of scenarios whose credible interval for $N^{*}$ contains the lifetime demand. If that fraction is small, we will recommend the deterministic tool.
Why not just compare throughput per watt, since heavily used accelerators are dominated by operational energy? For a fully loaded chip on a dirty grid, this may be right, and the model will show it. However, the file reports that manufacturing accounts for over 60% to 70% of modern systems' lifetime footprint. Private inference is also a niche workload whose utilization is uncertain. Carbon-aware batching then lowers $I$ further, which is precisely the regime where operational comparisons overstate the case for new silicon.
Why not just run everything on repurposed TPUs and skip the analysis? CROSS makes this attractive, and the file reports it as within 3–33× of custom ASICs. But "reuse is free" holds only when TPUs sit idle. Our utilization sweep shows at which fleet load the displacement term makes fabrication competitive. If the TPU route wins even at full commitment, that becomes a strong and simple recommendation.
Why not just amortize manufacturing carbon per inference, as the SCI metric does? The Sunk Carbon Fallacy shows that per-job amortization misdirects scheduling. We keep amortization out of scheduling entirely and confine manufacturing carbon to the one-time procurement inequality. The deterministic baseline above embodies the SCI approach, so the comparison measures what the corrected treatment changes.
Decisive Checks
The workloads are drawn from the file's benchmarks. They are the CryptoNets and LoLa networks on MNIST, CIFAR-10 networks as evaluated by CoFHE and the SoK paper, encrypted logistic regression, and homomorphic AES-128. For AES-128, Circuit Bootstrapping: Faster and Smaller reports 26.2 s single-thread, which makes it a bootstrapping-heavy stress case where dedicated silicon should look best.
The first check is operational validity: measured TPU, CPU and GPU energy must match the reported ratios closely enough to trust $e_T$.
The second check is decision sensitivity. We report, per workload and region, whether $N^{*}$ falls below lifetime demand, with its interval. We also report how often the naive and deterministic baselines disagree with our decision.
The third check is the CoFHE question stated plainly: under the spintronic prior, does its 26% energy saving repay its extra memory manufacturing within lifetime? We report the manufacturing-intensity threshold above which it does not.
The fourth check tests whether carbon-aware batching raises $N^{*}$ enough to flip the bootstrapping-heavy cases. If it does, the finding is that scheduling software substitutes for new silicon.
The file contains no reported lifetime carbon figure for any FHE system, so there is no reported result on Lacuna for embodied or lifetime carbon of FHE accelerators to compare against. Every such number in this study will be new, and it will be presented with its uncertainty.
References
- SoK: Fully Homomorphic Encryption Accelerators
- CoFHE: Software and hardware Co-design for FHE-based machine learning as a service
- Leveraging ASIC AI Chips for Homomorphic Encryption
- Unveiling the Uncertainty in Embodied and Operational Carbon of Large AI Models through a Probabilistic Carbon Accounting Model
- The Sunk Carbon Fallacy: Rethinking Carbon Footprint Metrics for Effective Carbon-Aware Scheduling
- A view of the sustainable computing landscape
- CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy
- Circuit Bootstrapping: Faster and Smaller