GPU Acceleration of TFHE-Based High-Precision Nonlinear Layers for Encrypted LLM Inference

Guoci Chen, Xiurui Pan, Qiao Li, Bo Mao, Congming Gao, Chengying Huan, Mingzhe Zhang, Jie ZhangarXiv 2026 · DOI

TL;DR

This paper introduces TIGER, a novel GPU-accelerated framework that enables high-precision inference for nonlinear layers in encrypted Large Language Models (LLMs) using TFHE. TIGER significantly speeds up these critical computations, making privacy-preserving LLM deployment more practical.

What Problem Does This Paper Address?

Deploying LLMs as cloud services raises significant privacy concerns, as sensitive user data can be exposed during inference. Fully Homomorphic Encryption (FHE) offers a solution by allowing computations on encrypted data. However, existing FHE schemes struggle with efficiently and accurately evaluating the nonlinear functions (like GELU, Softmax, LayerNorm) crucial for LLM performance. While CKKS-based methods require high-degree polynomial approximations for these functions, leading to high computational costs, TFHE's Programmable Bootstrapping (PBS) offers exact lookup-table (LUT) evaluation. Despite this advantage, TFHE implementations have lacked high-precision nonlinear layer support and haven't effectively utilized GPU parallelism, creating a bottleneck for encrypted LLM inference.

Methods

TIGER tackles these challenges by:

  • GPU-Optimized WoP-PBS: It leverages the Without-Padding PBS (WoP-PBS) technique, enhanced with numerical algorithms, to achieve higher precision in LUT evaluations than native capabilities allow.
  • High-Precision Nonlinear Layer Implementations: TIGER provides efficient and precise implementations of key nonlinear layers (GELU, Softmax, LayerNorm) by combining WoP-PBS with numerical refinement techniques (like Taylor expansion and range reduction).
  • Batch-Driven Design for GPU Efficiency: The framework exploits inter-input parallelism by processing multiple LLM inputs simultaneously in batches. This, along with hardware-aware optimizations like intelligent batch scheduling and memory coalescing, maximizes GPU utilization and throughput.
  • Modular Architecture: TIGER is structured into modules for composite nonlinear layers, high-precision function evaluation, fixed-point operators, and low-level TFHE primitives, allowing for flexibility and extensibility.

Datasets

Not discussed in the paper. The evaluation focuses on the performance of specific LLM components (GELU, Softmax, LayerNorm) and a GPT-2 Transformer block, rather than training on specific datasets.

Key Insights and Ideas

  • Nonlinear Layers are the Bottleneck: The paper strongly emphasizes that nonlinear layers are far more sensitive to precision reduction than linear layers in LLMs. Aggressively quantizing them severely degrades model accuracy, making high-precision evaluation essential for encrypted inference.
  • TFHE's Potential for Nonlinearity: TFHE, with its exact LUT evaluation via PBS, is a more natural fit for high-precision nonlinear functions than CKKS's polynomial approximations, which become prohibitively expensive.
  • GPU Parallelism is Key: Effectively leveraging GPU's massive parallelism is crucial for making encrypted LLM inference practical. TIGER's batch-driven design and optimizations are central to achieving significant speedups.
  • Co-design is Essential: The success of TIGER comes from the co-design of cryptographic primitives (WoP-PBS), numerical methods, and GPU parallelization, bridging the gap between low-level FHE operations and high-level LLM layers.

Key Results

  • Layer-wise Speedups: TIGER achieves substantial speedups over a CPU baseline for individual nonlinear layers: 7.17x for GELU, 16.68x for Softmax, and 17.05x for LayerNorm.
  • End-to-End Speedup: For a full GPT-2 Transformer block, TIGER delivers an impressive 15.54x speedup.
  • Precision vs. Performance Trade-off: The paper demonstrates that while aggressive quantization of nonlinear layers drastically increases perplexity (e.g., 668.18x for LayerNorm at 4-bit), TIGER's approach maintains high precision with significant performance gains.
  • Ablation Study Highlights: The KSPBS split strategy provides the largest performance gain (25-30%), followed by Karatsuba multiplication (5-10%) and 4-radix FFT optimization (5-8%).

Limitations and Future Work

  • TFHE Parameter Tuning: The paper notes that the optimal batch size for PBS operations needs careful tuning, and while they use an offline profiling approach, adaptive strategies could further enhance performance.
  • Multi-GPU Scaling: While TIGER is designed to scale to multiple GPUs, the paper suggests further exploration of owner-computes partitioning and bulk-synchronous parallel execution models.
  • Broader Applicability: The authors believe TIGER's core design is applicable to other privacy-preserving inference workloads requiring high-precision nonlinear computation, such as encrypted probabilistic prediction and scientific analysis.

Summary updated

Abstract

Deploying large language models (LLMs) as cloud services raises privacy concerns as inference may leak sensitive data. Fully Homomorphic Encryption (FHE) allows computation on encrypted data, but current FHE methods struggle with efficient and precise nonlinear function evaluation. Specifically, CKKS-based approaches require high-degree polynomial approximations, which are costly when target precision increases. Alternatively, TFHE's Programmable Bootstrapping (PBS) outperforms CKKS by offering exact lookup-table evaluation. But it lacks high-precision implementations of LLM nonlinear layers and underutilizes GPU resources. We propose \emph{TIGER}, the first GPU-accelerated framework for high-precision TFHE-based nonlinear LLM layer evaluation. TIGER offers: (1) GPU-optimized WoP-PBS method combined with numerical algorithms to surpass native lookup-table precision limits on nonlinear functions; (2) high-precision and efficient implementations of key nonlinear layers, enabling practical encrypted inference; (3) batch-driven design exploiting inter-input parallelism to boost GPU efficiency. TIGER achieves 7.17$\times$, 16.68$\times$, and 17.05$\times$ speedups over a CPU baseline for GELU, Softmax, and LayerNorm, respectively.
BibTeX
@misc{chen2026gpu,
  author = {Guoci Chen and Xiurui Pan and Qiao Li and Bo Mao and Congming Gao and Chengying Huan and Mingzhe Zhang and Jie Zhang},
  title = {{GPU Acceleration of TFHE-Based High-Precision Nonlinear Layers for Encrypted LLM Inference}},
  year = {2026},
  eprint = {2604.04783},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.04783},
}
Figure