When executing deep learning models on encrypted data using Homomorphic Encryption (HE) or Multi-Party Computation (MPC), the traditional deep learning paradigm of modular, layer-by-layer execution breaks down. In plaintext environments, executing a convolution, followed by batch normalization, and then an activation function sequentially is computationally clean. In the secure domain, however, each boundary between layers introduces severe cryptographic penalties: noise accumulation in HE, and network round-trips or representation conversions in MPC.
To make privacy-preserving machine learning (PPML) practical, researchers have shifted toward operator fusion. Instead of treating a neural network as a chain of isolated blocks, this paradigm mathematically or cryptographically collapses consecutive operations into a single unified step. By expressing a sequence of operations as a single composite function, protocols can bypass the intermediate cryptographic overhead that typically cripples secure inference performance.
Controlling Noise Growth in Homomorphic Encryption
In Leveled Homomorphic Encryption, ciphertexts contain a small amount of cryptographic noise that grows with every multiplication operation. If this noise exceeds a certain threshold, the data becomes uncorreadable, forcing the system to perform an incredibly expensive "bootstrapping" operation to refresh the ciphertext. The number of sequential multiplications on the critical path is known as the multiplicative depth. Minimizing this depth is the primary way to avoid bootstrapping and reduce latency.
Operator fusion achieves this by algebraically combining adjacent layers before they are evaluated in the encrypted domain. In the foundational work CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy, Ran Gilad-Bachrach et al. demonstrated that consecutive linear operations—such as a convolution followed by a mean pooling layer—can be mathematically collapsed into a single linear transformation. Because the two layers are evaluated as one, the circuit's multiplicative depth drops, preventing the accumulation of computational noise.
This principle extends to non-linear layers when activations are approximated using polynomials. Qian Lou in HEMET: A Homomorphic-Encryption-Friendly Privacy-Preserving Mobile Neural Network Architecture showed that constants from plaintext masking, polynomial activations, and batch normalization can be pre-computed and merged into a single set of coefficients. Instead of performing sequential multiplications on ciphertexts for each distinct layer, the system executes them in a single pass.
Similarly, Hongwu Peng et al. in LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference and Eunsang Lee et al. in Low-Complexity Deep Convolutional Neural Networks on Fully Homomorphic Encryption Using Multiplexed Parallel Convolutions focused on fusing batch normalization directly into preceding convolution operations. By eliminating the independent multiplication levels required by standalone normalization layers, these frameworks delay bootstrapping steps and unlock substantial speedups for deep architectures like ResNets and GCNs. This structural optimization is echoed by Francisco Vargas et al. in Efficient Representations for Privacy-Preserving Inference, where composing convolutions and pooling into a single homomorphic operation minimizes the overall instruction depth.
Eliminating Communication Rounds and Representation Shifts in MPC
While HE suffers from noise growth, Multi-Party Computation (MPC) is primarily bottlenecked by network latency. In MPC, computing a non-linear activation function or transitioning between different types of layers requires the participating parties to interact. This introduces two major overheads: communication rounds (network round-trips) and representation transformations (e.g., converting secret shares from a linear arithmetic ring to a binary ring).
Operator fusion mitigates these bottlenecks by allowing multiple layers to be synchronized or evaluated simultaneously. For instance, Mengxin Zheng et al. developed Primer: Fast Private Transformer Inference on Encrypted Data, which consolidates adjacent transformer layers into unified computation blocks. This consolidation minimizes the synchronization overhead between the client and server, allowing private inference to scale to complex architectures without a linear penalty in network latency. A similar strategy is employed by the authors of PrivCirNet: Efficient Private Inference via Block Circulant Transformation, who fuse entire residual blocks into single computational units to reduce cryptographic rounds.
Beyond reducing round counts, fusion can eliminate representation transformations entirely. In Leia: A Lightweight Cryptographic Neural Network Inference System at the Edge, Xiaoning Liu 0002 et al. integrated batch normalization and binary activation into a single secure protocol step. This fusion completely avoids the expensive ring conversions that typically occur between linear and non-linear layers in multi-party settings. For non-neural workloads like decision trees, Aikaterini Mitrokotsa et al. in Constant-Round Private Decision Tree Evaluation for Secret Shared Data applied this concept by fusing share conversion and logical operations into a single primitive, processing comparison results and logical gates simultaneously.
Multi-Layer Lookup Consolidation
A radical extension of this idea is to replace arithmetic circuits entirely with private lookup tables (LUTs). Instead of computing a convolution and then an activation, quantized network operators can be treated as tabular mappings.
Kui Ren et al. introduced this paradigm in A New PPML Paradigm for Quantized Models, demonstrating that sequential operations—such as convolution, activation, and pooling—can be consolidated into a single private lookup table. As a result, an entire sequence of layers can be evaluated for the communication cost of a single query, drastically reducing round complexity.
Traditional Layer-by-Layer MPC:
[Conv] --> (Network Round-Trip) --> [Activation] --> (Network Round-Trip) --> [Pooling]
Fused Private Lookup Table:
[ Fused Conv + Activation + Pooling LUT ] --> (Single Network Query)
To make these lookup tables practical for large-scale models, Lifan Zhao et al. in Secure Outlier-Aware Large Language Model Inference showed that isolating outlier values allows the system to shrink the input domain for activation functions. This domain reduction simplifies the underlying lookup tables and iterative approximations, accelerating complex operations like Softmax and RMSNorm with fewer computational rounds.
Furthermore, cross-layer fusion can eliminate other expensive cryptographic primitives. Qiao Zhang et al. in United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus treated consecutive layers as a single composite function, allowing the protocol to completely eliminate homomorphic encryption rotations across layer boundaries—one of the most computationally taxing operations in hybrid HE-MPC systems.
Low-Level and Hardware-Level Fusion
The philosophy of operator fusion is not restricted to high-level neural network layers; it operates effectively at the level of cryptographic primitives and hardware instruction sets.
At the primitive level, Riad S. Wahby in Fast and simple constant-time hashing to the BLS12-381 elliptic curve demonstrated that combining field inversion and square root calculations into a single exponentiation operation significantly reduces the total number of expensive field operations. This fusion is highly effective for constant-time implementations where branching must be avoided to prevent side-channel leakage. Similarly, in industrial computing environments, Zheng Yang et al. in PLCrypto: A Symmetric Cryptographic Library for Programmable Logic Controllers combined substitution and permutation layers into unified lookup and assignment sequences to maintain real-time performance in scan-based processing environments.
At the hardware architecture level, Yi Huang 0036 et al. designed EFFACT: A Highly Efficient Full-Stack FHE Acceleration Platform, which integrates auxiliary maintenance operations—such as Montgomery transformations and constant scaling—directly into primary computational steps like base conversion. By merging these maintenance tasks into the core arithmetic pipeline, the hardware eliminates redundant instruction cycles and data movement.
Additionally, protocol-level co-optimization can compress multiple distinct cryptographic steps. Tianshi Xu et al. in PrivQuant: Communication-Efficient Private Inference with Quantized Network/Protocol Co-Optimization mathematically merged truncation and bit-extension operations into a single step. This eliminates redundant communication rounds that typically occur when handling fixed-point precision shifts between linear and non-linear layers.
Summary
Operator fusion represents a fundamental shift from modular security design to holistic security design. By analyzing the entire computational graph and collapsing adjacent operations, researchers are able to bypass the intrinsic bottlenecks of HE and MPC. Whether by reducing multiplicative depth to control noise, eliminating network round-trips via multi-layer lookup tables, or merging low-level mathematical transformations, operator fusion provides the structural optimization necessary to make secure, private computing practical.
Key Foundational Work
CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy (2016) — contributed 2 concept elements to this direction.
Fast and simple constant-time hashing to the BLS12-381 elliptic curve (2019) — contributed 2 concept elements to this direction.