Modern deep learning is built on the back of floating-point arithmetic. Whether it is a ResNet-101 or a Llama-3, these models rely on high-precision decimals to represent weights and activations. However, the world of cryptography—specifically Zero-Knowledge Proofs (ZKP), Secure Multi-Party Computation (MPC), and Homomorphic Encryption (HE)—operates almost exclusively over finite fields and integer rings.
This fundamental mismatch creates a "precision gap." To verify a model's output without seeing its weights, or to compute on encrypted data, researchers must translate the continuous math of neural networks into the discrete math of cryptography. The recurring solution across the field is the systematic use of integer quantization and fixed-point arithmetic.
The Arithmetic Mismatch
The primary challenge, as noted by Wenjie Qu and Zonghui Wang in An Efficient and Extensible Zero-knowledge Proof Framework for Neural Networks, is that floating-point operations are inherently incompatible with the algebraic structures used in ZKPs. Simulating a single floating-point multiplication inside a cryptographic circuit is prohibitively expensive, often requiring "bit decomposition" that slows down the system by orders of magnitude.
To solve this, researchers map real-valued parameters to integers. In CryptoNets, Ran Gilad-Bachrach and Kristin Lauter demonstrated that scaling floating-point weights by a large constant and rounding them to the nearest integer allows homomorphic encryption schemes to perform standard neural network arithmetic. This "fixed-point" approach treats the last few bits of an integer as the fractional part, enabling decimal-like behavior using only integer logic.
Bridging the Gap with Fixed-Point Scaling
The intuition behind this transformation is simple: if you multiply every number in your network by $2^{10}$ (a scaling factor), a value like $0.1234$ becomes $126$. You can then perform additions and multiplications as usual. The "cost" is that every multiplication doubles the scaling factor (e.g., $2^{10} \times 2^{10} = 2^{20}$), requiring periodic "truncation" or bit-shifting to keep the numbers within the bounds of the cryptographic field.
This technique is now standard across diverse privacy-preserving tasks:
- Verifiable Inference: Daniel Kang and Yi Sun showed in Scaling up Trustless DNN Inference with Zero-Knowledge Proofs that 8-bit quantized representations allow for ImageNet-scale verification with negligible accuracy loss.
- Secure Training: Ke Sun and Marcel Keller demonstrated in Secure Quantized Training for Deep Learning that even complex optimizers like Adam can be implemented using fixed-point arithmetic, matching plaintext accuracy within 0.2%.
- Federated Learning: In healthcare settings, Riccardo Taiello and Marc Vesin use uniform quantization to map gradients into finite integer ranges, ensuring they can be securely aggregated by a central server without leaking patient data (Enhancing Privacy in Federated Learning).
Handling Non-Linearity
While linear operations (convolutions and fully connected layers) are easy to compute on quantized integers, non-linear functions like ReLU, Sigmoid, and Softmax are not. Cryptographic systems cannot "see" the values they are processing, so they cannot easily check if a number is greater than zero (the basis of ReLU).
Two main strategies have emerged to handle this:
- Lookup Tables: Kui Ren and Bingsheng Zhang propose treating quantized operators as private lookup tables (A New PPML Paradigm for Quantized Models). If the input is restricted to 8 bits, there are only 256 possible outputs, which can be pre-computed and retrieved securely.
- Polynomial Approximation: Functions are replaced with integer-friendly polynomials. For example, Hongxiao Wang uses piecewise functions that are "friendly" to homomorphic encryption to replace slow Taylor-polynomial approximations, achieving a 10x speedup in SFPDML.
Optimization and Automation
The field is moving toward "push-button" solutions that automate this conversion. CrypTFlow, developed by Nishant Kumar and Rahul Sharma, acts as a compiler that automatically searches for the best precision scale for a given TensorFlow model. Similarly, zkPyTorch by Tiancheng Xie and Dawn Song provides a hierarchical compiler that tailors quantization strategies to specific finite fields, enabling the verification of massive models like Llama-3.
The success of these methods suggests that "integer is enough" (Pengyu Qiu et al., Integer Is Enough). By embracing the constraints of cryptographic arithmetic rather than fighting them, researchers have made verifiable and private AI a practical reality.