Memory-Centric Acceleration for Fully Homomorphic Encryption

This direction is merged into another direction: homomorphic encryption overhead. Its last version stays readable and citable here.

Version 1 of 1117 papers137 conceptsfrozen 2026-10-04

Fully Homomorphic Encryption (FHE) is often described as the "holy grail" of cryptography because it allows arbitrary computation on encrypted data without ever needing a decryption key. In theory, this enables a future where sensitive medical records or proprietary machine learning models can be processed in the cloud with zero risk of exposure.

In practice, however, FHE faces a "performance wall." While early criticisms focused on the raw computational complexity of modular arithmetic, recent research has revealed a more fundamental obstacle: FHE is not compute-bound, but severely memory-bound.

The Data Inflation Problem

The primary reason FHE is slow is data inflation. To maintain security and manage the "noise" that accumulates during mathematical operations, FHE transforms a small piece of plaintext into a massive ciphertext. Louis J. M. Aslett and colleagues have noted that a single dataset can expand by over 16,000 times its original size when encrypted with standard security parameters.

This expansion turns simple operations into logistical nightmares. As Praneeth Vepakomma points out, individual data points can require megabytes of storage, and the resulting runtime memory requirements for even basic statistical models can reach gigabytes. This makes FHE particularly difficult to scale for high-parameter architectures like Transformers, as noted by Siyang Jiang.

The Arithmetic Intensity Gap

In high-performance computing, we often measure efficiency through arithmetic intensity—the ratio of operations performed to bytes of data moved. Standard algorithms (like matrix multiplication in plaintext) are compute-intensive, meaning the processor spends most of its time calculating.

FHE is the opposite. Nandan Kumar Jha and Brandon Reagen have identified that FHE primitives exhibit extremely low arithmetic intensity, often around 0.1 operations per byte. This means for every mathematical operation performed, the system must move ten bytes of data.

Because of this, simply adding more CPU cores or increasing clock speeds provides diminishing returns. As Leo de Castro and Chiraag Juvekar argue, FHE acceleration must be "memory-first." If the memory bandwidth cannot feed the processing units fast enough, those units sit idle, a phenomenon known as "data starvation."

Breaking the Memory Wall

To overcome this bottleneck, researchers are moving away from general-purpose CPUs toward specialized hardware architectures that prioritize data movement.

1. Near-Memory and In-Memory Processing

One of the most direct ways to solve the bandwidth problem is to move the computation closer to the data. Minxuan Zhou proposed FHEmem, a Processing-In-Memory (PIM) accelerator that integrates compute units directly into High Bandwidth Memory (HBM). By eliminating the need to move massive ciphertexts across a slow off-chip bus, PIM architectures can achieve significant speedups and energy efficiency. Similarly, shangyi shi introduced FlexMem, which uses a hierarchical interconnect to reach 95.7% utilization of available near-memory bandwidth.

2. Specialized Dataflows

Standard hardware is optimized for sequential data access, but FHE algorithms like the Number-Theoretic Transform (NTT) require irregular, non-sequential patterns. Austin Ebel developed Osiris, a systolic architecture that uses "limb interleaving" to allow different FHE kernels to operate in lockstep, maximizing the use of every bit of data fetched from memory.

3. Reducing Intermediate State

A major source of memory pressure is the "key switching" operation required after every multiplication. This process generates massive intermediate data structures that can overwhelm on-chip buffers. Negar Neda and Austin Ebel proposed CiFlow, an output-centric dataflow that reorders computations to minimize these intermediate states, reducing on-chip memory requirements by over 12x.

Hardware-Software Co-Design

The most successful approaches today don't just build faster chips; they rethink the algorithms to fit the hardware. M. Sadegh Riazi and Kim Laine demonstrated with HEAX that a highly parallelized NTT engine on an FPGA can achieve a 200x improvement over software by carefully managing the transfer of keys between off-chip and on-chip memory.

Furthermore, Lei Ju suggests that a hybrid memory architecture—combining high-speed SRAM with energy-efficient non-volatile memory—is necessary to handle the massive scratchpad requirements of FHE without excessive power leakage.

The Path Forward

The consensus across the field, from the survey work of Junxue Zhang to the specific implementations like Taiyi (Mingzhe Zhang), is that the "compute vs. memory" trade-off has shifted. While early FHE research was a quest for mathematical feasibility, the current frontier is an architectural battle against data inflation. Practical, real-world FHE will likely not come from a faster processor, but from a system that can move ciphertexts as efficiently as it calculates them.

Papers in this version

Showing 60 of 117 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_memory_centric_acceleration_for_fully_ho_86bee3f3,
  title = {Memory-Centric Acceleration for Fully Homomorphic Encryption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/memory-centric-acceleration-for-fully-homomorphic-encryption/rdr_86bee3f306d746d7bc038debd68d28e0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_memory_centric_acceleration_for_fully_ho_86bee3f3_v1,
  title = {Memory-Centric Acceleration for Fully Homomorphic Encryption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/memory-centric-acceleration-for-fully-homomorphic-encryption/rdr_86bee3f306d746d7bc038debd68d28e0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}