Neural Inference via Pre-Computed Look-Up Tables

This direction is merged into another direction: embedding caching, exponential memory scaling. Its last version stays readable and citable here.

Version 1 of 130 papers44 conceptsfrozen 2026-10-04

The fundamental bottleneck in deploying deep learning to edge devices is the sheer density of floating-point operations (FLOPs). Every forward pass requires millions of matrix-vector multiplications, which consume significant power and time. However, for many tasks, the input space is either discrete or can be effectively discretized. This observation allows us to "memoize" the neural network—pre-calculating its outputs and storing them in Look-Up Tables (LUTs) to replace active computation with simple memory retrieval.

The Memoization of Neural Layers

At its core, a neural network layer is a deterministic function $y = f(x)$. If the set of possible inputs $x$ is finite and small, we can compute $f(x)$ for every possible $x$ ahead of time. During inference, instead of executing a series of dot products and activations, the system simply uses the input as an index to fetch the pre-calculated result from memory.

In Natural Language Processing, this is particularly effective because the input vocabulary is finite. Jacob Devlin, Chris Quirk, and Arul Menezes demonstrated in Pre-Computable Multi-Layer Neural Network Language Models that the dot products between fixed input embeddings and the first hidden layer weights can be pre-calculated. This "pre-computation trick" transforms the initial matrix multiplication into a series of vector additions or simple table lookups, achieving a 10x speedup. Similarly, Christopher D Manning and Danqi Chen applied this to dependency parsing in A Fast and Accurate Dependency Parser using Neural Networks, caching the contributions of frequent words to the hidden layer to process over 1,000 sentences per second.

Overcoming the Curse of Dimensionality

While LUTs are incredibly fast, they suffer from the "exponential curse": a table for an $N$-pixel patch with $P$ possible values requires $P^N$ entries. To make this practical for image processing, researchers use small receptive fields and interpolation.

In Practical Single-Image Super-Resolution Using Look-Up Table, Younghyun Jo and Seon Joo Kim showed that by caching outputs for all possible local $2 \times 2$ patches, a model can perform upscaling faster than standard bicubic interpolation while maintaining neural-level quality. To handle larger receptive fields without the memory explosion, Cheng Ma and colleagues introduced Learning Series-Parallel Lookup Tables for Efficient Image Super-Resolution (SPLUT). By structuring LUTs in series and parallel, they decompose a large input space into smaller, manageable dimensions, allowing the model to "see" more of the image without requiring gigabytes of storage.

Shifting from Computation to Memory

The transition to LUT-based inference represents a fundamental trade-off: we spend memory (RAM and storage) to save computation (FLOPs). This is ideal for modern hardware where memory access is often more efficient than high-intensity mathematical operations, especially on CPUs or low-power IoT devices.

Recent work has pushed this paradigm into increasingly complex domains:

Learning the Table Directly

The most recent evolution in this field is moving away from "training then converting" to "learning the table." Instead of distilling a pre-trained CNN into a table, researchers are designing architectures where the LUTs themselves are the learnable parameters.

Zhongnan Cai and colleagues demonstrated this in Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables, which processes massive satellite images in milliseconds. Similarly, Sidi Yang and Binxiao Huang introduced DnLUT, a channel-aware LUT framework for denoising that uses only 0.1% of the energy required by standard convolutional networks. By making the tables learnable, these models can be optimized directly for the hardware constraints they will face during deployment, ensuring that the memory-retrieval process captures the most relevant features for the task at hand.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_inference_via_pre_computed_look_u_6a638612,
  title = {Neural Inference via Pre-Computed Look-Up Tables},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-inference-via-pre-computed-look-up-tables/rdr_6a63861244204a418baaebfacb0bd371},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_inference_via_pre_computed_look_u_6a638612_v1,
  title = {Neural Inference via Pre-Computed Look-Up Tables},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-inference-via-pre-computed-look-up-tables/rdr_6a63861244204a418baaebfacb0bd371/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}