The fundamental bottleneck in deploying deep learning to edge devices is the sheer density of floating-point operations (FLOPs). Every forward pass requires millions of matrix-vector multiplications, which consume significant power and time. However, for many tasks, the input space is either discrete or can be effectively discretized. This observation allows us to "memoize" the neural network—pre-calculating its outputs and storing them in Look-Up Tables (LUTs) to replace active computation with simple memory retrieval.
The Memoization of Neural Layers
At its core, a neural network layer is a deterministic function $y = f(x)$. If the set of possible inputs $x$ is finite and small, we can compute $f(x)$ for every possible $x$ ahead of time. During inference, instead of executing a series of dot products and activations, the system simply uses the input as an index to fetch the pre-calculated result from memory.
In Natural Language Processing, this is particularly effective because the input vocabulary is finite. Jacob Devlin, Chris Quirk, and Arul Menezes demonstrated in Pre-Computable Multi-Layer Neural Network Language Models that the dot products between fixed input embeddings and the first hidden layer weights can be pre-calculated. This "pre-computation trick" transforms the initial matrix multiplication into a series of vector additions or simple table lookups, achieving a 10x speedup. Similarly, Christopher D Manning and Danqi Chen applied this to dependency parsing in A Fast and Accurate Dependency Parser using Neural Networks, caching the contributions of frequent words to the hidden layer to process over 1,000 sentences per second.
Overcoming the Curse of Dimensionality
While LUTs are incredibly fast, they suffer from the "exponential curse": a table for an $N$-pixel patch with $P$ possible values requires $P^N$ entries. To make this practical for image processing, researchers use small receptive fields and interpolation.
In Practical Single-Image Super-Resolution Using Look-Up Table, Younghyun Jo and Seon Joo Kim showed that by caching outputs for all possible local $2 \times 2$ patches, a model can perform upscaling faster than standard bicubic interpolation while maintaining neural-level quality. To handle larger receptive fields without the memory explosion, Cheng Ma and colleagues introduced Learning Series-Parallel Lookup Tables for Efficient Image Super-Resolution (SPLUT). By structuring LUTs in series and parallel, they decompose a large input space into smaller, manageable dimensions, allowing the model to "see" more of the image without requiring gigabytes of storage.
Shifting from Computation to Memory
The transition to LUT-based inference represents a fundamental trade-off: we spend memory (RAM and storage) to save computation (FLOPs). This is ideal for modern hardware where memory access is often more efficient than high-intensity mathematical operations, especially on CPUs or low-power IoT devices.
Recent work has pushed this paradigm into increasingly complex domains:
- Video Processing: Zefan Qu and others developed Online Video Quality Enhancement with Spatial-Temporal Look-Up Tables (ST-LUT), which uses temporal information to enhance video at 40 FPS on consumer hardware.
- Large Language Models: In Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens, Ziqian Zeng and colleagues use a lightweight trigram-based LUT to "draft" tokens for LLMs, reducing the total computation required for generation.
- Sensor Data: Yusuke Sekikawa and Hideo Saito applied this to event cameras in EventNet, where the discrete nature of pixel coordinates and polarities makes them perfect candidates for pre-computed indexing.
Learning the Table Directly
The most recent evolution in this field is moving away from "training then converting" to "learning the table." Instead of distilling a pre-trained CNN into a table, researchers are designing architectures where the LUTs themselves are the learnable parameters.
Zhongnan Cai and colleagues demonstrated this in Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables, which processes massive satellite images in milliseconds. Similarly, Sidi Yang and Binxiao Huang introduced DnLUT, a channel-aware LUT framework for denoising that uses only 0.1% of the energy required by standard convolutional networks. By making the tables learnable, these models can be optimized directly for the hardware constraints they will face during deployment, ensuring that the memory-retrieval process captures the most relevant features for the task at hand.