No article text was recorded for this version.
memory bandwidth bottleneck (memory-bound inference)
Version 2 of 2548 papers676 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
- Blog: AI and Memory Wall
- Blog: Survey and Evaluation of Converging Architecture in LLMs based on Footsteps of Operations
- Blog: FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
- Blog: Harnessing On-Device Large Language Model: Empirical Results and Implications for AI PC
- Blog: Learning to Compress Prompts with Gist Tokens
- Blog: RelayAttention for Efficient Large Language Model Serving with Long System Prompts
- Blog: FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
- Blog: Fast Matrix Multiplications for Lookup Table-Quantized LLMs
- Blog: Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
- Blog: LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
- Blog: ETS: Efficient Tree Search for Inference-Time Scaling
- Blog: Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy
- Blog: Efficient Large Language Models: A Survey
- Blog: High-throughput Generative Inference of Large Language Models with a Single GPU
- Blog: FBQuant: FeedBack Quantization for Large Language Models
- Blog: SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
- Blog: MELTing point: Mobile Evaluation of Language Transformers
- Blog: Accelerating Large Language Model Decoding with Speculative Sampling
- Blog: LLM Inference Unveiled: Survey and Roofline Model Insights
- Blog: Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
- Blog: From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation up to 100K Tokens
- Blog: Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
- Blog: BASS: Batched Attention-optimized Speculative Sampling
- Blog: FBQuant: FeedBack Quantization for Large Language Models
- Blog: Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
- Blog: Dynamic-Width Speculative Beam Decoding for LLM Inference
- Blog: Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
- Blog: Faster LLM Inference using DBMS-Inspired Preemption and Cache Replacement Policies
- Blog: Faster and Better LLMs via Latency-Aware Test-Time Scaling
- Blog: Fast Inference from Transformers via Speculative Decoding
- Blog: Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
- Blog: Small Language Models: Survey, Measurements, and Insights
- Blog: Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
- Blog: Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
- Blog: A Review on Edge Large Language Models: Design, Execution, and Applications
- Blog: A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
- Blog: Hydragen: High-Throughput LLM Inference with Shared Prefixes
- Blog: Accelerating LLM Inference with Staged Speculative Decoding
- Blog: Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
- Blog: Single Stream Parallelization of Recurrent Neural Networks for Low Power and Fast Inference
- Blog: Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
- Blog: EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
- Blog: Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM
- Blog: The Effect of Scheduling and Preemption on the Efficiency of LLM Inference Serving
- Blog: SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification
- Blog: CO2-Meter: A Comprehensive Carbon Footprint Estimator for LLMs on Edge Devices
- Blog: Shortened LLaMA: A Simple Depth Pruning for Large Language Models
- Blog: Ripple: Accelerating LLM Inference on Smartphones with Correlation-Aware Neuron Management
- Blog: FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
- Blog: CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
- Blog: Multi-Token Joint Speculative Decoding for Accelerating Large Language Model Inference
- Blog: Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
- Blog: Accelerating Large Language Model Inference via Early-Exiting Algorithms
- Blog: Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
- Blog: Optimized Multi-Token Joint Decoding With Auxiliary Model for LLM Inference
- Blog: PaSS: Parallel Speculative Sampling
- Blog: Kinetics: Rethinking Test-Time Scaling Law
- Blog: Stochastic Sparse Attention for Memory-Bound Inference
- Blog: BASS: Batched Attention-optimized Speculative Sampling
Showing 60 of 548 papers.
History & citation
Current · Permanent link
- Parallel Verification for Memory-Bound LLM Inference merged into this direction (84% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_memory_bandwidth_bottleneck_memory_bound_80800511,
title = {memory bandwidth bottleneck (memory-bound inference)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/memory-bandwidth-bottleneck-memory-bound-inference/rdr_808005116c2647aebf33e1df0807a38c},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_memory_bandwidth_bottleneck_memory_bound_80800511_v2,
title = {memory bandwidth bottleneck (memory-bound inference)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/memory-bandwidth-bottleneck-memory-bound-inference/rdr_808005116c2647aebf33e1df0807a38c/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}