Efficient Inference for Large Language Models –Algorithm, Model, and System

Xuefei Ning, Guohao Dai, Haoli Bai, Lu Hou, Yu Wang, Qun LiuEMNLP 2025

TL;DR

This tutorial addresses the significant computational and memory overhead of Large Language Models (LLMs) during inference. It proposes a comprehensive, full-stack taxonomy of efficiency methods, offering a designer's perspective on diagnosing bottlenecks and selecting optimal strategies across algorithms, models, and systems.

What Problem Does This Paper Address?

Large Language Models (LLMs) have revolutionized natural language processing, but their impressive capabilities come at a steep price: high computational costs, substantial memory requirements, and significant energy consumption during inference. This makes deploying and running LLMs efficiently a major challenge, impacting latency, throughput, and overall accessibility. The authors aim to demystify this complex landscape by providing a structured understanding of the factors contributing to LLM inference inefficiency and presenting a systematic approach to optimizing them.

Methods

The tutorial outlines a "full-stack taxonomy" of efficient inference methods for LLMs. This taxonomy is organized into categories that likely cover algorithmic optimizations (e.g., quantization, pruning, knowledge distillation), model architecture improvements (e.g., efficient attention mechanisms, smaller model variants), and system-level enhancements (e.g., hardware acceleration, optimized memory management, distributed inference strategies). The authors emphasize a designer's perspective, guiding readers on how to diagnose efficiency bottlenecks for specific workloads and hardware configurations. They plan to illustrate each category with representative methods, detailing their design logic and the specific inefficiency factors they target.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • A Designer's Perspective is Crucial: The tutorial highlights the importance of understanding LLM inference from a practical, design-oriented viewpoint, rather than just a theoretical one. This involves diagnosing real-world bottlenecks on specific hardware.
  • Full-Stack Taxonomy: The proposed taxonomy offers a structured and comprehensive way to categorize the diverse range of efficiency methods, providing a roadmap for researchers and practitioners.
  • Bridging the Gap: By covering algorithms, models, and systems, the tutorial aims to bridge the gap between theoretical advancements and practical deployment challenges in LLM inference.

Key Results

Not discussed in the paper. This appears to be a tutorial abstract, not a research paper with experimental results.

Limitations and Future Work

The tutorial acknowledges the complexity of LLM inference and the need for a systematic approach to tackle its inefficiencies. While specific limitations of individual methods are not detailed in this abstract, the authors suggest that future research directions will be discussed, likely focusing on novel algorithmic, model, and system-level innovations to further improve LLM inference efficiency.

Summary updated

BibTeX
@inproceedings{ning2025efficient,
  author = {Xuefei Ning and Guohao Dai and Haoli Bai and Lu Hou and Yu Wang and Qun Liu},
  title = {{Efficient Inference for Large Language Models –Algorithm, Model, and System}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure