QDA: Closed-Loop LLM Quantization Design with Agents

Xingyang Li, Tianlang Zhao, Song Han, Ligeng ZhuarXiv 2026 · arXiv

Quantizing Large Language Models (LLMs)—the process of shrinking their weights from 16-bit to 4-bit or even 3-bit—is essential for making them run on consumer hardware or reducing data center costs. However, finding the right "recipe" of quantization techniques is often considered a black art. A method that works perfectly for a Llama model might fail miserably on a Qwen model or at a different bit-precision.

In a new paper, researchers from MIT and NVIDIA introduce QDA (Closed-Loop LLM Quantization Design with Agents), a system that uses an LLM agent to automatically design and refine quantization recipes tailored to specific models and hardware.

TL;DR

QDA automates the design of quantization recipes by using an LLM agent to search through a library of techniques, guided by a fast KL-divergence-based evaluator. It consistently outperforms the best hand-designed public recipes, recovering up to 98.3% of original accuracy on massive 320B-parameter models with zero inference overhead.

What Problem Does This Paper Address?

Quantization isn't just one algorithm; it’s a "recipe" involving transformations (like rotations), rounding methods (like GPTQ), and calibration data. The challenge is threefold:

  1. Recipe Sensitivity: The best recipe changes depending on the model architecture and the target precision. There is no "one size fits all."
  2. Evaluation Cost: Modern "reasoning" models (like DeepSeek-R1) generate long chains of thought. A single full evaluation can take nearly 90 GPU-hours, making iterative search impossible.
  3. Engineering Friction: Most quantization tools are siloed. Mixing a technique from one codebase with another often requires manual, error-prone porting.
Figure 1. QDA beats the strongest public recipe at low cost. Left: average accuracy of four reasoning models at each precision, for the strongest public recipe and for QDA. Middle: QDA check- points run on existing kernels with no inference overhead. Right: the GPU-hours and agent tokens of one search.

Methods

QDA treats quantization design as a closed-loop optimization problem managed by an LLM agent (specifically Claude 3.5 Opus). The system consists of three main pillars:

  • The Agentic Loop: The agent proposes a recipe, the system builds it, and the evaluator scores it. The agent then reads the logs, diagnoses failures (like repetition loops), and adjusts the recipe for the next round.
  • Fast Proxy Evaluation: To avoid the 90-hour evaluation bottleneck, QDA scores candidates using KL divergence. It measures how much the quantized model's token predictions differ from the original BF16 model on a set of cached "reference traces." This process takes minutes rather than hours and is highly correlated with final accuracy.
  • Composable Library: The authors implemented 21 different Post-Training Quantization (PTQ) methods (including GPTQ, AWQ, and various rotations) behind a unified interface. This allows the agent to "stack" methods—for example, applying a Hadamard rotation followed by GPTQ and then scale distillation.
The QDA loop: Propose, Build, Score by KL, and Check via generation.

Datasets

The researchers evaluated QDA across four reasoning models: Qwen3.5-9B, Qwen3.6-35B-A3B (MoE), Qwen3.8-27B, and DeepSeek-R1-Distill-Llama-8B.

Evaluation benchmarks included:

  • Math: AIME 2025 and HMMT.
  • Science: GPQA-Diamond.
  • Code: LiveCodeBench v6.
  • Instruction Following: IFBench and IFEval.

Calibration was performed on a mix of standard datasets (UltraChat) and "self-generated" data where the model reasons on its own.

Key Insights and Ideas

  • KL Divergence is a Superior Signal: The authors found that KL divergence on reference traces agrees with final accuracy 82% of the time, compared to only 73% for standard layer-wise error metrics. This makes it a reliable "compass" for the agent.
  • The Agent as a Diagnostician: KL divergence can miss "catastrophic" failures like infinite repetition loops. QDA includes a "generation check" step where the agent runs full answers for promising candidates. If it sees a model looping, it can specifically add "scale distillation" or change calibration data to fix it.
  • No Free Lunch in Recipes: Figure 2 highlights how the "best" public method swaps places depending on the model. For instance, AWQ beats GPTQ on Qwen3.5 at 4-bit, but loses to it at 3-bit. QDA's agent successfully navigates these flips.
Figure 2. The best recipe differs across models and precisions, and a full evaluation costs hundreds of times more than a KL score. (a) Accuracy change from BF16 of three public recipes and QDA at W4A16 on four models. A star marks the most accurate public recipe. On Q3.5-9B at W3A16, AWQ and GPTQ swap order. (b) Accuracy change at W4A8 of FP8-aware GPTQ, which fits GPTQ’s codes to the kernel’s FP8 scales instead of BF16 scales. This helps Q3.5-9B but hurts the other three. (c) Left: share of 44 checkpoint pairs that each measure ranks in the same order as accuracy. Right: GPU-minutes to compute each measure on one R1-8B checkpoint. Q3.5-9B, Q3.6-35B, Q3.8-27B and R1-8B denote Qwen3.5-9B, Qwen3.6-35B-A3B, Qwen3.8-27B and DeepSeek-R1-Distill-Llama-8B.

Key Results

  • Accuracy Gains: Across four models, QDA improved average accuracy over the strongest public recipes by 0.9, 1.4, and 2.2 percentage points at W4A16, W4A8, and W3A16 precisions, respectively.
  • Scaling to 320B Parameters: On the massive GLM-5.3-Flash model using the new NVFP4 format, QDA recovered 98.3% of the original FP8 model's accuracy, outperforming official vendor-released checkpoints from NVIDIA and Red Hat AI.
  • Efficiency: A full search costs only 64 GPU-hours and roughly $53 in agent tokens. Because QDA uses standard formats, the resulting models run on existing high-performance kernels (like Marlin or CUTLASS) with zero inference overhead.

Limitations and Future Work

While QDA significantly reduces human effort, it isn't entirely "hands-off" for brand-new architectures. A small amount of human-written code (an "adapter") is still needed to help the library understand the connectivity of a completely novel model type.

Future work could involve allowing the agent to design the kernels themselves or expanding the search space to include mixed-precision quantization, where different layers use different bit-widths.

Paper Title: QDA: Closed-Loop LLM Quantization Design with Agents
Authors: Xingyang Li, Tianlang Zhao, Song Han, Ligeng Zhu (MIT & NVIDIA)

Summary updated

Abstract

Post-training quantization is the standard way to serve large language models at low cost. However, the best quantization recipe differs across models and precisions. As a result, experts must redesign it for every new model, which is time-consuming and often sub-optimal. Fortunately, an LLM agent can take over this design by refining the recipe over rounds of building and evaluating candidates, so that more rounds lead to better recipes. However, evaluating one quantized reasoning model takes tens of GPU-hours, and combining methods from different codebases means porting each to the new architecture by hand. We present QDA, a system in which an agent designs the recipe for each model and precision. To make evaluation fast, QDA scores each candidate in minutes by its output KL divergence from the original model on reasoning traces that the agent never sees, and checks promising ones with long generations to catch failures such as repetition loops. To make building fast, QDA provides a library of public PTQ methods that are compatible with one another and with new model architectures. Across four reasoning models, QDA improves accuracy over the strongest public recipe by 0.9, 1.4 and 2.2 percentage points on average at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. On the 320B-parameter MoE model GLM-5.3-Flash at NVFP4 W4A4, it recovers 98.3% of the FP8 model's accuracy, against 96.9% for the best vendor checkpoint. Every QDA checkpoint also runs on existing serving kernels with no inference overhead. We will release the library, the evaluator and the harness.
BibTeX
@misc{li2026qda,
  author = {Xingyang Li and Tianlang Zhao and Song Han and Ligeng Zhu},
  title = {{QDA: Closed-Loop LLM Quantization Design with Agents}},
  year = {2026},
  eprint = {2611.23388},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23388},
}
Figure