MPQ: A Message-Passing View of Post-Training Quantization

Yanshuo Chen, Chenxi Liu, Reza Shirkavand, Heng HuangarXiv 2026 · arXiv

TL;DR

MPQ (Message-Passing Quantization) rethinks post-training quantization as a probabilistic inference problem rather than a greedy search. By using message passing to maintain "soft" beliefs about weight assignments before committing to a final value, it significantly reduces error in ultra-low-bit (2-3 bit) large language model compression.

What Problem Does This Paper Address?

To run massive Large Language Models (LLMs) on consumer hardware, we must compress their weights—often from 16-bit floats down to just 2, 3, or 4 bits. This process, called Post-Training Quantization (PTQ), is tricky because changing one weight affects the optimal value for its neighbors.

Existing state-of-the-art solvers like GPTQ or LDLQ are "greedy." They process weight blocks sequentially, making an immediate "hard decision" on the best discrete codeword for a block and then trying to compensate for the resulting error in subsequent blocks. In the aggressive low-bit regime (e.g., 2 bits), there are very few codewords to choose from. A greedy choice that looks good locally might be globally disastrous, with no way for the algorithm to "backtrack" or reconsider.

Methods

The authors propose Message-Passing Quantization (MPQ), which treats PTQ as a posterior inference problem. Instead of picking one codeword immediately, the algorithm maintains a "soft" probability distribution over multiple likely candidates for every block.

The method consists of two main stages:

  1. Finite-Temperature VAMP: It uses Vector Approximate Message Passing (VAMP) to let blocks "talk" to each other. Instead of passing a single hard value, blocks pass "messages" representing their current beliefs about which codewords are likely. This allows the solver to explore uncertainty and find a better global basin before committing.
  2. Hard Refinement: Once the soft updates converge, the algorithm projects the distributions to the most likely hard codewords (Marginal MAP) and performs a final "polishing" step using Block Coordinate Descent (CD) to ensure local optimality.
Comparison between greedy quantization and the MPQ message-passing approach.

This framework unifies existing solvers: the authors show that GPTQ and LDLQ are essentially "zero-temperature" versions of MPQ that discard all uncertainty information instantly.

Datasets

The researchers evaluated MPQ across several popular model families, including Llama (1B to 70B), Qwen3 (8B), and Mistral (7B).

  • Calibration: 128 sequences from the C4 dataset.
  • Evaluation: Perplexity was measured on WikiText-2 and C4.
  • Downstream Tasks: Zero-shot accuracy was tested on a suite of benchmarks including ARC (Challenge & Easy), HellaSwag, PIQA, and Winogrande.

Key Insights and Ideas

  • The "Early Commitment" Tax: Greedy solvers pay a heavy price in low-bit regimes because they commit to a single trajectory through the search space too early. MPQ's "soft" approach postpones this commitment, leading to better global solutions.
  • Probabilistic Unification: By casting PTQ as inference, the authors provide a theoretical bridge between classical signal processing (message passing) and modern LLM compression.
  • Synergy with Preprocessing: MPQ isn't a replacement for tricks like activation scaling (AWQ) or weight rotations (QuIP/QuRot); it's a better solver that composes favorably with them. The paper includes theoretical analysis showing how diagonal scaling (like AWQ) actually improves the "effective precision" that the message-passing stage sees.

Key Results

MPQ consistently outperforms strong baselines like GPTQ, QuantEase, and OJBKQ, with the most dramatic gains seen at the lowest bit-widths.

  • 2-Bit Lattice Quantization: On Llama 3.2-1B, MPQ achieved a WikiText-2 perplexity of 31.67, a massive improvement over the LDLQ baseline of 42.80.
  • 3-Bit Scalar Quantization: Across the Llama family, MPQ reduced perplexity by significant margins (e.g., from 14.26 to 13.65 on Llama 3.2-1B compared to the strongest randomized baseline).
  • Zero-Shot Accuracy: On Qwen3-8B at 2-bit, MPQ improved average zero-shot accuracy by 1.9% over the next best baseline.
Table 1: Llama-family PTQ perplexity scaling.

Limitations and Future Work

  • Computational Overhead: MPQ is slower than simple greedy methods. While it's still an "offline" process (done once per model), the extra VAMP iterations and CD passes add to the quantization time. However, the authors show that for scalar quantization, even a few VAMP iterations without CD can beat existing baselines.
  • Codebook Specificity: The current implementation focuses on uniform scalar and specific lattice (E8P12, D4) codebooks. Adapting it to learned vector codebooks (like AQLM) or trellis-coded quantization (like QTIP) would require extending the discrete denoiser logic.
  • Theoretical Gap: The theoretical analysis relies on "Right Orthogonally Invariant" (ROI) surrogates. While these provide excellent qualitative guidance, a tight characterization for real-world, non-ROI weight matrices remains an open challenge.

The code is available at https://github.com/poseidonchan/MPQ.

Summary updated

Abstract

Post-training quantization (PTQ) is a key technique for efficiently deploying large language models, as it compresses weights into low-bit representations without retraining. Existing PTQ solvers, including GPTQ, LDLQ, and block coordinate descent, commit each weight block to a single codeword before its neighbors are considered, leaving near-optimal alternatives unexplored. In this work, we cast row-wise PTQ as posterior inference over discrete codebooks, in which existing solvers correspond to a zero-temperature, hard-decision limit retaining only the top codeword per block. We propose Message-Passing Quantization (MPQ), which preserves soft posterior information across blocks during inference and commits to hard codewords only at the output stage. MPQ first uses vector approximate message passing to compute approximate blockwise posteriors, then refines a hard-decision initialization by exact block coordinate descent. Across 2-bit lattice and 3- and 4-bit scalar quantization on dense language models from Llama, Qwen, and Mistral families, MPQ improves perplexity and zero-shot accuracy over strong hard-decision baselines, and composes favorably with preprocessing techniques such as activation-aware scaling and incoherence processing. Our implementation is available at https://github.com/poseidonchan/MPQ.
BibTeX
@misc{chen2026mpq,
  author = {Yanshuo Chen and Chenxi Liu and Reza Shirkavand and Heng Huang},
  title = {{MPQ: A Message-Passing View of Post-Training Quantization}},
  year = {2026},
  eprint = {2610.32127},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.32127},
}
Figure