PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models

Introduction

As Large Language Models (LLMs) scale to hundreds of billions of parameters, their deployment on consumer-grade hardware or edge devices faces significant hurdles due to memory and computational demands. While 4-bit quantization has become a standard practice, pushing further to ultra-low bit-widths (1 or 2 bits) often leads to "reasoning collapse," where models lose critical abilities in logic, mathematics, and coding.

Existing approaches typically fall into two categories: Quantization-Aware Training (QAT), which is effective but computationally prohibitive, and Post-Training Quantization (PTQ), which is efficient but often fails to maintain high-level reasoning at sub-2-bit levels. PTQTP (Post-Training Quantization to Trit-Planes) bridges this gap by offering a "plug-and-play" framework that achieves near-lossless performance at approximately 1.58 bits without requiring expensive retraining.

The PTQTP Framework

The core innovation of PTQTP lies in its Magnitude-Topology Decoupling using Dual Trit-Planes. Unlike binary quantization (±1), PTQTP utilizes ternary values $\{-1, 0, 1\}$. The inclusion of the "0" state allows the model to effectively "silence" noisy or unimportant features, preserving the delicate logical pathways necessary for complex reasoning.

Expand

Source, p. 3

The framework decomposes weight matrices into two separate ternary planes: a "coarse structure" plane and a "fine-grained correction" plane. This dual-plane approach creates a "constructive interference" effect, where the second plane compensates for quantization noise introduced by the first. A progressive optimization algorithm, utilizing adaptive regularization and ridge regression, iteratively refines these planes to match the original FP16 weight distribution.

Evaluation and Performance

PTQTP was evaluated across various model families, including LLaMA-2, LLaMA-3.x, and Qwen3, ranging from 0.6B to 70B parameters. Benchmarks spanned language modeling (WikiText-2, C4), general reasoning (MMLU, ARC), and complex tasks like mathematics (GSM8K) and coding (HumanEval).

Expand

Source, p. 1

Key Results:

• Inference Speed: PTQTP achieved up to a 4.63× faster end-to-end decode speed compared to FP16 baselines on an NVIDIA RTX 3090. This is made possible by replacing floating-point multiplications with simple additions and sign flips via LUT-GEMM.
• Quantization Efficiency: The process is remarkably fast, completing in about one hour on a single GPU. It is 73.4× faster than AQLM and significantly more efficient than AWQ.
• Reasoning Resilience: On the Qwen3-14B model, PTQTP retained nearly 95% of the original FP16 reasoning performance. Notably, it maintains high accuracy on mathematical tasks where traditional binary PTQ methods often fail completely.

Conclusion

PTQTP demonstrates that ultra-low-bit quantization can be achieved efficiently without sacrificing the reasoning capabilities of LLMs. By leveraging the "power of zero" in ternary representations and a dual-plane decomposition strategy, it provides a viable path for deploying massive models on resource-constrained hardware. Future work will focus on optimizing software-hardware co-design, such as custom CUDA kernels, to fully realize the potential of multiplication-free ternary inference.

@misc{xiao2025ptqtp,
  author = {He Xiao and Runming Yang and Qingyao Yang and Wendong Xu and Zheng Li and Yupeng Su and Zhengwu Liu and Hongxia Yang and Ngai Wong},
  title = {{PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models}},
  year = {2025},
  eprint = {2509.16989},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.16989},
}