PTNQ: Post-Training Non-Linear Quantization

Diogo Venâncio, Nuno P. LopesOpenReview 2025 · Submission to ICLR 2025 · Rejected · OpenReview

Abstract

Quantization is one of the leading techniques to reduce the memory usage of machine learning models. It works by approximating the weights of a model by some function with a smaller domain (e.g., replace 32-bit floats with 8-bit integers that are coefficients in some function that maps back to 32-bit floats). Although most quantization methods approximate weights with a linear or affine function, the weights of current machine learning models often exhibit non-linear behavior at the extremities. Moreover, some studies suggest that the extremities are important for the end-to-end accuracy. In this paper, we introduce PTNQ, a novel post-training quantization technique that approximates weights by searching through a pool of non-linear functions. We show that PTNQ provides significant advantages over affine functions, achieving similar accuracy while requiring 2 to 4 fewer bits per coefficient.
BibTeX
@misc{venancio2025ptnq,
  author = {Diogo Venâncio and Nuno P. Lopes},
  title = {{PTNQ: Post-Training Non-Linear Quantization}},
  year = {2025},
  url = {https://openreview.net/forum?id=AEvu2ifH1r},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2025},
}

Loading related content...

Figure