CompSRT: Quantization and Pruning for Image Super Resolution Transformers

Dorsa Zeinali, Hailing Wang, Yitian Zhang, Yun FuOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

TL;DR

CompSRT is a new compression framework for Image Super-Resolution (SR) Transformers that significantly narrows the gap between low-bit models and full-precision performance. By combining a statistically-backed Hadamard transform approach with a novel learnable scalar decomposition, the authors achieve state-of-the-art results—surpassing existing methods by up to 1.53 dB—while remaining compatible with 40% weight pruning.

What Problem Does This Paper Address?

Image Super-Resolution (SR)—the task of upscaling low-resolution images into high-quality versions—is computationally demanding. While Transformer-based models like SwinIR-light are more efficient than older CNNs, they are still too heavy for many edge devices.

To fix this, researchers use quantization (reducing the precision of weights/activations from 32-bit to 2-4 bits) and pruning (removing unimportant weights). However, current post-training quantization (PTQ) methods for SR Transformers often result in blurry images and a significant drop in Peak Signal-to-Noise Ratio (PSNR) compared to the original models. There has been a lack of deep theoretical understanding regarding how data distributions change during compression, which has limited the effectiveness of these techniques.

Methods

The authors introduce CompSRT, which relies on three main pillars:

  1. Hadamard Transformation: The paper applies Hadamard transforms to weights and activations. Unlike previous work that simply claimed this "flattens" distributions, the authors provide a rigorous statistical analysis showing that the transform actually reduces the range of values and concentrates more data points around zero, making the signal much easier to quantize.
  2. Scalar Decomposition: Standard quantization uses a scale ($S$) and a zero-point ($l$). CompSRT "decomposes" these into $S' = S + \alpha$ and $l' = l + \beta$, where $\alpha$ and $\beta$ are two additional learnable parameters. This tiny increase in model capacity (just two parameters per layer) provides better gradient flow during optimization, allowing for much finer tuning of the quantization process.
  3. Magnitude-based Pruning: Because the Hadamard transform pushes more values toward zero, the authors found they could prune 40% of the weights with minimal performance loss at 3-4 bits. They use a 1-bit mask to track the locations of these pruned values.

Datasets

The model was trained on the DF2K dataset (a combination of DIV2K and Flickr2K). Evaluation was performed on five standard benchmarks in the SR field:

  • Set5 and Set14 (Standard natural images)
  • B100 (Berkeley Segmentation Dataset)
  • Urban100 (Urban scenes with repetitive structures)
  • Manga109 (Japanese comics, known for sharp lines and complex textures)

Key Insights and Ideas

  • The "Why" of Hadamard: Through Shapiro-Wilk and Wilcoxon tests, the authors proved that Hadamard transforms make distributions more Gaussian and statistically significantly lower the range of values. This is the "aha moment": it’s not just about removing outliers; it’s about compacting the entire signal into a more manageable "quantization-friendly" shape.
  • Pruning as Noise Reduction: Interestingly, at 3 bits, pruning actually improved performance slightly in some cases. The authors suggest that in low-capacity models, small values near zero act more like noise; removing them allows the model to focus on a "smoother," more representable signal.
  • Learnable Flexibility: The addition of $\alpha$ and $\beta$ parameters highlights that even in "post-training" quantization, a small amount of targeted learnability can overcome the bias inherent in rigid, simpler quantization formulas.

Key Results

  • Quantitative Superiority: CompSRT outperformed the previous SOTA (CondiQuant) across all bitwidths and scales. The most dramatic gain was a +1.53 dB PSNR and +0.03 SSIM improvement on the Manga109 dataset at 2-bit (x4) resolution.
  • Near-Lossless 4-bit: At 4 bits, the model performs remarkably close to the full-precision baseline (within 0.01–0.11 dB), effectively reducing model size by 8x with almost no visible quality loss.
  • Efficiency Gains: By pruning 40% of weights at 4 bits, they achieved a 15% reduction in bits per parameter compared to standard 4-bit quantization, while maintaining performance comparable to unpruned SOTA methods.
  • Visual Quality: Qualitative results show that CompSRT produces significantly sharper edges and less graininess than 2DQuant, particularly in text and manga illustrations.

Limitations and Future Work

  • Computational Overhead: The Hadamard transform requires an additional dense matrix multiplication per quantized layer. While it doesn't increase storage needs (the matrices are generated on the fly), it does add a modest amount of latency (approx. 0.044 seconds per forward pass in their tests).
  • Pruning at 2-bit: The authors noted that pruning doesn't provide storage gains at the 2-bit level because the overhead of storing the 1-bit location mask outweighs the savings from removing weights.
  • Structured Pruning: While they experimented with structured (channel-wise) pruning in the appendix, it led to more performance degradation than unstructured pruning. Future work could focus on making structured pruning more robust for Transformers.

Summary updated

Abstract

Model compression has emerged as a way to reduce the cost of using image super resolution models by decreasing storage size and inference time. However, the gap between the best compressed models and the full precision model still remains large and a need for deeper understanding of compression theory on more performant models remains. Prior research on quantization of LLMs has shown that Hadamard transformations lead to weights and activations with reduced outlier, which leads to improved performance. We argue that while the Hadamard transform does reduce the effect of outliers, an empirical analysis on how the transform functions remains needed. By studying the distributions of weights and activations of SwinIR-light, we show with statistical analysis that lower errors is caused by the Hadamard transforms ability to reduce the ranges, and increase the proportion of values around $0$. Based on these findings, we introduce CompSRT, a more performant way to compress the image super resolution transformer network SwinIR-light. We perform Hadamard-based quantization, and we also perform scalar decomposition to introduce two additional trainable parameters. Our quantization performance statistically significantly surpasses the SOTA in metrics with gains as large as 1.53 dB, and visibly improves visual quality by reducing blurriness at all bitwidths. At 3-4 bits, to show our method is compatible with pruning for increased compression, we also prune 40% of weights and show that we can achieve 6.67-15% reduction in bits per parameter with comparable performance to SOTA.
BibTeX
@misc{zeinali2026compsrt,
  author = {Dorsa Zeinali and Hailing Wang and Yitian Zhang and Yun Fu},
  title = {{CompSRT: Quantization and Pruning for Image Super Resolution Transformers}},
  year = {2026},
  url = {https://openreview.net/forum?id=ZmOmEJn5MM},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure