MedPTQ: a practical pipeline for real post-training quantization in 3D medical image segmentation

Chongyu Qu, Ritchie Zhao, Ye Yu, Bin Liu, Tianyuan Yao, Junchao Zhu, Bennett A. Landman, Yucheng Tang, Yuankai Z HuoJournal of Medical Imaging 2026 · DOI

TL;DR

This paper introduces MedPTQ, a practical pipeline for real post-training quantization of 3D medical image segmentation models to INT8 precision. MedPTQ significantly reduces model size and inference latency on modern GPUs without sacrificing segmentation accuracy, making advanced AI models more accessible for clinical deployment.

What Problem Does This Paper Address?

Deep learning models have revolutionized medical imaging, but their substantial computational demands and large memory footprints pose significant challenges for deployment in resource-constrained clinical environments. Quantization, which reduces the precision of model computations (e.g., from 32-bit floating-point to 8-bit integers), offers a promising solution by decreasing model size and accelerating inference. However, many existing quantization methods only simulate lower precision during inference, failing to achieve actual reductions in model size or real-world speedups. Furthermore, the practical application of real, low-bit quantization for complex 3D medical image segmentation models on modern GPUs remains largely unexplored. This gap hinders the widespread adoption of powerful AI models in clinical settings where efficiency is paramount.

Methods

MedPTQ tackles this problem by implementing a two-step real post-training quantization (PTQ) pipeline using NVIDIA TensorRT.

  1. Simulated Quantization with ModelOpt: The process begins by exporting a pre-trained FP32 PyTorch model to the ONNX format. Then, NVIDIA's TensorRT Model Optimizer (ModelOpt) is used to insert QuantizeLinear and DequantizeLinear nodes. This stage simulates INT8 quantization by mapping FP32 values to INT8 integers based on calibration data. Crucially, the model still runs in FP32, meaning no actual efficiency gains are realized yet. This step's primary purpose is to determine the necessary calibration parameters for the subsequent real quantization.
  2. Real Quantization with TensorRT Engine: The simulated quantized ONNX model is then fed into TensorRT to build a real INT8 engine. TensorRT intelligently fuses the FP32 weights with their associated QuantizeLinear operations into INT8 weights, which are stored and accessed directly in 8-bit format. It also merges activation and weight DequantizeLinear operations, along with convolution, batch normalization, and ReLU layers, into a single fused convolution block. This fusion allows TensorRT to leverage optimized INT8 convolution kernels, leading to actual reductions in model size and inference latency by minimizing data conversions and memory traffic.

Datasets

The paper evaluates MedPTQ on three diverse datasets:

  • BTCV: Consists of 70 CT volumes with 13 labeled anatomies, used for pre-training U-Net and TransUNet, and evaluating them.
  • TotalSegmentator V2: A large-scale dataset with 1228 full-body CT volumes and 117 labeled anatomies. Used for pre-training nnU-Net, SegResNet, SwinUNETR, and VISTA3D, and evaluating them on 104 labels. This dataset was also used to evaluate the STU-Net models.
  • Whole Brain Segmentation Dataset: Comprises 4859 T1w MRI volumes, with a subset of 50 manually traced scans used for evaluating UNesT.

Key Insights and Ideas

  • Real vs. Simulated Quantization Matters: The core insight is that merely simulating INT8 operations doesn't yield practical benefits. MedPTQ's success hinges on converting these simulations into real INT8 computations on the GPU, which is essential for actual model compression and speedup.
  • TensorRT as a Powerful Tool: The paper demonstrates that NVIDIA TensorRT is a highly effective tool for achieving real INT8 quantization for complex 3D medical imaging models, handling the necessary graph optimizations and kernel generation.
  • Scalability Synergy: MedPTQ highlights a crucial synergy between model scaling (larger models) and quantization. While larger models offer better performance, they also increase computational burden. Quantization, as shown with STU-Net, is vital for making these scaled-up, high-performance models practical.
  • Robustness Across Architectures and Modalities: MedPTQ's effectiveness is validated across a wide range of 3D segmentation models (CNNs and transformers) and diverse medical imaging datasets (CT and MRI, various body regions), underscoring its generalizability.
  • Small Structures Remain Accurate: A key concern with quantization is potential degradation on small or clinically important anatomical structures. MedPTQ shows minimal impact on the segmentation accuracy of these small organs, demonstrating its clinical viability.

Key Results

MedPTQ achieved significant efficiency gains across seven SOTA 3D medical segmentation models:

  • Model Size Reduction: Up to 3.8x reduction in model size.
  • Inference Latency Speedup: Up to 2.7x faster inference.
  • GPU Memory Reduction: 1.1x to 1.5x reduction in GPU memory usage.
  • Accuracy Preservation: Maintained nearly identical segmentation accuracy, with absolute Mean Dice Similarity Coefficient (mDSC) degradation of ≤ 0.006.
  • Small Organ Performance: Demonstrated minimal impact on the segmentation accuracy of small anatomical structures, with class-level DSC changes typically below 0.003.
  • Large Model Efficiency: For the STU-Net-H model (1.4B parameters), MedPTQ reduced model size by 3.65x and inference latency by 3.26x, making a massive model practical.

Limitations and Future Work

  • TensorRT Compatibility: A limitation noted is TensorRT's potential lack of full support for models with dynamic blocks or layers requiring runtime flexibility (e.g., variable input sizes, conditional operations). This could restrict MedPTQ's applicability to a subset of AI models.
  • Future Directions:
    • More Clinical Validation: Further validation on edge cases and with radiologist review.
  • Lower Precision Quantization: Exploring INT4 quantization for even greater model size and computational reduction, which will require advanced techniques to mitigate accuracy loss.
  • Hardware Accelerators: Researching hardware accelerators optimized for INT4.
  • Memory Savings: Investigating additional techniques beyond INT8 quantization to achieve greater GPU memory savings, as current reductions are modest due to TensorRT's memory management strategy.

Summary updated

Abstract

We have introduced MedPTQ, a real post-training quantization pipeline that delivers INT8 inference for SOTA 3D artificial intelligence (AI) models in medical imaging segmentation. MedPTQ effectively reduces real-world model size, computational requirements, and inference latency without compromising segmentation accuracy on modern GPUs, as evidenced by mDSC comparable to full-precision baselines. We validate MedPTQ across a diverse set of AI architectures, ranging from convolutional-neural-network-based to transformer-based models, and a wide variety of medical imaging datasets. These datasets are collected from multiple hospitals with distinct imaging protocols, cover different body regions (such as the brain, abdomen, or full body), and include multiple imaging modalities [computed tomography (CT) and magnetic resonance imaging (MRI)]. Collectively, these results highlight our MedPTQ's strong generalizability and adaptability for a broad spectrum of medical imaging tasks.
BibTeX
@article{qu2026medptq,
  author = {Chongyu Qu and Ritchie Zhao and Ye Yu and Bin Liu and Tianyuan Yao and Junchao Zhu and Bennett A. Landman and Yucheng Tang and Yuankai Z Huo},
  title = {{MedPTQ: a practical pipeline for real post-training quantization in 3D medical image segmentation}},
  journal = {Journal of Medical Imaging},
  year = {2026},
  doi = {10.1117/1.jmi.13.1.014006},
}
Figure