Abstract
Deep neural networks (DNNs) have expanded into medical fields, triggering a revolution in applications by extracting complex features and achieving high accuracy. However, large-scale networks impose high requirements for memory storage and computational resources, particularly for portable medical devices and embedded systems. In this work, we train a DNN for pneumonia detection using the RSNA Pneumonia Detection Challenge dataset. To overcome hardware limitations, we develop a systematic structured weight pruning method incorporating filter sparsity, column sparsity, and combined sparsity. Experiments demonstrate a compression ratio of up to 36x for a 106-layer model with no significant accuracy degradation. We evaluate these methods on an embedded low-power device, the NVIDIA Jetson TX2, achieving low power usage and high energy efficiency.
Introduction
Pneumonia is a leading cause of death globally, and its diagnosis from chest X-rays typically requires highly trained specialists. While deep learning has shown promise in automating this process, the resulting models are often too massive for the portable, battery-powered devices needed in remote or resource-limited clinical settings. Traditional "irregular" weight pruning can reduce model size on paper but results in fragmented network structures that are inefficient for hardware accelerators. This paper presents a hardware-friendly structured pruning approach that shrinks a state-of-the-art object detector (YOLOv3) to fit within the strict power and memory constraints of edge devices.
Methodology
The authors treat pneumonia diagnosis as an object detection task, utilizing YOLOv3 with a Darknet-53 backbone to identify and bound afflicted lung areas. The core of their approach is an ADMM-based structured weight pruning algorithm implemented in three phases: pre-pruning, masked mapping, and retraining.
To ensure the resulting model remains efficient on hardware, the pruning focuses on the General Matrix Multiplication (GEMM) operations within convolutional layers through three specific strategies:
- Filter Pruning: Removes entire filters, effectively reducing the number of rows in the weight matrix.
- Column Pruning: Removes entire columns in the converted 2D weight matrix.
- Combined Pruning: A sequential approach that first applies filter masking followed by column pruning to maximize sparsity while maintaining a dense matrix structure compatible with hardware accelerators.
Experimental Results
The model was trained and tested on a subset of the RSNA Pneumonia Detection Challenge Dataset, consisting of 6,002 labeled images. The original YOLOv3 model, containing 61.5 million parameters and requiring 246.4 MB of storage, served as the baseline.
Compression and Accuracy
The results demonstrate that medical vision models are highly over-parameterized. The authors achieved a 36.02x compression ratio using combined pruning, reducing the parameter count to just 1.7 million and the storage requirement to 6.84 MB. Remarkably, this massive reduction resulted in virtually no loss in detection performance. As shown in the table below, the mAP at an IoU threshold of 0.5 only dropped from 71.2 to 71.0.
Hardware Deployment
When deployed on the NVIDIA Jetson TX2, the 36x compressed model exhibited significant efficiency gains:
- Power Consumption: Only 7.3 W during inference.
- Energy Efficiency: Achieved 0.69 Inferences Per Second per Watt (IPS/W).
- Compute Reduction: FLOPs were slashed from 38.63 Billion to 1.32 Billion.
Conclusion
This research proves that it is possible to remove over 97% of a deep object detection network's parameters without sacrificing clinical diagnostic accuracy. By utilizing structured pruning, the authors transformed a heavy 106-layer model into a lightweight version suitable for low-power microcontrollers and edge devices. This paves the way for deploying advanced AI diagnostic tools in portable medical hardware, potentially expanding access to high-quality pneumonia screening in underserved regions.
Summary updated