Progressive Transmission of Medical Images via a Bank of Generative Adversarial Networks

Ching‐Chun Chang, Xu Wang, Ji-Hwei Horng, Isao EchizenJournal of Healthcare Engineering 2021 · DOI

TL;DR

This paper tackles the challenge of efficiently transmitting large medical images over limited bandwidth networks by introducing a novel deep learning approach. They leverage Generative Adversarial Networks (GANs) to predict image bit-planes progressively, significantly reducing the amount of data that needs to be transmitted and then compress the prediction residuals with a tailored algorithm.

What Problem Does This Paper Address?

The increasing volume of medical images generated by advanced imaging equipment poses a significant challenge for transmission, especially over low-bandwidth public networks. This can lead to critical network pressure and delays in diagnosis. Traditional methods for transmitting images progressively (Progressive Image Transmission or PIT) often involve sending lower resolution versions first and then gradually adding detail. However, existing PIT techniques, like bit-plane methods, can still be inefficient. The authors identify a gap in applying deep learning, specifically GANs, to this problem of medical image transmission.

Methods

The core of the proposed solution is a two-step pipeline:

  1. GAN-based Bit-Plane Prediction: The authors adapt the pix2pix GAN architecture, originally designed for image-to-image translation, to predict medical image bit-planes. Instead of translating between different image styles, the GAN learns to predict a bit-plane based on previously transmitted, higher-order bit-planes. This is framed as a "predictor bank" where multiple GANs are trained to predict each bit-plane sequentially. The generator (U-Net) learns to predict the bit-plane, and a discriminator (PatchGAN) helps refine the prediction. The loss function combines traditional GAN loss with an L1 loss to ensure high similarity between the predicted and actual bit-planes.
  2. Adaptive Residual Bit-Plane Compression: After the GAN predicts a bit-plane, the difference between the predicted and the actual bit-plane (the residual) is calculated. Since the GAN predictions are highly accurate, these residual bit-planes are expected to have many zeros. The paper proposes a novel adaptive compression technique for these residuals. This method divides the residual bit-plane into blocks and uses a location map to indicate whether a block is uniform (all zeros or all ones) or contains varying bits. Uniform blocks are encoded efficiently, while blocks with variations have their bits recorded. This is followed by arithmetic coding of the location map.

Datasets

The study uses greyscale chest X-ray images collected from the National Institutes of Health (NIH) under the U.S. Department of Health and Human Services.

  • Training: 10,000 images from the first compressed file.
  • Testing/Evaluation: 10 images from the fifth compressed file. All images were downsampled to 256x256 pixels to fit the network architecture.

Key Insights and Ideas

  • GANs for Predictive Compression: The paper demonstrates that GANs, typically used for image generation, can be effectively repurposed for predictive tasks like bit-plane prediction in image transmission. This opens up new avenues for using deep learning in compression.
  • Progressive Prediction for Efficiency: By predicting bit-planes progressively (from most significant to least significant), the system can leverage already transmitted information to make highly accurate predictions for subsequent bit-planes.
  • Residual Compression is Key: The success of the transmission efficiency hinges on the ability to compress the residuals (prediction errors). The paper's tailored adaptive compression method is designed specifically for these sparse residual bit-planes.
  • Block-based Compression for Sparsity: The adaptive compression method's effectiveness relies on the observation that residual bit-planes often contain large uniform regions, which can be efficiently encoded using block-based strategies.

Key Results

  • High Prediction Accuracy: The GAN-based predictor bank achieved high accuracy in predicting bit-planes. Error rates for the residual bit-planes generally increased from the most significant bit (r7) to the least significant bit (r1), with an average error rate of 19.2673% across all bit-planes.
  • Effective Residual Compression: The proposed adaptive residual bit-plane compression technique achieved significantly better compression ratios compared to conventional arithmetic coding, especially for higher-order bit-planes (r7 and r6).
  • Overall Improvement: The combined approach of GAN prediction and adaptive residual compression leads to improved efficiency for progressive medical image transmission.

Limitations and Future Work

  • Block Size Sensitivity: The compression ratio is affected by the chosen block size in the adaptive compression method. The paper notes that a block size of 4 generally yielded better compression ratios.
  • Texture Dependence: The effectiveness of the predictor bank is noted to be influenced by the smoothness of residual bit-planes, with textures in the region of interest being particularly suitable.
  • Predictor Bank Protection: The authors highlight that the parameters of the pretrained predictor bank are crucial for both sender and receiver and need to be protected for security and applicability.
  • Future Work: The authors plan to explore and integrate more advanced convolutional neural networks for improved compression and transmission of medical images.

Summary updated

Abstract

The healthcare sector is currently undergoing a major transformation due to the recent advances in deep learning and artificial intelligence. Despite a significant breakthrough in medical imaging and diagnosis, there are still many open issues and undeveloped applications in the healthcare domain. In particular, transmission of a large volume of medical images proves to be a challenging and time-consuming problem, and yet no prior studies have investigated the use of deep neural networks towards this task. The purpose of this paper is to introduce and develop a deep-learning approach for the efficient transmission of medical images, with a particular interest in the progressive coding of bit-planes. We establish a connection between bit-plane synthesis and image-to-image translation and propose a two-step pipeline for progressive image transmission. First, a bank of generative adversarial networks is trained for predicting bit-planes in a top-down manner, and then prediction residuals are encoded with a tailored adaptive lossless compression algorithm. Experimental results validate the effectiveness of the network bank for generating an accurate low-order bit-plane from high-order bit-planes and demonstrate an advantage of the tailored compression algorithm over conventional arithmetic coding for this special type of prediction residuals in terms of compression ratio.
BibTeX
@article{chang2021progressive,
  author = {Ching‐Chun Chang and Xu Wang and Ji-Hwei Horng and Isao Echizen},
  title = {{Progressive Transmission of Medical Images via a Bank of Generative Adversarial Networks}},
  journal = {Journal of Healthcare Engineering},
  year = {2021},
  doi = {10.1155/2021/9917545},
}
Figure