Plaintext-Free Deep Learning for Privacy-Preserving Medical Image Analysis through Frequency Information Embedding

Mengyu Sun, Ziyuan Yang, Maosong Ran, Zhiwen Wang, Hui Yu, Yi ZhangICASSP 2025 · DOI

Introduction

The integration of Deep Learning (DL) into medical diagnostics has demonstrated transformative potential, yet its reliance on vast quantities of sensitive patient data presents a significant privacy challenge. Traditional privacy-preserving techniques, such as simple anonymization or adversarial perturbations, often fail to provide robust visual privacy or suffer from a steep utility-privacy trade-off. To address these limitations, this research introduces the Frequency-domain Exchange Style Fusion (FESF) framework. FESF enables DL models to be trained and deployed on medical images without ever accessing the original plaintext data, effectively "cloaking" sensitive diagnostic information within non-sensitive host images.

Methodology: Frequency-domain Exchange Style Fusion

The FESF framework leverages the mathematical properties of the Fast Fourier Transform (FFT) to decouple an image's visual structure from its statistical diagnostic features.

Frequency Domain Decomposition

In the frequency domain, an image is represented by its phase and amplitude spectra. The phase spectrum primarily encodes high-level structural and semantic information—the elements that allow the human eye to recognize objects. Conversely, the amplitude spectrum contains low-level distribution and "style" features, which are often the primary signals utilized by neural networks for classification tasks.

Image Hidden Module (IHM)

The core of the FESF approach is the Image Hidden Module. This module takes a generic, non-sensitive "host" image and a sensitive "plaintext" medical image. By swapping the low-frequency amplitude components of the medical image into the host image while preserving the host's original phase, the IHM creates a synthetic surrogate. This surrogate visually resembles the host image but carries the diagnostic "DNA" of the medical image.

Image Quality Enhancement Module (IQEM)

To ensure the synthetic images are stable and suitable for model training, the authors employ a GAN-based Image Quality Enhancement Module. The IQEM refines the synthetic images to better align with the distribution of the host domain, resulting in a "surrogate image" that is visually indistinguishable from a standard non-medical image. A secondary pass through the IHM is often utilized to further stabilize the utility of the surrogate data.

Privacy-Preserving Workflow

The FESF workflow is designed for decentralized privacy. Patients or local clinics convert their medical images into surrogates locally. Only these visually anonymous surrogates are uploaded to the central server. Consequently, the server-side DL model performs training and inference entirely on "plaintext-free" data, ensuring that sensitive medical information never leaves the patient's control.

Experimental Evaluation

The framework was evaluated across four diverse medical imaging datasets:

  • Br35H: Brain tumor detection (MRI).
  • Retinal Fundus: Eye disease detection.
  • Cervical Cancer: Cellular-level imaging.
  • Skin Cancer: Dermoscopic lesion images.

The authors tested the framework using four popular architectures: VGG16, ResNet34, MobileNetv3, and EfficientNetv2. Images were processed at 512x512 resolution with a 6:4 training-to-testing split.

Results and Discussion

The experimental results highlight a successful balance between privacy and utility in several domains.

Diagnostic Utility

For datasets like Cervical Cancer and Br35H, the performance drop was negligible. For instance, using a ResNet34 architecture on the Cervical Cancer dataset, accuracy only decreased from 99.96% (plaintext) to 98.67% (surrogate). This suggests that diagnostic features in these modalities are heavily concentrated in the low-frequency amplitude components.

However, a more pronounced performance dip was observed in the Skin Cancer and Retinal Fundus datasets (e.g., Skin Cancer accuracy dropped from ~88% to ~79%). This is attributed to the reliance of these specific diagnoses on fine textures and color nuances, which the IQEM may partially suppress to maintain visual consistency with the host image.

Visual Privacy

Qualitative analysis confirmed that surrogate images are visually indistinguishable from the host images. This "cloaking" effect ensures that even if a server is compromised, the data retrieved would appear as generic, non-sensitive imagery, providing a robust defense against human inspection and reconstruction attacks.

Conclusion and Future Work

FESF demonstrates that deep learning models can achieve high diagnostic accuracy on images that are visually unrecognizable to humans. While the framework excels in structural diagnostics, its current limitation lies in color-sensitive tasks where the IQEM prioritizes host-domain consistency over subtle color variations. Future research will focus on refining the IQEM to better preserve color-based diagnostic features and automating the hyperparameter tuning of the frequency exchange process to optimize the utility-privacy balance across diverse medical tasks.

Summary updated

BibTeX
@inproceedings{sun2025plaintext,
  author = {Mengyu Sun and Ziyuan Yang and Maosong Ran and Zhiwen Wang and Hui Yu and Yi Zhang},
  title = {{Plaintext-Free Deep Learning for Privacy-Preserving Medical Image Analysis through Frequency Information Embedding}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10888987},
}
Figure