Recovering medical images from CT film photos

Quan Quan, Qiyuan Wang, Yuanqi Du, Liu Li, S. Kevin ZhouarXiv 2022 · OpenReview

Abstract

While medical images such as computed tomography (CT) are stored in DICOM format in hospital PACS, it is still quite routine in many countries to print a film as a transferable medium for the purposes of self-storage and secondary consultation. Also, with the ubiquitousness of mobile phone cameras, it is quite common to take pictures of CT films, which unfortunately suffer from geometric deformation and illumination variation. In this work, we study the problem of recovering a CT film, which marks the first attempt in the literature, to the best of our knowledge. We start with building a large-scale head CT film database CTFilm20K, consisting of approximately 20,000 pictures, using the widely used computer graphics software Blender. We also record all accompanying information related to the geometric deformation (such as 3D coordinate, depth, normal, and UV maps) and illumination variation (such as albedo map). Then we propose a deep framework called Film Image Recovery Network (FIReNet) to tackle geometric deformation and illumination variation using the multiple maps extracted from the CT films to collaboratively guide the recovery process. Finally, we convert the dewarped images to DICOM files with our cascade model for further analysis such as radiomics feature extraction. Extensive experiments demonstrate the superiority of our approach over the previous approaches.

Introduction

In many clinical settings, physical CT films remain a primary medium for patients to transport their medical history between institutions. The convenience of smartphone cameras has led to the widespread practice of digitizing these films via photography. However, these "in-the-wild" photos are often unsuitable for clinical analysis due to two primary factors: geometric deformation caused by the non-planar nature of the film and illumination variation (glare, shadows) from ambient lighting. Unlike standard document scanning, medical imaging requires extreme precision; minor distortions can obscure pathology or create artifacts that lead to misdiagnosis.

The CTFilm20K Dataset

To address the lack of paired training data (warped photos vs. original DICOMs), we developed CTFilm20K. This dataset contains approximately 20,000 simulated images created using Blender. By simulating the physical properties of CT film—including its flexibility and reflectivity—within diverse High Dynamic Range (HDR) environments, we generated photorealistic distortions.

The CTFilm20K dataset generation process, showing the various maps (3D, Normal, Depth, UV, Albedo) used to simulate and recover the film.

Crucially, each sample includes ground-truth metadata: 3D coordinates, surface normals, depth maps, and UV maps. This rich auxiliary information allows the recovery network to learn the underlying physics of the deformation.

Methodology: FIReNet

The proposed Film Image Recovery Network (FIReNet) employs a two-stage architecture to restore clinical utility to distorted photos.

1. Dewarping Module

Traditional document dewarping often relies on backward mapping, which can introduce local artifacts. FIReNet instead utilizes a UV-mapping approach. By predicting how a 2D texture wraps around a 3D surface, the network can "unwrap" the film more smoothly. The module generates multiple maps (3D, depth, normal) to collaboratively guide the transformation process, ensuring geometric consistency.

The FIReNet architecture, featuring a Dewarping Module for geometric correction and a Quality Restoration Module for de-illumination and DICOM conversion.

2. Quality Restoration Module

Following dewarping, the image still contains photographic artifacts. This module uses a cascade architecture for:

  • De-illumination: Extracting an albedo map to remove shadows and reflections.
  • Medical Image Restoration: Converting 8-bit RGB photographic data back into high-bit-depth values. This allows the final output to be saved in the standard DICOM format, making it compatible with existing medical software.

Results and Analysis

FIReNet demonstrates significant improvements over existing state-of-the-art methods like DewarpNet. Quantitative evaluations show a Peak Signal-to-Noise Ratio (PSNR) of 25.30 dB and a Structural Similarity Index (SSIM) of 0.8621, nearly doubling the performance of baseline models.

Beyond visual metrics, we validated the clinical reliability of the recovered images using Radiomics. By extracting thousands of quantitative features (e.g., entropy, energy) and performing Chi-square tests, we confirmed that the recovered DICOM files are statistically similar to the original digital scans. This suggests that FIReNet-restored images could potentially support automated diagnostic tools and longitudinal clinical follow-ups.

Conclusion

FIReNet represents a significant step toward making smartphone-captured medical images clinically viable. By combining 3D physics-based simulation with a multi-map deep learning framework, we successfully mitigate the challenges of geometric and illumination distortion. Future work will focus on expanding the dataset to include diverse anatomical regions and further validating the model with "in-the-wild" clinical data.

Summary updated

Abstract

While medical images such as computed tomography (CT) are stored in DICOM format in hospital PACS, it is still quite routine in many countries to print a film as a transferable medium for the purposes of self-storage and secondary consultation. Also, with the ubiquitousness of mobile phone cameras, it is quite common to take pictures of CT films, which unfortunately suffer from geometric deformation and illumination variation. In this work, we study the problem of recovering a CT film, which marks \textbf{the first attempt} in the literature, to the best of our knowledge. We start with building a large-scale head CT film database CTFilm20K, consisting of approximately 20,000 pictures, using the widely used computer graphics software Blender. We also record all accompanying information related to the geometric deformation (such as 3D coordinate, depth, normal, and UV maps) and illumination variation (such as albedo map). Then we propose a deep framework called \textbf{F}ilm \textbf{I}mage \textbf{Re}covery \textbf{Net}work (\textbf{FIReNet}) to tackle geometric deformation and illumination variation using the multiple maps extracted from the CT films to collaboratively guide the recovery process. Finally, we convert the dewarped images to DICOM files with our cascade model for further analysis such as radiomics feature extraction. Extensive experiments demonstrate the superiority of our approach over the previous approaches. We plan to open source the simulated images and deep models for promoting the research on CT film image analysis.
BibTeX
@misc{quan2022recovering,
  author = {Quan Quan and Qiyuan Wang and Yuanqi Du and Liu Li and S. Kevin Zhou},
  title = {{Recovering medical images from CT film photos}},
  year = {2022},
  eprint = {2203.05567},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.05567},
}
Figure