Image2Points: A 3D Point-based Context Clusters GAN for High-Quality PET Image Reconstruction

Jiaqi Cui, Yan Wang, Lu Wen, Pinxian Zeng, Xi Wu, Jiliu Zhou, Dinggang ShenICASSP 2024

Abstract

Positron Emission Tomography (PET) is essential for diagnosing neurodegenerative diseases, yet the trade-off between radiation exposure and image quality remains a significant challenge. Standard-dose PET (SPET) provides high diagnostic value but involves high radiation, while low-dose PET (LPET) is safer but prone to noise and artifacts. This paper presents PCC-GAN (Point-based Context Clusters GAN), a novel framework that reimagines PET reconstruction by treating 3D images as sets of points rather than rigid voxel grids. By incorporating geometric coordinates and a context clustering strategy, PCC-GAN achieves state-of-the-art performance in reconstructing high-quality SPET images from LPET data with significantly higher parameter efficiency than traditional voxel-based models.

1. Introduction

The "clinical dilemma" of PET imaging centers on the necessity of high-quality imagery for accurate diagnosis versus the health risks associated with radiopharmaceutical exposure. While deep learning has advanced the field of LPET-to-SPET reconstruction, most existing methods rely on voxel-based architectures, such as Convolutional Neural Networks (CNNs) and Transformers. These models operate on fixed 3D grids, which often struggle to preserve intricate structural boundaries and fail to capture the complex, non-local relationships between small biological structures.

To address these limitations, we introduce the Image2Points framework. By shifting the data representation from voxels to points, we allow the model to leverage explicit geometric information and flexible contextual relationships, leading to superior structural precision and diagnostic utility.

2. Methodology

2.1 Point-Based Representation

Unlike traditional methods that treat a PET scan as a 3D matrix, PCC-GAN converts the LPET volume into a set of points. Each point is defined not only by its intensity feature but also by its explicit 3D geometric coordinates $(x, y, z)$. This integration ensures that spatial structure is a fundamental component of the feature space, allowing the model to better preserve the morphology of brain tissues.

2.2 Context Clusters (CoC) and Transposed Blocks

The architecture is built around Context Clusters (CoC) blocks for encoding and Transposed Context Clusters (TCoC) blocks for decoding. The process involves three key stages:

  1. Points Reducer/Expander: To manage the computational demands of 3D point sets, the model utilizes a reduction strategy that selects "anchor" points and groups their nearest neighbors. This effectively downsamples the data while maintaining local structure, which is later restored during the expansion phase.
  2. Context Clustering: Inspired by SuperPixel algorithms, points are grouped into clusters based on contextual affinity (calculated via cosine similarity). This allows the model to aggregate information from points that share similar biological features, even if they are not immediately adjacent in the spatial grid.
  3. Aggregating and Dispatching: Within each cluster, information is shared globally. This mechanism enables the model to resolve noise-induced ambiguities by considering the broader context of similar tissue types.

2.3 Point-Based Discriminator

In the Generative Adversarial Network (GAN) framework, the discriminator is tasked with distinguishing between real SPET images and reconstructed ones. PCC-GAN employs a discriminator that operates directly on point sets rather than image patches. This provides the generator with more granular feedback regarding structural disparities, further driving the reconstruction toward high fidelity.

3. Experimental Results

3.1 Datasets and Evaluation

The model was rigorously evaluated using two datasets:

  • Clinical Dataset: Comprising 16 subjects (8 Normal Control and 8 Mild Cognitive Impairment). LPET images were simulated from 3-minute scan windows, while SPET images were derived from 12-minute scans.
  • Phantom Dataset: 20 simulated subjects from the BrainWeb database, simulating a quarter-count level of normal radiation.

Evaluation was performed using Leave-One-Out Cross-Validation (LOOCV) to ensure the generalizability of the results.

3.2 Quantitative and Qualitative Performance

PCC-GAN consistently outperformed state-of-the-art voxel-based models, including Hi-Net, Auto-Context, and the Transformer-based Trans-GAN. On the clinical dataset, PCC-GAN achieved a Peak Signal-to-Noise Ratio (PSNR) of 24.890, surpassing Trans-GAN’s 24.818. Qualitatively, the point-based approach produced significantly lower error maps and sharper edges, effectively mitigating the blurring common in voxel-grid reconstructions.

3.3 Efficiency and Clinical Utility

A standout feature of PCC-GAN is its efficiency. The model achieves superior results with only 12 million parameters, a stark contrast to the 76 million parameters required by Trans-GAN. Furthermore, the clinical utility of the reconstructed images was validated through a downstream Alzheimer’s disease classification task. Images reconstructed by PCC-GAN yielded a diagnostic accuracy of 88.9%, nearly matching the 90.0% accuracy achieved using ground-truth SPET images.

4. Discussion and Conclusion

The Image2Points framework demonstrates that treating medical images as geometric point sets offers significant advantages over traditional voxel-based methods. By prioritizing similarity over proximity and embedding geometry as a first-class citizen, PCC-GAN provides a more accurate and efficient pathway for low-dose PET reconstruction.

While the results are promising, future work will focus on scaling the model to larger multi-center datasets and exploring its application in whole-body PET and low-dose CT scans. Additionally, optimizing the computational overhead of k-nearest neighbor searches remains a priority for real-time clinical deployment. In conclusion, PCC-GAN represents a significant step forward in high-quality, low-radiation medical imaging.

Summary updated

BibTeX
@inproceedings{cui2024image,
  author = {Jiaqi Cui and Yan Wang and Lu Wen and Pinxian Zeng and Xi Wu and Jiliu Zhou and Dinggang Shen},
  title = {{Image2Points: A 3D Point-based Context Clusters GAN for High-Quality PET Image Reconstruction}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
}
Figure