Abstract
Medical image analysis using deep learning has recently been prevalent, showing great performance for various downstream tasks including medical image segmentation and its sibling, volumetric image segmentation. Particularly, a typical volumetric segmentation network strongly relies on a voxel grid representation which treats volumetric data as a stack of individual voxel 'slices', which allows learning to segment a voxel grid to be as straightforward as extending existing image-based segmentation networks to the 3D domain. However, using a voxel grid representation requires a large memory footprint, expensive test-time and limiting the scalability of the solutions. In this paper, we propose Point-Unet, a novel method that incorporates the efficiency of deep learning with 3D point clouds into volumetric segmentation. Our key idea is to first predict the regions of interest in the volume by learning an attentional probability map, which is then used for sampling the volume into a sparse point cloud that is subsequently segmented using a point-based neural network. We have conducted the experiments on the medical volumetric segmentation task with both a small-scale dataset Pancreas and large-scale datasets BraTS18, BraTS19, and BraTS20 challenges. A comprehensive benchmark on different metrics has shown that our context-aware Point-Unet robustly outperforms the SOTA voxel-based networks at both accuracies, memory usage during training, and time consumption during testing. Our code is available at https://github.com/VinAIResearch/Point-Unet.
Introduction
The gold standard for 3D medical segmentation has long been voxel-based convolutional neural networks (CNNs), such as the ubiquitous U-Net. However, these models suffer from the "curse of dimensionality." To process a high-resolution volume, they require massive amounts of GPU memory, often forcing researchers to either downsample the data or process small patches, which leads to a loss of either detail or global context.
Medical volumes are inherently sparse; for instance, in a brain scan, the majority of the data represents healthy tissue or background. Voxel-based methods waste significant computational resources processing these uninteresting areas. Point-Unet addresses this by shifting from a dense grid of cubes (voxels) to a sparse collection of points, focusing computation where it matters most.
Methodology
Point-Unet introduces a three-stage pipeline designed to marry the efficiency of point clouds with the structural requirements of medical analysis:
1. Saliency Attention Module
Instead of sampling points uniformly, the model utilizes a lightweight saliency network to predict an attentional probability map. This map identifies regions likely to contain objects of interest, such as tumors.
2. Context-aware Sampling
Using the saliency map, the system performs non-uniform sampling. It densely samples points in salient regions to capture fine-grained details while sparsely sampling the remainder of the volume to maintain global spatial context.
3. Point-Unet Architecture
The sampled points are processed by a point-based neural network. The architecture is a U-Net-like structure based on RandLA-Net, utilizing Dilated Residual Blocks (DRB) to capture local dependencies and skip connections to preserve multi-scale features.
Experimental Results
The authors evaluated Point-Unet on challenging benchmarks including BraTS (2018, 2019, 2020) and Pancreas-CT.
- Accuracy: On BraTS20, Point-Unet achieved a mean Dice score of 83.02, significantly outperforming the baseline 3D U-Net (74.25).
- Efficiency: Point-Unet demonstrated superior memory efficiency, requiring only 17.22 GB of memory compared to the 48GB+ often required by high-resolution voxel models.
- Inference Speed: For a standard 240 x 240 x 155 volume, Point-Unet completed inference in 1.24 seconds, a drastic improvement over competing models that can take over a minute.
Conclusion
Point-Unet represents a significant shift in volumetric medical segmentation. By leveraging the sparsity of medical data through point cloud representations and context-aware sampling, it achieves state-of-the-art performance while drastically reducing computational overhead. Future work will focus on improving label reconstruction and boundary adaptive sampling to further refine segmentation at tissue interfaces.
Summary updated