Introduction
3D medical imaging, such as MRI and CT, is a cornerstone of modern diagnostics and treatment planning. However, the manual segmentation of these volumes—identifying and outlining structures like tumors or lesions—is a labor-intensive, expensive, and often subjective task. Weakly supervised learning offers a potential solution by training models on simpler, image-level labels (e.g., "lesion present" vs. "absent") rather than pixel-perfect 3D masks.
The current standard for weak supervision is Class Activation Mapping (CAM). While effective for localization, CAMs are typically derived from deep, downsampled layers of a neural network, resulting in blurry, low-resolution heatmaps that struggle to capture the fine details necessary for clinical applications. This paper introduces ToNNO (Tomographic Reconstruction of a Neural Network’s Output), a framework that leverages the mathematics of tomography to transform 2D classifiers into high-resolution 3D segmenters.
Methodology: Segmentation via Tomography
The core innovation of ToNNO is treating the output of a 2D neural network as a series of projections, analogous to how a CT scanner reconstructs a 3D volume from 1D X-ray projections.
Training with Label Noise
The process begins by training a standard 2D encoder (e.g., ResNet) to classify whether a single slice contains a lesion. Because the available labels are at the volume level, the authors assign the volume's label to every constituent slice. This introduces significant "label noise," as many slices in a "positive" volume will not actually contain the target pathology.
Inference and Reconstruction
During inference, ToNNO extracts thousands of 2D slices from a 3D volume at various orientations using a Fibonacci lattice. This ensures an even distribution of angles over a sphere. Each slice is processed by the 2D encoder to generate a prediction score (logit).
These scores are then treated as projections. By applying the Inverse Radon Transform—specifically filtered backprojection—the framework reconstructs a 3D heatmap. The "filtering" step, which involves taking the second derivative along the slice axis, is critical. It suppresses the low-frequency blur inherent in simple backprojection and provides the high-frequency detail required for sharp segmentation.
Experimental Results
The researchers evaluated ToNNO across four diverse 3D medical datasets: Multiple Sclerosis (MS) brain MRI, AutoPET-II whole-body PET-CT, MosMedData COVID-19 thoracic CT, and Duke Breast Cancer MRI.
Performance and Resolution
ToNNO consistently outperformed traditional CAM-based methods. On the MS dataset, ToNNO achieved an F1-score of 0.82, significantly higher than the 0.73 achieved by LayerCAM. A hybrid approach, "Tomographic LayerCAM," further improved this to 0.84.
Crucially, ToNNO is not limited by the resolution of internal feature maps. Because it operates on slice-level predictions, it can generate heatmaps at the original input resolution, making it particularly effective at identifying small, subtle lesions that are often lost in the downsampling layers of standard CAM models.
Discussion and Limitations
One of the most compelling aspects of ToNNO is its mathematical robustness to label noise. While many slices in a positive volume are false positives, their signals do not align across different projection angles. Only true lesions consistently "intersect" during the backprojection process, allowing the actual signal to emerge from the noise.
However, this precision comes at a computational cost. ToNNO requires thousands of inferences per volume, leading to reconstruction times of 40–60 seconds on an RTX 3090, compared to less than a second for slice-wise CAM. Additionally, the authors noted that the choice of normalization in the 2D encoder is vital; Batch Normalization can negatively impact performance by breaking slice independence, whereas Group Normalization or frozen Batch Norm layers are preferred.
Conclusion
ToNNO represents a significant step forward in weakly supervised 3D segmentation. By reframing neural network outputs as tomographic projections, it achieves state-of-the-art performance and high-resolution results without the need for expensive 3D ground-truth masks. Future work may explore iterative reconstruction techniques—common in modern CT imaging—to further refine the quality and efficiency of the reconstructed heatmaps.
Summary updated