Fully-Automated Analysis of Body Composition from CT in Cancer Patients Using Convolutional Neural Networks

Christopher P. Bridge, Michael H. Rosenthal, Bradley Wright, Gopal Kotecha, Florian J. Fintelmann, Fabian M. Troschel, Nityanand Miskin, Khanant Desai, William Wrobel, Ana Babic, Natalia Khalaf, Lauren Brais, Marisa Welch, Caitlin Zellers, Neil A. Tenenholtz, Mark Michalski, Brian Wolpin, Katherine P. Andriole

TL;DR

Measuring a patient’s muscle and fat distribution from CT scans is a proven prognostic marker in oncology, but manually finding and contouring the target slice takes over 20 minutes per patient. This paper introduces a fast, fully automated two-stage deep learning pipeline (DenseNet for slice localization and U-Net for tissue segmentation) that processes an entire 3D CT scan in about one second. Evaluated across independent patient cohorts, it matches expert annotations with correlation coefficients of $R \approx 0.99$.

What Problem Does This Paper Address?

Body composition—specifically the cross-sectional areas of skeletal muscle, visceral adipose tissue, and subcutaneous adipose tissue—is strongly linked to cancer cachexia, chemotherapy toxicity, cardiovascular disease, and overall survival.

The clinical gold standard measures these tissues on a single 2D CT slice located at the third lumbar vertebra (L3). However, manually identifying the L3 vertebra and hand-tracing the tissue compartments takes an expert reader 20+ minutes per scan. This manual bottleneck makes large-scale epidemiologic studies prohibitively expensive and prevents opportunistic body composition screening in routine clinical workflows.

Methods

The authors break down the problem into two sequential 2D convolutional neural network (CNN) tasks:

Full 3D CT Volume 
       │
       ▼
[ DenseNet Regression ] ──► Sigmoid Target r(z) ──► 1D Gaussian Smoothing ──► Select L3 Slice
                                                                                    │
                                                                                    ▼
                                                                           [ 2D U-Net ]
                                                                                    │
                                                                                    ▼
                                                              Muscle / Subcut. Fat / Visceral Fat Masks
  1. L3 Slice Selection (DenseNet):
    Instead of using compute-heavy 3D CNNs or generating maximum intensity projections (MIPs), the model processes each axial slice independently (downsampled to $256 \times 256$). The network predicts the craniocaudal distance from L3, mapped through a sigmoid function: $$r(z) = \frac{1}{1 + e^{-\tau(z - z_{L3})}}$$ (with $\tau = 20\text{ mm}$). This forces the model to dedicate capacity specifically around L3. The slice-wise predictions are smoothed along the $z$-axis with a 1D Gaussian kernel ($\sigma = 2\text{ slices}$), and the zero-crossing ($0.5$) closest to the head is selected.

  2. Multi-Class Tissue Segmentation (U-Net):
    The full-resolution ($512 \times 512$) selected slice is fed into a U-Net trained with a multi-class soft Dice loss across three non-background classes: skeletal muscle, subcutaneous fat, and visceral fat.

Datasets

The authors used two distinct cancer cohorts with varying acquisition parameters and scanners:

  • Dataset A (Training & Validation): 595 abdominal CT scans from patients with pancreatic adenocarcinoma across multiple institutions. Slices at L3 were manually segmented using Slice-O-Matic and validated by a board-certified radiologist (divided into 412 train, 94 validation, and 89 test).
  • Dataset B (External Test Cohort): 534 CT scans from patients with lymphoma treated at Massachusetts General Hospital. Slices were manually selected and segmented using Osirix and revised by a different board-certified radiologist (up to 514 annotated series per tissue class).

Key Insights and Ideas

  • Sigmoidal Distance Target: Slices far from L3 look very different but their exact millimetric distance is clinically irrelevant. Saturating the regression target between 0 and 1 via a sigmoid prevents distant slices from dominating the training loss, focusing network attention on the local vertebral anatomy.
  • 1D Post-Processing Over 3D Convolutions: Applying a 2D network slice-by-slice followed by simple 1D temporal-style smoothing leverages spatial context across the volume without the memory footprint, varying slice-thickness challenges, or computational cost of volumetric 3D convolutions.
  • Separation of Adipose Compartments: Unlike earlier automated methods that lump all fat together, this model reliably distinguishes visceral from subcutaneous fat—a clinically critical distinction, as visceral adiposity drives metabolic risk.

Key Results

  • Slice Localization: The mean absolute localization error on the external test set (Dataset B) was $9.4\text{ mm}$, staying well within the anatomical span of the L3 vertebra.
  • Segmentation Accuracy (Dataset A test): High Dice Similarity Coefficients (DSC):
  • Muscle: $0.968 \pm 0.034$ (outperforming prior state-of-the-art like Lee et al.'s $0.93$)
  • Subcutaneous Fat: $0.984 \pm 0.021$
  • Visceral Fat: $0.954 \pm 0.100$
  • End-to-End Area Estimation (Dataset B):
  • Muscle: Mean Absolute Percentage Error (MAPE) of $3.1\%$, $R = 0.986$
  • Subcutaneous Fat: MAPE of $5.9\%$, $R = 0.986$
  • Visceral Fat: MAPE of $6.5\%$, $R = 0.994$
  • Inference Speed: On an Nvidia V100 GPU, slice selection required $0.5\text{--}1.0\text{ s}$ per scan, and segmentation required $0.02\text{--}0.025\text{ s}$, cutting processing time from over 20 minutes to approximately one second.

Limitations and Future Work

  • 2D Proxy vs. Full Volumetric Analysis: The method strictly replicates the single-slice L3 biomarker convention. While standardized, a single slice may not fully reflect whole-body distribution in atypical anatomies.
  • Outliers and Boundary Confusion: In challenging cases (such as severe disease or atypical anatomy), the network occasionally struggled to delineate the peritoneal boundary between subcutaneous and visceral fat depots.
  • Cohort Scope: Validation was restricted to oncologic CT scans (pancreatic cancer and lymphoma); further validation on non-cancer, post-surgical, or pediatric populations remains for future work.

Summary updated

Abstract

The amounts of muscle and fat in a person’s body, known as body composition, are correlated with cancer risks, cancer survival, and cardiovascular risk. The current gold standard for measuring body composition requires time-consuming manual segmentation of CT images by an expert reader. In this work, we describe a two-step process to fully automate the analysis of CT body composition using a DenseNet to select the CT slice and U-Net to perform segmentation. We train and test our methods on independent cohorts. Our results show Dice scores (0.95−0.98) and correlation coefficients (R = 0.99) that are favorable compared to human readers. These results suggest that fully automated body composition analysis is feasible, which could enable both clinical use and large-scale population studies.
BibTeX
@inproceedings{bridge2018fully,
  author = {Christopher P. Bridge and Michael H. Rosenthal and Bradley Wright and Gopal Kotecha and Florian J. Fintelmann and Fabian M. Troschel and Nityanand Miskin and Khanant Desai and William Wrobel and Ana Babic and Natalia Khalaf and Lauren Brais and Marisa Welch and Caitlin Zellers and Neil A. Tenenholtz and Mark Michalski and Brian Wolpin and Katherine P. Andriole},
  title = {{Fully-Automated Analysis of Body Composition from CT in Cancer Patients Using Convolutional Neural Networks}},
  booktitle = {OR 2.0/CARE/CLIP/ISIC @ MICCAI},
  year = {2018},
  url = {https://openreview.net/forum?id=oYEC2F4TPM},
}

Loading related content...

Figure