SMPL-A: Modeling Person-Specific Deformable Anatomy

Hengtao Guo, Benjamin Planche, Meng Zheng, Srikrishna Karanam, Terrence Chen, Ziyan Wu

Introduction

In medical procedures such as radiotherapy and surgery, the precise localization of internal organs is paramount. However, human anatomy is not static; internal organs undergo significant deformation and displacement based on a patient's external pose. For example, lifting arms above the head can shift the position of the lungs and liver considerably.

Currently, clinicians rely on restrictive "proxy procedures," such as custom molds, to ensure patients maintain identical poses between initial scans and treatment. Alternatively, frequent intraoperative imaging is used, which increases patient exposure to cumulative radiation. SMPL-A addresses this challenge by extending the standard SMPL (Skinned Multi-Person Linear) body model to predict internal anatomical changes based on external body movement.

Overview of the SMPL-A framework, showing how person-specific anatomy is estimated from medical scans and re-posed based on external body movement.

Methodology: The SMPL-A Framework

The core innovation of SMPL-A is the introduction of a person-specific anatomy parameter, $\alpha$, which complements the standard SMPL pose ($\theta$) and shape ($\beta$) parameters. This allows the model to disentangle individual anatomical identity from transient postural changes.

The Encoder-Decoder Architecture

The system utilizes two primary neural network components to manage these parameters:

  1. Anatomy Encoder ($E$): This component processes a 3D point cloud of an organ (derived from a medical scan) along with the patient's current pose and shape. It compresses this information into a low-dimensional vector $\alpha$, capturing the unique morphology and elastic properties of the patient's internal structures.
  2. Pose-Conditioned Decoder ($D$): The decoder takes the patient's $\alpha$ vector and a target pose ($\theta'$) to predict the resulting deformed organ mesh ($M_{pred}$).
The SMPL-A architecture, detailing the interaction between the Anatomy Encoder (E) and the Pose-Conditioned Decoder (D) to predict organ shapes.

To ensure anatomical consistency across different patients, the researchers employed 3D deformable registration (Elastix), establishing a vertex-to-vertex correspondence for each organ. Furthermore, a smoothness regularization loss ($L_{smooth}$) was implemented to ensure that the predicted deformations remain physically and medically plausible.

Data and Training

The model was developed using a dataset of 30 clinical cases featuring segmentations of the skin, lungs, kidneys, liver, and spleen. Due to the practical and ethical difficulties of obtaining multiple medical scans of the same individual in various poses, the authors utilized Finite Element Modeling (FEM). This simulation-based approach allowed for the generation of synthetic "ground truth" sequences, modeling how organs stretch and shift during arm-lifting motions to train the model's pose-dependency.

Key Results and Insights

SMPL-A demonstrates a significant improvement in anatomical reconstruction accuracy compared to standard baseline models. For instance, the reconstruction error for the left lung was reduced from a baseline of 4.48mm to 2.25mm using the SMPL-A framework. Statistical analysis via paired t-tests confirmed these improvements were significant ($p < 0.05$) across all six studied organ parts.

Beyond accuracy, the model's latent space proved to be interpretable. Visualizing the dimensions of the $\alpha$ vector revealed correlations with physical traits such as organ orientation and thickness. The inclusion of the smoothness constraint was also found to be critical; without it, predicted meshes often exhibited unrealistic surface artifacts.

Conclusion and Future Directions

SMPL-A represents a meaningful bridge between computer vision tasks like human mesh recovery and medical imaging tasks like anatomical reconstruction. By learning to "re-pose" internal anatomy, the model offers a potential path toward reducing the need for frequent radiation-based imaging.

Future work will focus on reducing the dependency on FEM simulations by incorporating more diverse clinical data. Additionally, the researchers aim to expand the model to account for more complex articulations, respiratory motion (breathing), and temporal anatomical changes, such as tumor shrinkage or weight loss during long-term treatment courses.

Summary updated

Abstract

A variety of diagnostic and therapeutic protocols rely on locating in vivo target anatomical structures, which can be obtained from medical scans. However, organs move and deform as the patient changes his/her pose. In order to obtain accurate target location information, clinicians have to either conduct frequent intraoperative scans, resulting in higher exposition of patients to radiations, or adopt proxy procedures (e.g., creating and using custom molds to keep patients in the exact same pose during both preoperative organ scanning and subsequent treatment. Such custom proxy methods are typically sub-optimal, constraining the clinicians and costing precious time and money to the patients. To the best of our knowledge, this work is the first to present a learning-based approach to estimate the patient's internal organ deformation for arbitrary human poses in order to assist with radiotherapy and similar medical protocols. The underlying method first leverages medical scans to learn a patient-specific representation that potentially encodes the organ's shape and elastic properties. During inference, given the patient's current body pose information and the organ's representation extracted from previous medical scans, our method can estimate their current organ deformation to offer guidance to clinicians. We conduct experiments on a well-sized dataset which is augmented through real clinical data using finite element modeling. Our results suggest that pose-dependent organ deformation can be learned through a point cloud autoencoder conditioned on the parametric pose input. We hope that this work can be a starting point for future research towards closing the loop between human mesh recovery and anatomical reconstruction, with applications beyond the medical domain.
BibTeX
@inproceedings{guo2022smpl,
  author = {Hengtao Guo and Benjamin Planche and Meng Zheng and Srikrishna Karanam and Terrence Chen and Ziyan Wu},
  title = {{SMPL-A: Modeling Person-Specific Deformable Anatomy}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.02015},
}
Figure