TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research

William Ndzimbong, Cyril Fourniol, Loic Themyr, Nicolas Thome, Yvonne Keeza, Benoît Sauer, Pierre-Thierry Piechaud, Arnaud Mejean, Jacques Marescaux, Daniel George, Didier Mutter, Alexandre Hostettler, Toby CollinsarXiv 2023 · DOI · arXiv

Introduction

In the field of image-guided surgery and robotics, real-time visualization of target organs is essential for precision and safety. While CT and MRI provide high-resolution anatomical maps, they are not feasible for intra-operative use. Ultrasound (US) is an ideal companion due to its portability and lack of ionizing radiation, yet it remains difficult to interpret because of low contrast, speckle noise, and acoustic shadows from the rib cage. Inter-Modal Image Registration (IMIR) attempts to bridge this gap by overlaying pre-operative CT scans onto real-time US feeds. However, progress has been limited by a lack of publicly available, paired human datasets for training and benchmarking algorithms.

The TRUSTED Dataset

The Tridimensional Renal Ultra Sound TomodEnsitometrie (TRUSTED) dataset is the first public collection of paired 3D transabdominal US and CT scans from human patients specifically for kidney research. It includes data from 48 patients, providing a total of 96 kidneys for study.

  • CT Data: Three-phase scans (non-contrast, venous, and portal).
  • US Data: 59 high-quality 3DUS volumes where the kidney is fully contained within the field of view.
  • Annotations: The dataset features binary masks (segmentations), surface meshes, and 3D coordinates for seven anatomical landmarks per kidney.
Visual representation of manual segmentation and anatomical landmarking for both CT and US modalities.

Methodology

To establish a "gold standard" ground truth, the researchers employed a rigorous annotation pipeline. Two experienced radiographers independently annotated each kidney. To reconcile inter-annotator differences, the team used the Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm to generate consensus segmentations. The dataset was then used to benchmark five state-of-the-art deep learning (DL) models—including nnUNet and CoTr—and several registration methods such as Iterative Closest Point (ICP) and Bayesian Coherent Point Drift (BCPD).

Key Results and Insights

The "US Difficulty Gap"

The benchmarking results highlighted a significant performance gap between modalities. While AI models achieved nearly 90% accuracy (Dice score) on CT data, they peaked at approximately 79% for ultrasound. This discrepancy underscores that automated US segmentation remains a complex, unsolved challenge due to the modality's inherent noise and artifacts.

Segmentation Performance

Among the tested models, nnUNet was the most robust for ultrasound segmentation, achieving a Dice score of 79.4%. For CT scans, the CoTr (Convolutional Transformer) model performed best with a Dice score of 89.1%.

Quantitative comparison of segmentation performance metrics (Dice score, NN distance, and HD95 distance) across various deep learning models.

Registration and Surface-Based Approaches

The study found that surface-based registration—aligning the extracted shapes of the kidneys—generally outperformed intensity-based registration (aligning raw voxels). The Bayesian Coherent Point Drift (BCPD) method using affine transforms emerged as the top performer for registration, achieving an average Target Registration Error (TRE) of 4.53mm. Sensitivity analyses further revealed that surface-based methods are significantly more resilient to poor initial alignments than intensity-based methods.

Discussion and Future Work

The TRUSTED dataset reveals that even human experts exhibit slight disagreements (94% agreement), highlighting the inherent ambiguity in medical imaging. This "label uncertainty" is a critical feature for researchers developing AI that must navigate real-world clinical noise.

While the 59 3DUS volumes represent a major step forward for public data, the dataset size remains a limitation for training massive deep learning models from scratch. Future work will focus on developing "shadow-aware" architectures to handle rib interference and exploring "Double Target" training strategies that utilize multiple human annotations to improve model robustness.

Summary updated

Abstract

Inter-modal image registration (IMIR) and image segmentation with abdominal Ultrasound (US) data has many important clinical applications, including image-guided surgery, automatic organ measurement and robotic navigation. However, research is severely limited by the lack of public datasets. We propose TRUSTED (the Tridimensional Renal Ultra Sound TomodEnsitometrie Dataset), comprising paired transabdominal 3DUS and CT kidney images from 48 human patients (96 kidneys), including segmentation, and anatomical landmark annotations by two experienced radiographers. Inter-rater segmentation agreement was over 94 (Dice score), and gold-standard segmentations were generated using the STAPLE algorithm. Seven anatomical landmarks were annotated, important for IMIR systems development and evaluation. To validate the dataset's utility, 5 competitive Deep Learning models for automatic kidney segmentation were benchmarked, yielding average DICE scores from 83.2% to 89.1% for CT, and 61.9% to 79.4% for US images. Three IMIR methods were benchmarked, and Coherent Point Drift performed best with an average Target Registration Error of 4.53mm. The TRUSTED dataset may be used freely researchers to develop and validate new segmentation and IMIR methods.
BibTeX
@misc{ndzimbong2023trusted,
  author = {William Ndzimbong and Cyril Fourniol and Loic Themyr and Nicolas Thome and Yvonne Keeza and Benoît Sauer and Pierre-Thierry Piechaud and Arnaud Mejean and Jacques Marescaux and Daniel George and Didier Mutter and Alexandre Hostettler and Toby Collins},
  title = {{TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research}},
  year = {2023},
  eprint = {2310.12646},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.12646},
}
Figure