TL;DR
This paper tackles the challenge of automatically segmenting placentas in ultrasound images, which are often hampered by poor image quality, variable annotations, and limited field-of-view. They introduce a novel multi-task learning approach that combines placental location classification with segmentation, achieving human-level performance and enabling whole-placenta segmentation through a multi-view imaging pipeline.
What Problem Does This Paper Address?
Fetal ultrasound (US) is a crucial tool for monitoring pregnancy, but assessing the placenta can be difficult. The authors highlight three main challenges:
- Variable Placenta Appearance: Placentas can appear very differently in US images, especially depending on their location (anterior vs. posterior), with posterior placentas often suffering from shadowing and attenuation artifacts.
- Annotation Uncertainty: US image quality is inherently limited, leading to inconsistent and subjective manual annotations by experts, making it hard to train robust models.
- Limited Field-of-View (FoV): Especially in later gestation, US probes can't capture the entire placenta in a single image, preventing a complete assessment.
These challenges limit the ability to quantitatively assess placental health and identify potential pathologies, which can impact maternal and fetal outcomes.
Methods
The core of the paper is a multi-task learning framework designed to improve placenta segmentation.
- Multi-Task Learning: They propose a single convolutional neural network (CNN) that simultaneously performs two tasks:
- Placental Location Classification (Pretext Task): Classifying whether an image contains an anterior placenta, a posterior placenta, or no placenta. This task leverages readily available clinical data (placental location is routinely recorded) and acts as a "pretext" task to help the model learn more robust features.
- Semantic Placenta Segmentation (Downstream Task): The primary goal of segmenting the placenta pixels.
- Transfer Learning (TUNet): They also explore a transfer learning approach where a pre-trained classification network's encoder is used to initialize a segmentation network.
- Multi-View Imaging Pipeline: To overcome the limited FoV, they developed a pipeline involving:
- Multi-Probe Image Acquisition: Using a custom-designed holder to acquire US images from multiple angles simultaneously.
- Image Fusion: A voxel-based weighted fusion strategy to combine these multi-view images, suppressing artifacts and extending the effective FoV.
- Segmentation: Applying their multi-task segmentation model to these fused multi-view images.
- Uncertainty Modeling: They use Monte Carlo (MC) dropout during inference to estimate model uncertainty, allowing them to compare automatic segmentation uncertainty with human inter- and intra-observer variability.
Datasets
- Classification Dataset (IC): A dataset of 1188 images from 71 healthy volunteers, labeled with anterior, posterior, or no placenta.
- Segmentation Dataset (IS): A subset of 292 images from 57 patients, with detailed voxel-wise manual segmentations.
- Multi-View Dataset: A subset of images acquired using their multi-probe system, with manual segmentations for both single and fused multi-view images.
- Variability Dataset: A subset of 53 images from 12 patients, annotated by multiple raters (including intra-rater comparisons) to assess variability.
The authors did not introduce a new benchmark dataset but rather utilized existing data and created new multi-view acquisitions.
Key Insights and Ideas
- Placental Location as a Powerful Pretext Task: Classifying placental location (anterior/posterior) significantly improves segmentation performance, especially for "out-of-distribution" (OoD) data (e.g., training on anterior placentas and testing on posterior). This suggests that anterior and posterior placentas represent distinct data distributions for segmentation.
- Human-Level Performance: The multi-task approach achieves segmentation accuracy (Dice scores of 0.86 for anterior and 0.83 for posterior) comparable to intra- and inter-observer variability, indicating human-level performance.
- Multi-View Fusion is Key for Late Gestation: The multi-probe acquisition and fusion pipeline is essential for capturing the entire placenta in later pregnancy stages, overcoming the FoV limitation of single probes.
- Posterior Placentas are Inherently More Challenging: Results consistently show that segmenting posterior placentas is more difficult due to image artifacts and lower quality, and the multi-task approach shows greater improvement in these cases.
Key Results
- Classification Performance: The baseline classification model (EncNet) achieved high balanced accuracy, precision, and F1-scores (e.g., ~0.91 for anterior, ~0.90 for none, ~0.90 for posterior). Multi-task models performed competitively, even with smaller classification training sets.
- Segmentation Performance (Single View):
- In-Distribution (InD): Models trained on specific locations (e.g., set A for anterior) achieved high Dice scores (~0.84 for anterior, ~0.79 for posterior).
- Out-of-Distribution (OoD): Segmentation performance dropped significantly for models trained on only one location when tested on the other (e.g., Dice of 0.26 for posterior when trained on anterior).
- Multi-Task Improvement: Multi-task models (MTUNet, TMTUNet) significantly improved OoD performance, with TMTUNet showing the best results (e.g., Dice of 0.45 for posterior when trained on anterior).
- Overall Best (Set AP): MTUNet achieved the best overall performance on the combined anterior/posterior training set (AP), with Dice scores around 0.86 for anterior and 0.80 for posterior.
- Segmentation Performance (Multi-View): Multi-view segmentation showed improved performance, especially for OoD data, likely due to artifact reduction. MTUNet performed best on the multi-view data.
- Variability: Intra-observer agreement was higher than inter-observer agreement, highlighting the subjective nature of manual annotation. Posterior placentas showed lower agreement. Automatic segmentations from MTUNet were found to lie within human inter-rater variability.
- Placental Volume Analysis: Volumes extracted from MTUNet segmentations showed good agreement with manual segmentations and reference values from MRI studies, though outliers were observed for anterior placentas due to FoV limitations.
Limitations and Future Work
- Limited Placenta Location Classes: The study primarily used two classes (anterior/posterior). Future work could incorporate a finer classification of placental locations.
- Shared Encoder Architecture: While effective, the shared encoder for both tasks might not be optimal. Exploring more sophisticated network architectures for task branching could be beneficial.
- Generalizability to Abnormal Cases: The current study focused on healthy volunteers. Extending the models to detect placental pathologies and abnormal cases is a crucial next step.
- Twin Pregnancies: The study focused on singleton pregnancies. Investigating performance on twin pregnancies (monochorionic and dichorionic) is a potential future direction.
- Limited Rater Comparison: The variability analysis involved a limited number of raters. Expanding this to a larger group would strengthen the findings.
Summary updated