FetalNet: Multi-task Deep Learning Framework for Fetal Ultrasound Biometric Measurements

Szymon Plotka, Tomasz Wlodarczyk, Adam Klasa, Michal Lipa, Arkadiusz Sitek, Tomasz Trzcinski

Abstract

FetalNet is an end-to-end multi-task deep learning framework designed to automate fetal biometric measurements from ultrasound videos. By leveraging spatio-temporal features through a ConvLSTM bottleneck and refining results with attention gates and multi-scale feature stacking, it outperforms current state-of-the-art methods in both identifying and measuring fetal body parts.

Introduction

Fetal biometry—measuring the fetal head, abdomen, and femur—is a cornerstone of prenatal care used to monitor growth and estimate gestational age. However, performing these measurements is notoriously difficult. Ultrasound images are plagued by "speckle noise," blur, and shadows. Furthermore, a sonographer must navigate a 3D space to find a specific 2D "standard plane" that meets strict clinical guidelines for measurement.

Most existing computer-aided tools treat this as a static image problem, analyzing single frames in isolation and ignoring the temporal context available in a live video feed. Additionally, many models are "single-task," focusing only on segmentation or only on classification. FetalNet aims to bridge this gap by creating a system that works with video sequences to simultaneously localize, classify, and measure fetal structures just like a human expert would.

Methods: The FetalNet Architecture

The researchers proposed a novel architecture called FetalNet, built on an encoder-decoder (U-Net) backbone with several sophisticated components designed to handle the complexities of ultrasound video:

The FetalNet architecture featuring ConvLSTM, Attention Gates (AG), and Stacked Modules (SM).
  • Spatio-Temporal Bottleneck: Instead of a standard flat bottleneck, FetalNet uses ConvLSTM cells. This allows the network to retain spatial-temporal features in memory, helping it understand how structures move and appear across consecutive video frames.
  • Attention Gates (AG): Integrated into the skip connections, these help the model focus on the region of interest (the fetus) while suppressing irrelevant noise and background artifacts.
  • Stacked Module (SM): To improve segmentation accuracy, the model aggregates multi-scale feature maps from the decoder, learning both global context and local boundary details.
  • Multi-task Learning: The network splits into two branches: one for binary segmentation masks and another for frame classification (head, abdomen, femur, or background).
  • Automated Biometry: Post-processing techniques like Ramer-Douglas-Peucker approximation and ellipse fitting are used to calculate clinical metrics like Head Circumference (HC).

Datasets

The study utilized a substantial dataset of 700 2D fetal ultrasound video sequences from 700 different patients, totaling over 274,000 frames. Six experienced sonographers annotated 62,324 "standard views" and over 211,000 background frames. Data was collected across six research institutes using various GE Voluson scanners, ensuring high diversity and robustness.

Key Results

FetalNet demonstrated superior performance across the board compared to benchmarks like U-Net, DeepLabv3, and FCN. It achieved a Jaccard Index (IoU) of 0.905 and a Dice Coefficient of 0.962, significantly higher than the base U-Net (0.862 IoU). The classification branch reached an F1-score of 0.96, showing high reliability in distinguishing between body parts.

Visual results of FetalNet segmentation and ellipse fitting for the fetal head, abdomen, and femur.

Crucially, measurement accuracy improved significantly. The error for head measurements was reduced to 2.9mm ± 1.2mm, compared to 4.5mm for the standard U-Net. This brings AI precision closer to the level of experienced human operators, potentially reducing inter-operator variability in clinical settings.

Discussion and Future Work

By using video sequences rather than static images, FetalNet can "smooth out" transient noise. If a shadow obscures a bone in one frame, the ConvLSTM helps the model remember its position from the previous frame. The inclusion of a robust "background" class is also critical for real-world clinical use, where most scanning time is spent on non-standard planes.

While FetalNet is a major step forward, future work will focus on computational efficiency for portable devices and expanding the scope to detect fetal abnormalities and estimate gestational age directly.

Source Code: https://github.com/SanoScience/FetalNet

Summary updated

Abstract

Fetal biometric measurements are routinely done during pregnancy for the fetus growth monitoring and estimation of gestational age and fetal weight. The main goal in fetal ultrasound scan video analysis is to find standard planes to measure the fetal head, abdomen and femur. In this paper, we propose an end-to-end multi-task neural network called FetalNet with an attention mechanism and stacked module for spatio-temporal fetal ultrasound scan video analysis to simultaneously localize, classify and measure the fetal body parts. We employ an attention mechanism with a stacked module to learn salient maps to suppress irrelevant ultrasound regions and efficient scan plane localization. We train on the fetal ultrasound video from routine examinations of 700 different patients. Our method called FetalNet outperforms existing state-of-the-art methods in both classification and segmentation in fetal ultrasound video recordings. The source code and pre-trained weights are publicly available (https://github.com/SanoScience/FetalNet).
BibTeX
@inproceedings{plotka2021fetalnet,
  author = {Szymon Plotka and Tomasz Wlodarczyk and Adam Klasa and Michal Lipa and Arkadiusz Sitek and Tomasz Trzcinski},
  title = {{FetalNet: Multi-task Deep Learning Framework for Fetal Ultrasound Biometric Measurements}},
  booktitle = {ICONIP},
  year = {2021},
  url = {https://openreview.net/forum?id=mGsCthfhyp},
}
Figure