Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation

Hanchao Yu, Xiao Chen, Humphrey Shi, Terrence Chen, Thomas S. Huang, Shanhui Sun

Abstract

Cardiac motion estimation plays a key role in MRI cardiac feature tracking and function assessment such as myocardium strain. In this paper, we propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance. Our teacher model provides more accurate motion estimation as supervision through progressive motion compensations. Our student model learns from the teacher model to estimate motion in a single step while maintaining accuracy. The teacher-student knowledge distillation is performed in a cyclic way for a further performance boost. Our proposed method outperforms a strong baseline model on two public available clinical datasets significantly, evaluated by a variety of metrics and the inference time. New evaluation metrics are also proposed to represent errors in a clinically meaningful manner.

Introduction

Estimating the motion of the heart from Cardiac MRI (CMR) is vital for calculating clinical markers like myocardial strain, which can detect heart disease earlier than traditional metrics. However, this task faces three major hurdles: the lack of ground truth labels for a beating heart, the "smoothness paradox" where constraints to keep heart shapes realistic often prevent capturing rapid movements, and the inherent trade-off between efficiency and accuracy. Traditional registration methods are precise but slow, while many deep learning models struggle with the large deformations of a full cardiac cycle.

To address these challenges, we developed Motion Pyramid Networks (MPN). This framework achieves state-of-the-art accuracy in cardiac motion estimation while remaining fast enough for real-time clinical use by combining a multi-scale feature pyramid with a novel "cyclic teacher-student" distillation strategy.

Methodology

The MPN approach utilizes three core components to optimize motion estimation:

  1. Multi-scale Motion Pyramid: The network extracts features at multiple scales, predicting and fusing motion fields at each level. This allows the model to capture both broad global movements and fine local details simultaneously.
  2. Progressive Motion Compensation (The Teacher): To handle large motions without violating smoothness constraints, a "teacher" model (MPN-2) performs estimation in two steps, breaking down large deformations into manageable intermediate steps.
  3. Cyclic Knowledge Distillation (The Student): To maintain clinical efficiency, a "student" model (MPN-S) is trained to mimic the teacher's refined multi-step output in a single inference step. In a cyclic process, once the student converges, it becomes the new teacher for the next round of training, continuously boosting performance.
Overview of the MPN architecture, including feature extraction, motion prediction, and the cyclic teacher-student training strategy.

Results and Discussion

The proposed MPN-C (Cyclic) model was evaluated on the ACDC and Kaggle (Data Science Bowl) datasets. It significantly outperformed both traditional Free-Form Deformation (FFD) and strong deep-learning baselines in both speed and accuracy.

  • Efficiency: The MPN-C model processes a frame in just 26 milliseconds, a nearly 30x speedup compared to the 748 milliseconds required by traditional FFD methods.
  • Accuracy: On the ACDC dataset, the model achieved a Dice score of 0.918 for the Left Ventricle (LV), surpassing the baseline of 0.885.
  • Clinical Precision: We introduced Endpoint Error (EPE) decomposition, breaking errors down into radial ($\epsilon_{rr}$) and circumferential ($\epsilon_{cc}$) directions. This is crucial as heart function is assessed by muscle thickening (radial) and shortening (circumferential). Results indicated that radial tracking is captured with particularly high fidelity.
Quantitative results comparing MPN variants against baselines using clinical metrics including Endpoint Error (EPE) and strain errors.

Conclusion

Motion Pyramid Networks offer a promising path toward real-time, automated cardiac analysis. By using knowledge distillation for temporal compression—forcing a single-step student to match a multi-step teacher—we achieve the accuracy of iterative refinement without the computational tax at runtime. Future work will address 3D "out-of-plane" motion and further clinical validation to correlate these motion estimates with specific cardiac pathologies.

Summary updated

Abstract

Cardiac motion estimation plays a key role in MRI cardiac feature tracking and function assessment such as myocardium strain. In this paper, we propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance. Our teacher model provides more accurate motion estimation as supervision through progressive motion compensations. Our student model learns from the teacher model to estimate motion in a single step while maintaining accuracy. The teacher-student knowledge distillation is performed in a cyclic way for a further performance boost. Our proposed method outperforms a strong baseline model on two public available clinical datasets significantly, evaluated by a variety of metrics and the inference time. New evaluation metrics are also proposed to represent errors in a clinically meaningful manner.
BibTeX
@inproceedings{yu2020motion,
  author = {Hanchao Yu and Xiao Chen and Humphrey Shi and Terrence Chen and Thomas S. Huang and Shanhui Sun},
  title = {{Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=4DYS2a_eseb},
}
Figure