WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration

Xinxing Cheng, Xi Jia, Wenqi Lu, Qiufu Li, Linlin Shen, Alexander Krull, Jinming Duan

Abstract

Deep image registration has demonstrated exceptional accuracy and fast inference. Recent advances have adopted either multiple cascades or pyramid architectures to estimate dense deformation fields in a coarse-to-fine manner. However, due to the cascaded nature and repeated composition/warping operations on feature maps, these methods negatively increase memory usage during training and testing. Moreover, such approaches lack explicit constraints on the learning process of small deformations at different scales, thus lacking explainability. In this study, we introduce a model-driven WiNet that incrementally estimates scale-wise wavelet coefficients for the displacement/velocity field across various scales, utilizing the wavelet coefficients derived from the original input image pair. By exploiting the properties of the wavelet transform, these estimated coefficients facilitate the seamless reconstruction of a full-resolution displacement/velocity field via our devised inverse discrete wavelet transform (IDWT) layer. This approach avoids the complexities of cascading networks or composition operations, making our WiNet an explainable and efficient competitor with other coarse-to-fine methods. Extensive experimental results from two 3D datasets show that our WiNet is accurate and GPU efficient.

Introduction

Deformable image registration is a fundamental task in medical image analysis, essential for aligning anatomical structures across different subjects or time points. While deep learning-based registration has significantly reduced inference times compared to traditional iterative methods, current state-of-the-art architectures face a significant bottleneck: computational efficiency.

Most modern frameworks utilize "coarse-to-fine" strategies, employing cascaded networks or feature pyramids to handle deformations of varying magnitudes. While effective at capturing both global alignment and local details, these methods are notoriously memory-intensive. They rely on repeated warping and composition operations that consume vast amounts of GPU VRAM, often exceeding the capacity of standard hardware. Furthermore, these architectures are frequently criticized for being "black boxes," as they lack explicit mechanisms to explain how deformations are partitioned across different spatial scales.

To address these challenges, we present WiNet, a novel framework that leverages the Discrete Wavelet Transform (DWT) to achieve high-fidelity registration with unprecedented efficiency and explainability.

Methodology

WiNet departs from standard pyramid-based architectures by operating in the wavelet domain. The core innovation lies in treating deformation as a multi-scale signal that can be decomposed and reconstructed using wavelet coefficients.

1. DWT-Embedded Encoder

Instead of relying on traditional pooling layers for downsampling, WiNet utilizes a differentiable DWT layer. This decomposes the input image pair into one low-frequency sub-band and seven high-frequency sub-bands. This approach halves the spatial resolution while preserving all information from the original signal, allowing the network to process compressed representations without the loss of detail typical of max-pooling or strided convolutions.

2. Incremental Deformation Learning

WiNet estimates the deformation field incrementally. Rather than predicting a dense displacement field at every level, the network predicts scale-specific wavelet coefficients. The process begins at a low resolution to capture coarse global movements and progressively refines the coefficients to account for finer anatomical details. This frequency-based separation provides a natural proxy for scale, making the learning process more transparent.

3. Model-Driven Reconstruction (IDWT Layer)

To generate the final high-resolution deformation field ($\phi$), we introduce a parameter-free Inverse DWT (IDWT) layer. This layer acts as a model-driven reconstruction tool, seamlessly integrating the learned coefficients from various scales. By reconstructing the field through IDWT, WiNet eliminates the need for the redundant warping and composition operations that plague traditional cascaded networks, significantly reducing the memory footprint.

4. Diffeomorphic Variant (WiNet-Diff)

To ensure that the resulting transformations are physically plausible—meaning they preserve topology and prevent "folding" of tissues—we implemented WiNet-Diff. This variant estimates velocity fields rather than direct displacements, ensuring the final transformation is smooth, invertible, and biologically realistic.

Experimental Results

We evaluated WiNet on two distinct 3D medical imaging tasks: atlas-based brain MRI registration (IXI dataset) and cardiac MRI alignment (3D-CMR dataset).

Accuracy and Plausibility

WiNet demonstrated competitive and often superior accuracy compared to established models. On the IXI brain dataset, WiNet-Diff achieved a Dice score of 0.765, surpassing top-tier models such as LapIRN (0.760) and the transformer-based ModeT (0.758). Crucially, WiNet-Diff achieved 0% folding (measured by negative Jacobian determinants), ensuring the highest level of topological preservation.

Computational Efficiency

The most striking result of WiNet is its efficiency. By avoiding the memory-heavy operations of pyramid architectures, WiNet drastically reduced GPU resource requirements:

  • Memory Usage: WiNet consumed only 31.9% of the memory required by LapIRN and 23.2% of that required by ModeT. In practical terms, this allows WiNet to run on hardware with ~7GB of VRAM, whereas competitors often require ~30GB.
  • Inference Speed: WiNet maintained faster inference times than all other tested coarse-to-fine competitors, making it highly suitable for time-sensitive clinical environments.

Discussion and Conclusion

WiNet represents a significant step forward in making high-resolution medical image registration accessible and explainable. By leveraging the mathematical properties of wavelets, the framework achieves a sparse yet powerful representation of anatomical deformations. This sparsity is the key to its massive reduction in memory consumption without sacrificing registration accuracy.

While the current implementation shows excellent results using Haar and Daubechies wavelets, future work will explore different wavelet families tailored to specific tissue types. Additionally, we aim to test WiNet's robustness on more complex clinical cases, such as images containing large tumors or significant post-surgical changes.

In conclusion, WiNet offers a model-driven, efficient, and explainable alternative to existing registration frameworks, proving that frequency-domain operations can solve the scalability issues of modern deep learning in medical imaging.

Summary updated

Abstract

Deep image registration has demonstrated exceptional accuracy and fast inference. Recent advances have adopted either multiple cascades or pyramid architectures to estimate dense deformation fields in a coarse-to-fine manner. However, due to the cascaded nature and repeated composition/warping operations on feature maps, these methods negatively increase memory usage during training and testing. Moreover, such approaches lack explicit constraints on the learning process of small deformations at different scales, thus lacking explainability. In this study, we introduce a model-driven WiNet that incrementally estimates scale-wise wavelet coefficients for the displacement/velocity field across various scales, utilizing the wavelet coefficients derived from the original input image pair. By exploiting the properties of the wavelet transform, these estimated coefficients facilitate the seamless reconstruction of a full-resolution displacement/velocity field via our devised inverse discrete wavelet transform (IDWT) layer. This approach avoids the complexities of cascading networks or composition operations, making our WiNet an explainable and efficient competitor with other coarse-to-fine methods. Extensive experimental results from two 3D datasets show that our WiNet is accurate and GPU efficient. The code is available at https://github.com/x-xc/WiNet .
BibTeX
@inproceedings{cheng2024winet,
  author = {Xinxing Cheng and Xi Jia and Wenqi Lu and Qiufu Li and Linlin Shen and Alexander Krull and Jinming Duan},
  title = {{WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2024},
  url = {https://openreview.net/forum?id=pdKixsOyLS},
}
Figure