Abstract
Deep image registration has demonstrated exceptional accuracy and fast inference. Recent advances have adopted either multiple cascades or pyramid architectures to estimate dense deformation fields in a coarse-to-fine manner. However, due to the cascaded nature and repeated composition/warping operations on feature maps, these methods negatively increase memory usage during training and testing. Moreover, such approaches lack explicit constraints on the learning process of small deformations at different scales, thus lacking explainability. In this study, we introduce a model-driven WiNet that incrementally estimates scale-wise wavelet coefficients for the displacement/velocity field across various scales, utilizing the wavelet coefficients derived from the original input image pair. By exploiting the properties of the wavelet transform, these estimated coefficients facilitate the seamless reconstruction of a full-resolution displacement/velocity field via our devised inverse discrete wavelet transform (IDWT) layer. This approach avoids the complexities of cascading networks or composition operations, making our WiNet an explainable and efficient competitor with other coarse-to-fine methods. Extensive experimental results from two 3D datasets show that our WiNet is accurate and GPU efficient.
Introduction
Deformable medical image registration—the process of aligning a "moving" image to a "fixed" image by calculating a dense, non-linear deformation field—is a critical but computationally expensive step in medical imaging. While deep learning models have drastically sped up inference times compared to traditional iterative methods, capturing large and complex deformations remains a challenge. To solve this, recent state-of-the-art networks use "coarse-to-fine" cascades or pyramid architectures, estimating small deformations at low resolutions and progressively refining them at higher resolutions.
However, these pyramid methods rely on repeatedly warping and composing high-dimensional feature maps at multiple scales. This causes GPU memory usage to skyrocket, making them difficult to train on standard hardware. Furthermore, these "black box" compositions lack explicit constraints on how small deformations are learned at each scale, reducing the interpretability of the model.
Methods: Wavelet-Based Incremental Learning
To solve the memory and explainability bottleneck, WiNet embeds the Discrete Wavelet Transform (DWT) directly into its architecture.
The architecture consists of several key components:
- DWT Embedded Encoder: Instead of standard pooling, WiNet uses a parameter-free, differentiable DWT layer to decompose the 3D input images into eight frequency sub-bands. This halves the spatial resolution without losing structural information, saving immediate computational cost.
- Incremental Deformation Learning Module: Instead of directly predicting the deformation field at each scale, the decoder predicts wavelet coefficients. It starts by estimating coarse coefficients, then uses custom Refinement Blocks (RBs) to incrementally predict higher-frequency details.
- IDWT Reconstruction: Because the network explicitly learns these coefficients, it can use an Inverse Discrete Wavelet Transform (IDWT) layer to seamlessly reconstruct the full-resolution deformation field, bypassing memory-heavy feature map warping.
- Diffeomorphic Variant (WiNet-Diff): A variant using scaling and squaring layers ensures the resulting deformation field preserves topology, preventing image folding.
By operating in the wavelet domain, the network manipulates low-resolution frequency components rather than high-resolution spatial feature maps. This mathematical property is the primary driver of WiNet's massive memory savings. Furthermore, because the network explicitly outputs low- and high-frequency wavelet coefficients at each scale, the registration process becomes highly interpretable.
Experimental Results
The authors evaluated WiNet on two standard 3D medical imaging datasets: the IXI Dataset (3D brain MRI) and the 3D-CMR Dataset (3D cardiac MRI).
Efficiency and Performance
- Massive Memory Reduction: WiNet requires only ~6.9 GB (6977 MiB) of GPU memory for training. This is significantly lower than competing pyramid methods like LapIRN, PRNet++, and ModeT, representing a reduction of up to 75%.
- State-of-the-Art Accuracy: Despite the smaller memory footprint, WiNet-Diff achieved top Dice scores of 0.765 on IXI and 0.832 on 3D-CMR, outperforming traditional methods and matching or beating heavier pyramid architectures.
- Faster Inference: WiNet demonstrated faster average GPU inference times than all three competing pyramid architectures, as shown in the performance comparison.
- Anatomical Plausibility: The diffeomorphic version (WiNet-Diff) achieved near-zero folding (measured by the Jacobian determinant), ensuring smooth and realistic deformations.
Conclusion
WiNet represents a significant step forward in efficient medical image registration. By leveraging the mathematical properties of wavelets, it achieves state-of-the-art accuracy with a fraction of the memory requirements of previous methods. While the current framework relies on predefined orthogonal wavelet transforms, future work could explore learnable wavelet families or adapt the architecture for unsupervised multi-modal registration tasks.
Summary updated