Abstract
This paper introduces DeGIA (Deep learning based Generalized Image Alignment), a framework designed to align images in unseen target domains (e.g., Depth or X-Ray) using models trained exclusively on a single source domain (e.g., RGB). By integrating discriminative feature extraction with deep generative flows and an adversarial training strategy, DeGIA achieves robust domain generalization. Experimental results demonstrate significant performance gains over standard CNN architectures, particularly in cross-modal medical and hand-pose estimation tasks.
Introduction
Image alignment—the task of identifying specific landmark points on an object—is a fundamental problem in computer vision. While performance is high when training and testing data share the same distribution, real-world applications often face domain shift. In medical imaging, for instance, labeled RGB data may be abundant, while labeled X-ray or Depth maps are scarce.
Traditional domain adaptation requires target-domain data during training. However, this paper addresses the more challenging task of Domain Generalization, where the model must perform on entirely new modalities unavailable during the training phase. This is essential for privacy-sensitive or resource-constrained environments.
Methodology: The DeGIA Framework
The proposed Deep learning based Generalized Image Alignment (DeGIA) framework is an end-to-end architecture comprising two main components:
- Discriminative Template Feature Extraction: A CNN backbone (e.g., ResNet, VGG) trained to identify landmarks by minimizing Mean Squared Error (MSE).
- Deep Generative Flows: These map images into a learned latent space where data is represented as Gaussian distributions. This space is more "domain-invariant" than raw pixel values, focusing on structural features rather than stylistic noise.
Training Strategy
DeGIA employs a two-phase optimization process:
- Minimization Phase: The network learns to align landmarks on source data while maximizing the log-likelihood of the generative flow.
- Maximization (Perturbation) Phase: The model generates "hard" adversarial samples to maximize loss. This forces the model to become robust against distortions and shifts likely to be encountered in unseen domains.
Importantly, the generative flow is used only for regularization during training; inference speed at test time remains comparable to a standard CNN.
Experimental Evaluation
The authors evaluated DeGIA across several benchmarks to test its generalization capabilities:
- MNIST to MNIST-M: A proof-of-concept where the model trained on standard digits and was tested on digits with complex backgrounds.
- NYU Hand Pose Dataset: The primary testbed where models were trained on RGB images and tested on Depth and X-Ray modalities.
Key Results and Discussion
DeGIA demonstrated a significant performance boost in unseen domains:
- Digit Recognition: Improved accuracy on MNIST-M by 6.47% compared to a standard CNN.
- Hand Alignment Precision: In RGB $\rightarrow$ Depth tests using a ResNet backbone, the MSE dropped from 0.2596 to 0.1065, indicating a massive improvement in precision.
- Cross-Modal Robustness: The model maintained high accuracy on X-ray images despite their significant visual difference from the RGB training set.
The core insight is that mapping features to a probabilistic latent space allows the model to prioritize object structure over domain-specific pixel values. Adversarial training further simulates the variability of unseen domains, enhancing robustness.
Conclusion
DeGIA provides a flexible, backbone-agnostic framework for image alignment that generalizes to unseen domains without requiring target-domain training data. While the training phase is more computationally intensive due to the maximization phase, the resulting models offer high precision and fast inference. Future work may extend this approach to other medical tasks such as segmentation and lesion detection.
Summary updated