Introduction
In medical imaging, training robust deep learning models is often hindered by the lack of large-scale, accurately labeled real-world datasets. While physics-based simulations can generate infinite labeled data, a significant "sim-to-real" domain gap persists, where simulated images fail to capture the nuances of real clinical data. Traditional domain adaptation methods like CycleGANs often suffer from training instability, hallucinations, or mode collapse.
This paper introduces a novel unsupervised domain transfer method using Conditional Invertible Neural Networks (cINNs). By leveraging the mathematical property of invertibility, the proposed framework ensures cycle consistency by design and produces synthetic training data that is more realistic and effective for downstream diagnostic tasks.
Methodology: Conditional Invertible Neural Networks
The core of the approach is the cINN architecture, which maps both simulated and real data into a shared latent space. Unlike standard feed-forward networks, cINNs are designed to be fully reversible.
Key Architectural Features
- Inherent Cycle Consistency: The network's invertibility guarantees that mapping from the simulated domain to the real domain and back is perfectly consistent, eliminating the need for complex cycle-consistency loss functions.
- Shared Latent Space: Domain-invariant features (e.g., anatomy) are captured in a shared latent space, while domain-specific characteristics (e.g., sensor noise) are controlled via a domain label ($D$).
- Conditioning: The network is conditioned on both the domain ($D$) and a tissue label ($Y$), ensuring that the biological identity of the data is preserved during the transfer process.
- Hybrid Training: The model is trained using a combination of Maximum Likelihood (for stable distribution matching) and Adversarial Loss (to enhance visual realism).
Experimental Validation
The method was evaluated across two distinct spectral imaging modalities:
- Photoacoustic Tomography (PAT): Focused on image-level transfer using 960 real images from human volunteers and ~1,600 simulated images.
- Hyperspectral Imaging (HSI): Focused on pixel-level spectral transfer using ~6.4 million real spectra from porcine organs and ~210,000 simulated spectra.
Results and Insights
The cINN approach demonstrated significant improvements over baseline simulations and existing GAN-based methods like UNIT.
- Distribution Alignment: Principal Component Analysis (PCA) confirmed that cINN-transferred data distributions aligned much more closely with real-world data than raw simulations.
- Downstream Task Performance:
- In PAT artery/vein classification, cINN-transferred data yielded a 37% relative improvement in Balanced Accuracy over raw simulations.
- In HSI organ classification, the method achieved a 21% relative improvement in Balanced Accuracy and a 33% improvement in F1-score.
The primary innovation lies in using invertibility as a hard constraint. This prevents the network from "hallucinating" features, as every transformation must be reversible. Furthermore, maximum likelihood training provides a more stable alternative to GANs, effectively avoiding mode collapse.
Limitations and Future Outlook
While effective, cINNs are computationally intensive due to their volume-preserving nature, which prevents dimensionality reduction within the network. This makes scaling to high-dimensional 3D volumes challenging. Future research will focus on optimizing these architectures and validating them against physical "phantoms" where ground truth is known with absolute certainty.
Summary updated