Abstract
Image-level domain alignment is the de facto approach for unsupervised domain adaptation, where unpaired image translation is used to minimize the domain gap. Prior studies mainly focus on the domain shift between the source and target domains, whereas the intra-domain variability remains under-explored. To address the latter, an effective strategy is to diversify the styles of the synthetic target domain data during image translation. However, previous methods typically require intra-domain variations to be pre-specified for style synthesis, which may be impractical. In this paper, we propose an exemplar-based style synthesis method named IntraStyler, which can capture diverse intra-domain styles without any prior knowledge. Specifically, IntraStyler uses an exemplar image to guide the style synthesis such that the output style matches the exemplar style. To extract the style-only features, we introduce a style encoder to learn styles discriminatively based on contrastive learning. We evaluate the proposed method on the largest public dataset for cross-modality domain adaptation, CrossMoDA 2023. Our experiments show the efficacy of our method in controllable style synthesis and the benefits of diverse synthetic data for downstream segmentation.
What Problem Does This Paper Address?
In medical AI, a model trained on data from "Hospital A" often fails when deployed at "Hospital B" due to domain shift. While researchers have successfully developed methods for translating images between modalities (e.g., making a ceT1 MRI look like a T2 MRI), they often overlook intra-domain variability.
Even within the same modality (like T2 MRI), images can look vastly different depending on the scanner manufacturer, imaging protocol, or site-specific settings. Current Image-level Domain Alignment (ILDA) methods try to solve this by creating synthetic training data that mimics the target domain. However, these methods usually require researchers to manually group target data into sub-domains (e.g., "Scanner 1," "Scanner 2") beforehand. In real-world clinical settings, this metadata is often missing or insufficient to capture the full spectrum of visual styles.
The IntraStyler Framework
The authors introduce IntraStyler, which moves away from pre-defined categories and instead uses an exemplar-based approach. To make a translated image look like a specific target scan, the system simply uses that scan as an "exemplar" to guide the synthesis.
Key Technical Components
- Contrastive Style Encoder ($E_S$): To separate "style" (brightness, contrast, noise) from "anatomy" (the actual organs), the authors use a contrastive learning task. They take two different patches from the same image (same style, different anatomy) as a "positive pair" and compare them against patches that have been artificially distorted (different style, same anatomy) as "negatives." This forces the encoder to ignore anatomical structures and focus solely on the imaging style.
- Dynamic Instance Normalization (DIN): The style features extracted by the encoder are injected into the image generator ($G$) using DIN layers. These layers adjust the mean and variance of the neural network's internal features to match the exemplar's style.
- Style Consistency Loss: A feedback loop ensures that if the newly generated image is passed back through the style encoder, the resulting "style code" matches the original exemplar.
- Style Interpolation (SLERP): Because the style codes are normalized onto a sphere, the authors use Spherical Linear Interpolation (SLERP) to smoothly transition between two different styles. This allows for the generation of "in-between" styles that might not even exist in the original dataset, further diversifying the training data.
Experimental Evaluation
The method was evaluated on the CrossMoDA 2023 dataset, the largest public benchmark for cross-modality domain adaptation. The task involved segmenting the cochlea and vestibular schwannoma (VS) using 226 labeled ceT1 MRIs (source) and 295 unlabeled T2 MRIs (target) from multiple institutions with heterogeneous acquisition parameters.
Key Results
IntraStyler was compared against several baselines, including "MultiNets" (separate networks for known sub-domains) and "Unified" (a single network using sub-domain labels).
- Segmentation Accuracy: IntraStyler achieved more robust results, particularly in ASSD (Average Symmetric Surface Distance), which measures the proximity of predicted boundaries to the ground truth.
- Failure Reduction: In the challenging "Extra-meatal VS" segmentation task, IntraStyler significantly reduced "failure cases" where the model misses the structure entirely, compared to methods that did not account for style diversity.
- Qualitative Success: Visualization demonstrated that IntraStyler could successfully take one patient's anatomy and "re-skin" it to match the specific contrast and noise levels of various different scanners from the target pool.
Conclusion and Future Work
IntraStyler demonstrates that diversity breeds robustness. By treating every image in the target pool as a potential style guide rather than forcing them into rigid categories, the model captures a much finer grain of diversity. This leads to downstream segmentation models (like nnU-Net) that are harder to confuse during testing.
While successful, the authors note that the style encoder's effectiveness depends on the choice of intensity perturbations, which may need tuning for different modalities (e.g., Ultrasound). Future work may explore "local" style control—such as the specific texture of a tumor—beyond the "global" brightness and contrast currently captured.
Summary updated