Introduction
Unsupervised image-to-image (I2I) translation aims to map images between two domains (e.g., day to night, sunny to cloudy) without paired training data. While Generative Adversarial Networks (GANs) have become the standard for this task, they face a persistent challenge: the "style vs. structure" trade-off. Models often struggle to apply significant stylistic changes (color, lighting, texture) while faithfully preserving the underlying structural integrity (roads, buildings, fine details). Processing images primarily in the spatial domain often leads to structural distortions or "hallucinated" artifacts when style shifts are drastic.
Methodology
To address these limitations, the proposed SEIT (Structural Enhancement in Frequency Domain) framework shifts the translation process from the spatial domain to the frequency domain. By decoupling content and style using Fourier and Wavelet transforms, the model achieves superior style transfer without compromising structural fidelity.
Frequency Dynamic Adaptive (FDA) Module
The FDA module leverages the Discrete Fourier Transform (DFT). In the Fourier domain, an image is decomposed into its Amplitude and Phase components. The Amplitude typically carries style and texture information, while the Phase encapsulates structural and content information. The FDA module extracts the target domain's style from its amplitude and applies it to the source image while keeping the source's phase information intact, ensuring the "what" of the image remains unchanged while the "how it looks" is transformed.
Wavelet-based Structure Enhancement (WSE) Module
To prevent the blurring of sharp edges often associated with global Fourier transforms, the Wavelet-based Structure Enhancement (WSE) module is employed. Using the Discrete Wavelet Transform (DWT), the image is decomposed into low-frequency and high-frequency sub-bands. The WSE module specifically reinjects high-frequency details from the original source image back into the translated output, preserving fine structural elements like power lines and distant architectural features.
Experimental Results
SEIT was evaluated on three benchmark datasets: SYNTHIA (Day → Night), BDD100k (Sunny → Cloudy), and GTA5 → Cityscapes (Synthetic-to-Real). The model consistently outperformed state-of-the-art baselines such as CUT, MUNIT, and VSAIT.
Quantitative Performance
On the SYNTHIA Day → Night task, SEIT achieved a Structural Similarity (SSIM) of 0.87, a significant improvement over the previous state-of-the-art (LPTN at 0.83). Furthermore, SEIT reached the lowest Fréchet Inception Distance (FID) across tasks, such as 82.55 for GTA5 → Cityscapes, indicating that its generated images are statistically the most similar to real target-domain data.
Conclusion
SEIT demonstrates that frequency-domain decoupling is a highly effective strategy for unsupervised I2I translation. By treating the Fourier Phase as a structural anchor and using Wavelets for edge recovery, the model allows for aggressive style changes while maintaining rigid structural consistency. Future work will explore extending this framework to multimodal translation tasks and improving performance on non-rigid transformations.
Summary updated