Abstract
Information asymmetric image-to-image translation, such as converting sparse radar (SAR) data to optical imagery or sketches to rendered characters, presents a significant challenge for traditional architectures. CNNs often suffer from local bias and region blending, while Transformers may fail to capture fine-grained textural details. We present CSHNet, a hybrid CNN-Swin Transformer network designed to bridge this information gap. By employing a novel SEC-CES-Bottleneck (SCB), Interactive Guided Connections (IGC), and an Adaptive Edge Perception Loss (AEPL), CSHNet achieves superior structural consistency and detail preservation. Our model demonstrates state-of-the-art performance on the SEN12 and Sketch2Anime benchmarks while maintaining high parameter efficiency.
1. Introduction
Image-to-image translation has seen rapid advancement, yet "information asymmetric" tasks remain a hurdle. These tasks involve mapping from a "sparse" domain (limited information) to a "rich" domain (detailed content). Examples include SAR-to-Optical (S2O) translation and Sketch-to-Instance generation.
Current architectures face a "Goldilocks" problem: CNNs excel at local details but lack global context, leading to "region blending" where distinct objects bleed into one another. Conversely, pure Transformers capture global structure but often miss intricate details, such as eye reflections or sharp textural edges. CSHNet is proposed to resolve this tension by synergistically combining both paradigms.
2. Methodology
The core of CSHNet lies in its ability to balance local precision with global reasoning through three key innovations.
2.1 SEC-CES-Bottleneck (SCB)
Rather than simple sequential stacking, the SCB utilizes a cross-combination of modules. The Swin Embedded CNN (SEC) uses a CNN backbone for detail extraction while leveraging a Swin Transformer for structural context. The CNN Embedded Swin (CES) uses the Transformer as a foundation and embeds CNN modules to recover textural details that might otherwise be lost.
2.2 Interactive Guided Connection (IGC)
To prevent the loss of low-level details during deep feature abstraction, the IGC extracts early-stage features and applies a learnable linear transformation to modulate high-level features in the bottleneck. This ensures that fine details from the input "guide" the generation process.
2.3 Adaptive Edge Perception Loss (AEPL)
Standard edge-detection losses often fail due to fixed thresholds. AEPL utilizes information entropy to dynamically calculate optimal thresholds for each image, forcing the network to maintain sharp, accurate boundaries between regions, which is particularly crucial for remote sensing data.
3. Experiments and Results
CSHNet was evaluated on two primary benchmarks: SEN12 (satellite imagery) and Sketch2Anime (character generation).
3.1 Quantitative Performance
On the SEN12 dataset, CSHNet achieved a PSNR of 16.37 and an SSIM of 0.2895, significantly outperforming the CNN-based Pix2pix (15.84 / 0.2568). It also recorded the lowest RMSE (45.1), indicating high pixel-level accuracy.
3.2 Qualitative Analysis
Visual results demonstrate that CSHNet successfully avoids the blending artifacts common in CNNs (e.g., clearer river boundaries) and captures the intricate details missed by pure Transformers (e.g., accurate eye highlights in sketches).
3.3 Efficiency
Despite its hybrid nature, CSHNet is highly efficient. With only 30.6M parameters, it outperforms significantly larger Transformer-based models, making it suitable for practical deployment.
4. Conclusion
CSHNet effectively addresses the information asymmetry in image translation by integrating the strengths of CNNs and Swin Transformers. Its novel bottleneck structure and adaptive loss mechanisms provide a robust framework for generating high-quality, detailed images from sparse inputs. Future research will focus on multi-modal integration and leveraging large-scale pre-trained foundation models to further enhance generalizability.
Summary updated