CSHNet: A Novel Information Asymmetric Image Translation Method

Xi Yang, Haoyuan Shi, Zihan Wang, Nannan Wang, Xinbo Gao

Abstract

Information asymmetric image-to-image translation, such as converting sparse radar (SAR) data to optical imagery or sketches to rendered characters, presents a significant challenge for traditional architectures. CNNs often suffer from local bias and region blending, while Transformers may fail to capture fine-grained textural details. We present CSHNet, a hybrid CNN-Swin Transformer network designed to bridge this information gap. By employing a novel SEC-CES-Bottleneck (SCB), Interactive Guided Connections (IGC), and an Adaptive Edge Perception Loss (AEPL), CSHNet achieves superior structural consistency and detail preservation. Our model demonstrates state-of-the-art performance on the SEN12 and Sketch2Anime benchmarks while maintaining high parameter efficiency.

1. Introduction

Image-to-image translation has seen rapid advancement, yet "information asymmetric" tasks remain a hurdle. These tasks involve mapping from a "sparse" domain (limited information) to a "rich" domain (detailed content). Examples include SAR-to-Optical (S2O) translation and Sketch-to-Instance generation.

Current architectures face a "Goldilocks" problem: CNNs excel at local details but lack global context, leading to "region blending" where distinct objects bleed into one another. Conversely, pure Transformers capture global structure but often miss intricate details, such as eye reflections or sharp textural edges. CSHNet is proposed to resolve this tension by synergistically combining both paradigms.

2. Methodology

The core of CSHNet lies in its ability to balance local precision with global reasoning through three key innovations.

2.1 SEC-CES-Bottleneck (SCB)

Rather than simple sequential stacking, the SCB utilizes a cross-combination of modules. The Swin Embedded CNN (SEC) uses a CNN backbone for detail extraction while leveraging a Swin Transformer for structural context. The CNN Embedded Swin (CES) uses the Transformer as a foundation and embeds CNN modules to recover textural details that might otherwise be lost.

2.2 Interactive Guided Connection (IGC)

To prevent the loss of low-level details during deep feature abstraction, the IGC extracts early-stage features and applies a learnable linear transformation to modulate high-level features in the bottleneck. This ensures that fine details from the input "guide" the generation process.

2.3 Adaptive Edge Perception Loss (AEPL)

Standard edge-detection losses often fail due to fixed thresholds. AEPL utilizes information entropy to dynamically calculate optimal thresholds for each image, forcing the network to maintain sharp, accurate boundaries between regions, which is particularly crucial for remote sensing data.

Figure 1: (a) Visual comparison showing CSHNet's superior detail and structure preservation against CNN and Transformer baselines. (b) Detailed view of the CSHNet architecture and SCB components.

3. Experiments and Results

CSHNet was evaluated on two primary benchmarks: SEN12 (satellite imagery) and Sketch2Anime (character generation).

3.1 Quantitative Performance

On the SEN12 dataset, CSHNet achieved a PSNR of 16.37 and an SSIM of 0.2895, significantly outperforming the CNN-based Pix2pix (15.84 / 0.2568). It also recorded the lowest RMSE (45.1), indicating high pixel-level accuracy.

3.2 Qualitative Analysis

Visual results demonstrate that CSHNet successfully avoids the blending artifacts common in CNNs (e.g., clearer river boundaries) and captures the intricate details missed by pure Transformers (e.g., accurate eye highlights in sketches).

3.3 Efficiency

Despite its hybrid nature, CSHNet is highly efficient. With only 30.6M parameters, it outperforms significantly larger Transformer-based models, making it suitable for practical deployment.

4. Conclusion

CSHNet effectively addresses the information asymmetry in image translation by integrating the strengths of CNNs and Swin Transformers. Its novel bottleneck structure and adaptive loss mechanisms provide a robust framework for generating high-quality, detailed images from sparse inputs. Future research will focus on multi-modal integration and leveraging large-scale pre-trained foundation models to further enhance generalizability.

Summary updated

Abstract

Despite the considerable advancements in cross-domain image translation, a significant challenge remains in addressing information asymmetric translation tasks such as SAR-to-Optical and Sketch-to-Instance conversions. These tasks involve transforming data from a domain with limited information into one with more detailed and richer content. Traditional CNN-based methods, while effective at capturing intricate details, often struggle to grasp the overall structural composition of the image, leading to unintended blending or merging of distinct regions within the generated images. In light of these limitations, research has increasingly turned toward Transformers. Though Transformers excel at capturing global structures, they often lack the ability to preserve fine-grained details. Recognizing the importance of both detailed features and structural relationships in information asymmetric translation tasks, we introduce the CNN-Swin Hybrid Network (CSHNet). This network employs a novel bottleneck architecture featuring two key modules: Swin Embedded CNN (SEC) and CNN Embedded Swin (CES), which together form the SEC-CES-Bottleneck (SCB). Within this structure, SEC capitalizes on CNN’s capability for detailed feature extraction while incorporating the Swin Transformer’s inherent structural bias. In contrast, CES preserves the Swin Transformer’s strength in maintaining global structural integrity, while compensating for CNN’s tendency to emphasize detail. In addition to the SCB architecture, CSHNet integrates two essential components designed to improve cross-domain information retention and ensure structural consistency. The Interactive Guided Connection (IGC) fosters dynamic information exchange between SEC and CES, encouraging a deeper understanding of image details. At the same time, Adaptive Edge Perception Loss (AEPL) is implemented to preserve well-defined structural boundaries throughout the translation process. Experimental evaluations demonstrate that CSHNet surpasses current state-of-the-art methods, achieving superior results in both visualization and performance metrics across scene-level and instance-level datasets. Our code is available at: https://github.com/XduShi/CSHNet.
BibTeX
@article{yang2026cshnet,
  author = {Xi Yang and Haoyuan Shi and Zihan Wang and Nannan Wang and Xinbo Gao},
  title = {{CSHNet: A Novel Information Asymmetric Image Translation Method}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2026},
  url = {https://openreview.net/forum?id=i1KDvWm1JQ},
}
Figure