SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding

Ke Yang, Sixian Wang, Jincheng Dai, Xiaoqi Qin, Kai Niu, Ping Zhang

Abstract

As one of the key techniques to realize semantic communications, end-to-end optimized neural joint source-channel coding (JSCC) has made great progress over the past few years. A general trend in many recent works pushing the model adaptability or the application diversity of neural JSCC is based on the convolutional neural network (CNN) backbone, whose model capacity is yet limited, inherently leading to inferior system coding gain against traditional coded transmission systems. In this paper, we establish a new neural JSCC backbone that can also adapt flexibly to diverse channel conditions and transmission rates within a single model. Specifically, we show that with elaborate design, neural JSCC codec built on the emerging Swin Transformer backbone achieves superior performance than conventional neural JSCC codecs built upon CNN, while also requiring lower end-to-end processing latency. Paired with two spatial modulation modules that scale latent representations based on the channel state information and target transmission rate, our baseline SwinJSCC can further upgrade to a versatile version, which increases its capability to adapt to diverse channel conditions and rate configurations. Extensive experimental results show that our SwinJSCC achieves better or comparable performance versus the state-of-the-art engineered BPG + 5G LDPC coded transmission system with much faster end-to-end coding speed, especially for high-resolution images.

Introduction

Traditional wireless communication systems are built upon the Shannon separation principle, which treats source coding (compression) and channel coding (error correction) as independent modules. While theoretically sound for infinite-length codes, practical implementations often suffer from the "cliff effect"—a sudden loss of service when channel conditions deteriorate beyond a threshold—and high latency due to complex iterative decoding.

Deep Joint Source-Channel Coding (Deep JSCC) has emerged as a promising alternative, utilizing neural networks to map source data directly to channel symbols. This approach provides "graceful degradation," where reconstruction quality scales smoothly with signal-to-noise ratio (SNR). However, most existing Deep JSCC frameworks rely on Convolutional Neural Networks (CNNs). CNNs are limited by their local receptive fields, which hinders their ability to capture global dependencies in high-resolution images. Furthermore, these models are typically rigid, requiring separate training for every specific channel condition or transmission rate.

The SwinJSCC Architecture

To overcome these limitations, we propose SwinJSCC, a novel framework that leverages the Swin Transformer as its backbone. The architecture is designed for both high performance and flexibility.

The overall architecture of SwinJSCC, featuring the Swin Transformer backbone and modulation modules.

The core components of SwinJSCC include:

  1. Swin Transformer Backbone: By utilizing a hierarchical structure and shifted-window self-attention, the Swin Transformer achieves linear computational complexity relative to image size while maintaining a global receptive field. This allows the model to better preserve hierarchical details and global context compared to CNN-based counterparts.
  2. Channel ModNet: To enable SNR adaptability, we introduce a Channel ModNet. This module modulates latent features based on real-time Channel State Information (CSI), allowing a single model to optimize its transmission strategy for varying noise levels.
  3. Rate ModNet: To support multiple transmission rates (Channel Bandwidth Ratios), the Rate ModNet generates importance-based binary masks. These masks select the most critical features for transmission, enabling a single model to operate across a wide range of bandwidth constraints.

Experimental Evaluation

We evaluated SwinJSCC on several standard datasets, including DIV2K for training and Kodak, CLIC2021, and CIFAR10 for testing. Our results demonstrate that model capacity is a critical factor in neural communication performance.

Performance comparison of SwinJSCC (Proposed) against state-of-the-art JSCC methods in terms of PSNR and computational complexity (GFLOPs).

As shown in the performance analysis, SwinJSCC significantly outperforms existing CNN-based JSCC methods. Key findings include:

  • Superior Coding Gain: SwinJSCC achieves higher PSNR and MS-SSIM than the state-of-the-art BPG + 5G LDPC digital systems, particularly in low SNR regimes. On the Kodak dataset, it delivers a BD-rate saving of approximately 25-29%.
  • Efficiency and Latency: Despite the perceived complexity of Transformers, SwinJSCC offers much lower end-to-end latency than traditional systems. For instance, decoding a Kodak image takes ~16ms, whereas the BPG+LDPC baseline requires over 7 seconds due to iterative decoding.
  • Visual Fidelity: The model effectively avoids the blocking artifacts typical of traditional compression, producing cleaner reconstructions at low bitrates.

Discussion and Conclusion

SwinJSCC represents a significant step forward in semantic communications by demonstrating that Transformer-based backbones can provide the necessary capacity to surpass highly optimized traditional systems. The integration of Channel and Rate ModNets further ensures that this performance is accessible across a wide range of practical operating conditions without the need for multiple specialized models.

While the Rate ModNet introduces a small amount of side information (the feature mask), this overhead is negligible for high-resolution content. Future work will focus on extending this architecture to video transmission and exploring its robustness in more complex, non-linear channel environments.

Summary updated

Abstract

As one of the key techniques to realize semantic communications, end-to-end optimized neural joint source-channel coding (JSCC) has made great progress over the past few years. A general trend in many recent works pushing the model adaptability or the application diversity of neural JSCC is based on the convolutional neural network (CNN) backbone, whose model capacity is yet limited, inherently leading to inferior system coding gain against traditional coded transmission systems. In this paper, we establish a new neural JSCC backbone that can also adapt flexibly to diverse channel conditions and transmission rates within a single model, our open-source project aims to promote the research in this field. Specifically, we show that with elaborate design, neural JSCC codec built on the emerging Swin Transformer backbone achieves superior performance than conventional neural JSCC codecs built upon CNN, while also requiring lower end-to-end processing latency. Paired with two spatial modulation modules that scale latent representations based on the channel state information and target transmission rate, our baseline SwinJSCC can further upgrade to a versatile version, which increases its capability to adapt to diverse channel conditions and rate configurations. Extensive experimental results show that our SwinJSCC achieves better or comparable performance versus the state-of-the-art engineered BPG + 5G LDPC coded transmission system with much faster end-to-end coding speed, especially for high-resolution images, in which case traditional CNN-based JSCC yet falls behind due to its limited model capacity.
BibTeX
@article{yang2025swinjscc,
  author = {Ke Yang and Sixian Wang and Jincheng Dai and Xiaoqi Qin and Kai Niu and Ping Zhang},
  title = {{SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding}},
  journal = {IEEE Trans. Cogn. Commun. Netw.},
  year = {2025},
  doi = {10.1109/tccn.2024.3424842},
}
Figure