SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding

Ke Yang, Sixian Wang, Jincheng Dai, Xiaoqi Qin, Kai Niu, Ping Zhang

TL;DR

SwinJSCC introduces a powerful Swin Transformer-based backbone for Deep Joint Source-Channel Coding (JSCC), significantly outperforming traditional CNN-based models in image transmission. By incorporating two novel plug-in modules—Channel ModNet and Rate ModNet—the authors create a "universal" model that adaptively handles varying signal-to-noise ratios (SNR) and transmission rates with higher accuracy and much faster processing speeds than traditional digital systems.

What Problem Does This Paper Address?

Traditional wireless communication relies on the Shannon separation principle, which treats source coding (compression) and channel coding (error correction) as two independent steps. While theoretically optimal in infinite-length scenarios, this approach struggles with the "cliff effect" (sudden total failure when channel conditions drop) and high latency in real-world, delay-sensitive applications.

Recent "Deep JSCC" methods use neural networks to merge these steps, offering "graceful degradation" as signal quality drops. However, most existing Deep JSCC models use Convolutional Neural Networks (CNNs). CNNs have a limited "effective receptive field," meaning they struggle to capture the global context and hierarchical details of high-resolution images. Furthermore, most models are rigid—they are trained for one specific channel condition or transmission rate, requiring a library of different models for practical use.

Methods

The authors propose SwinJSCC, which replaces the standard CNN backbone with a Swin Transformer.

  1. Swin Transformer Backbone: Unlike standard Transformers that have quadratic complexity, the Swin Transformer uses a hierarchical structure and shifted-window self-attention. This allows the model to capture long-range dependencies in an image with linear computational complexity, making it ideal for high-resolution data.
  2. Channel ModNet: This is a "channel-aware" plug-in. It takes the current Channel State Information (CSI/SNR) as input and modulates the latent representations. This allows the model to implicitly adjust its coding strategy based on how noisy the "air" is.
  3. Rate ModNet: To handle different transmission bandwidths (Channel Bandwidth Ratio or CBR), this module generates a binary mask that selects the most important features to transmit. This enables a single model to support multiple transmission rates.
  4. End-to-End Optimization: The entire pipeline (Encoder $\rightarrow$ Channel $\rightarrow$ Decoder) is trained together to minimize distortion (MSE or MS-SSIM).

Datasets

The researchers used several standard benchmarks to train and evaluate the model:

  • DIV2K: A high-quality dataset of 800 images used for training.
  • Kodak: A classic set of 24 images used for evaluating general image compression performance.
  • CLIC2021: A dataset containing 2K resolution images to test the model's performance on high-definition content.
  • CIFAR10: Used specifically to validate the SNR-adaptive version on low-resolution images.

Key Insights and Ideas

  • Capacity is King: The paper demonstrates that the "bottleneck" in current neural communications isn't just the channel, but the representation power of the backbone. Moving from CNNs to Transformers provides the "model capacity" needed to beat engineered systems like BPG + 5G LDPC.
  • The Receptive Field Advantage: Transformers "see" the whole image much better than CNNs (larger Effective Receptive Field). This global context acts as a powerful prior that helps the decoder "hallucinate" missing details lost to channel noise.
  • Taming the Complexity: While Transformers are often seen as "heavy," the SwinJSCC architecture actually achieves lower end-to-end latency than traditional systems because it avoids the iterative, computationally expensive decoding process of traditional error-correcting codes (like LDPC).
  • Universal Adaptability: By using modulation modules (ModNets), the authors prove you don't need 100 different models for 100 different SNRs. A single model can learn to be a "jack of all trades" without losing its "mastery" of any specific condition.

Key Results

  • Superior Performance: SwinJSCC achieves better PSNR (Peak Signal-to-Noise Ratio) and MS-SSIM than the state-of-the-art BPG + 5G LDPC digital system, especially in low SNR regimes where traditional systems fail.
  • Efficiency: On the Kodak dataset, SwinJSCC achieved a BD-rate saving of approximately 25-29% compared to traditional benchmarks.
  • Speed: The end-to-end latency is significantly lower. For example, decoding a Kodak image takes about 16ms with SwinJSCC, compared to over 7.3 seconds for the BPG+LDPC baseline.
  • Visual Quality: Qualitatively, SwinJSCC avoids the "blocking artifacts" common in traditional compression, producing much cleaner reconstructions at low bandwidths.

Limitations and Future Work

  • Side Information: The Rate ModNet requires sending a "mask" as side information. While this overhead is negligible for high-resolution images, it becomes a significant burden for very small images (like 32x32 icons).
  • Feature Loss: The authors noted a marginal performance dip when using both SNR and Rate adaptation simultaneously compared to models optimized for a single condition, suggesting that the "resizing" of features in the ModNet might cause minor information loss.
  • Future Directions: The authors suggest extending this architecture to video transmission and exploring more complex, non-linear channel models beyond the standard AWGN and Rayleigh fading used in this study.

Summary updated

Abstract

As one of the key techniques to realize semantic communications, end-to-end optimized neural joint source-channel coding (JSCC) has made great progress over the past few years. A general trend in many recent works pushing the model adaptability or the application diversity of neural JSCC is based on the convolutional neural network (CNN) backbone, whose model capacity is yet limited, inherently leading to inferior system coding gain against traditional coded transmission systems. In this paper, we establish a new neural JSCC backbone that can also adapt flexibly to diverse channel conditions and transmission rates within a single model, our open-source project aims to promote the research in this field. Specifically, we show that with elaborate design, neural JSCC codec built on the emerging Swin Transformer backbone achieves superior performance than conventional neural JSCC codecs built upon CNN, while also requiring lower end-to-end processing latency. Paired with two spatial modulation modules that scale latent representations based on the channel state information and target transmission rate, our baseline SwinJSCC can further upgrade to a versatile version, which increases its capability to adapt to diverse channel conditions and rate configurations. Extensive experimental results show that our SwinJSCC achieves better or comparable performance versus the state-of-the-art engineered BPG + 5G LDPC coded transmission system with much faster end-to-end coding speed, especially for high-resolution images, in which case traditional CNN-based JSCC yet falls behind due to its limited model capacity.
BibTeX
@article{yang2025swinjscc,
  author = {Ke Yang and Sixian Wang and Jincheng Dai and Xiaoqi Qin and Kai Niu and Ping Zhang},
  title = {{SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding}},
  journal = {IEEE Trans. Cogn. Commun. Netw.},
  year = {2025},
  doi = {10.1109/tccn.2024.3424842},
}

Loading related content...

Figure