TL;DR
This paper introduces SNN-SC, a novel framework for Semantic Communication (SC) using Spiking Neural Networks (SNNs) to transmit features in Collaborative Intelligence (CI) systems over digital channels. SNN-SC efficiently compresses features into binary spike signals, eliminating the need for quantization and offering improved robustness and lower computational complexity compared to existing DNN-based SC methods, especially in noisy environments.
What Problem Does This Paper Address?
Collaborative Intelligence (CI) aims to deploy AI models on resource-constrained edge devices by splitting them between the edge and the cloud. A key challenge is transmitting intermediate features from the edge to the cloud efficiently and reliably, especially over noisy wireless channels. Traditional methods often involve transmitting raw features, leading to high bandwidth consumption. While Semantic Communication (SC) offers a solution by transmitting only task-relevant information, existing DNN-based SC models typically output floating-point values, requiring extra quantization steps to be compatible with digital channels. This quantization can be computationally expensive and sensitive to errors, particularly in low-quality channels. The paper addresses the need for a robust, efficient, and directly digital-compatible SC framework for CI.
Methods
The core of the proposed solution is SNN-SC, a framework that leverages Spiking Neural Networks (SNNs) for semantic communication.
- SNN-based Encoder/Decoder: Instead of DNNs, SNNs are used for both encoding semantic information on the edge and decoding it on the cloud. SNNs naturally output binary spike signals, making them directly compatible with digital channels without the need for explicit quantization. This also leads to lower computational complexity as spiking neurons primarily use accumulation (AC) operations instead of multiply-accumulate (MAC) operations.
- Integrate and Hybrid Fire (IHF) Neuron: A novel spiking neuron model, IHF, is introduced. It simultaneously outputs spike information (like traditional Integrate-and-Fire neurons) and membrane potential information. This dual output enriches the information available for reconstruction, enhancing the decoder's capability.
- Maximizing Semantic Information Entropy: To further improve robustness and efficiency, the framework maximizes the entropy of the semantic information transmitted. This ensures that the binary spike signals are evenly distributed, maximizing the mutual information between the transmitted and received semantic information and thus improving performance in poor channel conditions.
- Task-Oriented Training: The SNN-SC framework is trained in a task-oriented manner, optimizing for downstream task performance (e.g., image classification accuracy) rather than perfect feature reconstruction. A three-step training strategy is employed, starting with training the downstream AI model, then training SNN-SC with fixed AI model parameters, and finally fine-tuning the entire system.
Datasets
The paper uses standard datasets for evaluating the downstream AI tasks:
- CIFAR-100: Used for training the ResNet50 image classification model.
- FoodSeg103: Used for training the CCNet semantic segmentation model.
Key Insights and Ideas
- SNNs as a Natural Fit for Digital SC: The inherent binary output of SNNs makes them a perfect match for digital communication channels, bypassing the quantization bottleneck of DNN-based SC.
- IHF Neuron: More Than Just Spikes: The IHF neuron's ability to output both spike and membrane potential information is crucial. It demonstrates that leveraging both aspects of a spiking neuron's state can lead to richer semantic representations and better reconstruction.
- Entropy Maximization for Robustness: Maximizing the entropy of semantic information is a clever way to ensure that the transmitted bits are as informative as possible, leading to better performance under noisy conditions. This is a departure from simply minimizing reconstruction error.
- Overcoming the "Cliff Effect": SNN-SC shows a more graceful degradation in performance compared to traditional separate source and channel coding methods (like JPEG+Conv), avoiding the sharp performance drop known as the "cliff effect" in poor channel conditions.
Key Results
- Improved Robustness: SNN-SC consistently outperforms baseline DNN-based SC methods and traditional JPEG+Conv methods, especially under poor channel conditions (high bit error rates or erasure rates). For instance, on ResNet50, SNN-SC outperforms baselines when the bit error rate exceeds 0.15.
- Higher Compression Ratios: SNN-SC achieves significantly higher compression ratios (e.g., 256x and 512x for ResNet50 features) compared to JPEG+Conv, drastically reducing bandwidth requirements.
- Lower Computational Complexity: Despite its iterative nature, SNN-SC exhibits lower computational complexity than DNN-based SC baselines, making it more suitable for resource-constrained edge devices. For CCNet features, SNN-SC reduces computational complexity by about 30% under low bandwidth conditions.
- Effective IHF Neuron: Ablation studies show that the IHF neuron significantly improves performance over standard IF neurons, highlighting the benefit of its dual output.
- Impact of Entropy Maximization: Experiments with varying entropy loss coefficients (α) demonstrate that maximizing entropy leads to improved performance and robustness, with the optimal performance achieved at α=1.0.
Limitations and Future Work
- Limitations: The paper focuses on image classification and semantic segmentation tasks. While SNN-SC is presented as a general framework, its performance on other tasks is not explored. The current implementation relies on a specific three-step training strategy, which might need adaptation for different downstream models.
- Future Work: The authors suggest that SNN-SC is a plug-and-play feature transmission model that can be applied to other models under CI without significant alterations. Future work could explore its application to a wider range of CI tasks and different types of AI models. Further investigation into optimizing the IHF neuron and the entropy maximization loss for various scenarios could also be beneficial.
Summary updated