Rate Splitting Multiple Access-Enabled Adaptive Panoramic Video Semantic Transmission

Haixiao Gao, Mengying Sun, Xiaodong Xu, Shujun Han, Bizhu Wang, Jingxuan Zhang, Ping ZhangIEEE Trans. Wireless Communications 2025 · DOI

TL;DR

This paper introduces an adaptive panoramic video semantic transmission (APVST) framework that leverages Rate Splitting Multiple Access (RSMA) to significantly improve spectral efficiency and user experience. By combining deep joint source-channel coding with RSMA and a novel adaptive approach, the framework achieves substantial bandwidth savings and enhanced quality of service for immersive video content.

What Problem Does This Paper Address?

The increasing demand for immersive experiences, particularly through 360-degree panoramic videos, presents a significant challenge for current wireless communication systems. Transmitting high-resolution panoramic videos requires immense bandwidth, leading to latency and quality issues. Traditional video transmission methods struggle with the unique characteristics of panoramic content, such as large data volumes and the user's limited field of view (FoV). Semantic communication offers a promising solution by transmitting information at a higher level of meaning rather than raw bits, but integrating it effectively with advanced multiple access techniques like RSMA for panoramic video remains an open problem. This paper aims to bridge this gap by proposing a framework that optimizes both semantic understanding and efficient transmission for panoramic video.

Methods

The core of the proposed solution is the Adaptive Panoramic Video Semantic Transmission (APVST) framework. Key components include:

  • Deep Joint Source-Channel Coding (Deep JSCC): This forms the backbone of both the semantic transmitter and receiver, adaptively extracting and encoding semantic features from panoramic frames.
  • Entropy Model and Dimension-Adaptive Module: These modules are employed to control the transmission rate, enhancing spectral efficiency and conserving bandwidth by intelligently adapting to the information content.
  • Weighted Self-Attention Module: This module integrates weights and feature maps to improve the quality of the immersive experience by focusing on relevant aspects of the video.
  • Rate Splitting Multiple Access (RSMA): RSMA is crucial for splitting the panoramic video semantic streams into common and private messages. This allows for flexible resource allocation, enabling the system to partially decode interference and treat it as noise, thereby improving spectral efficiency.
  • Deep Reinforcement Learning (DRL) with Proximal Policy Optimization (PPO): A PPO-based algorithm is used to optimize the allocation ratios of power, common rate, and channel bandwidth. This DRL approach dynamically adapts to changing environments and user demands to maximize Quality of Service (QoS) scores.
  • Weighted-to-Spherically-Uniform Peak Signal-to-Noise Ratio (WS-PSNR) and WS-SSIM: These metrics are used to evaluate the distortion and quality of panoramic videos, specifically considering the immersive experience.

Datasets

  • 360VDS: Used for training the APVST network. This dataset contains 590 panoramic videos in equirectangular projection (ERP) format.
  • VR scene dataset: Used for evaluating the APVST network. This dataset includes 208 panoramic videos.
  • VR-HM48: Used as the training dataset for the RSMA-enabled transmission system.
  • AVTrack360: Used as the testing dataset for the RSMA-enabled transmission system.

Key Insights and Ideas

  • RSMA's Synergy with Semantic Communication: The paper demonstrates that RSMA is particularly well-suited for semantic video transmission. By splitting streams into common (overlapping FoV) and private (unique FoV) messages, RSMA effectively manages the diverse information needs of users viewing panoramic content.
  • Adaptive Rate Control is Crucial: The integration of an entropy model and a dimension-adaptive module allows for intelligent rate adaptation, which is essential for handling the variable information density in panoramic video and optimizing bandwidth usage.
  • Focus on Immersive Experience Metrics: The use of WS-PSNR and WS-SSIM, which are designed to capture the subjective quality of panoramic video viewing, highlights a shift towards user-centric evaluation in communication systems.
  • DRL for Dynamic Optimization: Employing PPO for optimizing RSMA parameters (power, rates) in a dynamic environment is a powerful approach to achieve optimal QoS by adapting to real-time channel conditions and user FoVs.
  • Bandwidth Savings through Semantic Transmission: The semantic approach, combined with RSMA and adaptive modules, leads to significant reductions in bandwidth requirements compared to traditional methods.

Key Results

  • Bandwidth Savings: The proposed APVST framework achieves bandwidth savings of up to 20% and 50% compared to other semantic and traditional video transmission schemes, respectively.
  • Performance Gains with RSMA: The RSMA-enabled APVST scheme demonstrates performance gains of 13% and 20% compared to NOMA and OFDMA, respectively, in terms of semantic stream transmission for panoramic videos.
  • Improved Immersive Quality: The weighted self-attention module enhances the quality of the immersive experience, as evidenced by improvements in WS-PSNR and WS-SSIM metrics.
  • Efficient DRL Optimization: The PPO-based algorithm quickly converges and maintains stability, effectively optimizing the complex trade-off between latency and immersive experience quality.

Limitations and Future Work

  • Complexity of RSMA Mapping: The paper acknowledges that mapping FoV information to semantic coding maps can be complex, and further research is needed to fully interpret RSMA in this context.
  • Approximation of fAPVST: To accelerate optimization, polynomial regression, nonlinear regression, and random forests were used to approximate the fitting of the fAPVST function. While effective, these approximations might introduce some inaccuracies.
  • Future Work: The authors suggest that future research could explore more advanced DRL algorithms and investigate the impact of different network architectures and channel conditions on the proposed framework. The paper also hints at the potential for further optimization by considering more sophisticated methods for handling the non-convex nature of the optimization problem.

Summary updated

BibTeX
@article{gao2025rate,
  author = {Haixiao Gao and Mengying Sun and Xiaodong Xu and Shujun Han and Bizhu Wang and Jingxuan Zhang and Ping Zhang},
  title = {{Rate Splitting Multiple Access-Enabled Adaptive Panoramic Video Semantic Transmission}},
  journal = {IEEE Trans. Wireless Communications},
  year = {2025},
  doi = {10.1109/twc.2025.3570465},
}
Figure