Guest Editorial Signal Processing for XR Communications and Systems

Yongpeng Wu, Erik G. Larsson, Jing Li, Angel Lozano, Luce Morin, Mai Xu, Chengshan Xiao, Wei YangIEEE JSTSP 2023 · DOI · OpenAlex

Abstract

Future wireless networks are expected to support ubiquitous extended reality (XR) with human-to-human communications. XR encompasses augmented reality (AR), mixed reality (MR), and virtual reality (VR), providing immersive experiences through ultra-high resolution, panoramic scenery, and multi-sensory inputs. However, transmitting XR video with ultra-high bit rates and low latency presents critical challenges. This editorial introduces a special issue of the IEEE Journal of Selected Topics in Signal Processing featuring 15 research papers that address these bottlenecks through advanced signal processing, MIMO systems, AI-assisted communications, and optimized resource allocation.

Introduction

Extended Reality (XR) represents the next frontier in human-machine interaction, merging real and virtual environments. To achieve true immersion, wireless networks must overcome the "holy trinity" of XR challenges: ultra-high bandwidth requirements, sub-millisecond latency, and a consistent Quality of Experience (QoE). Current wireless standards often fall short of the high bit rates needed for 8K+ video while maintaining the low latency required to prevent "motion-to-photon" delay, which causes simulator sickness. Furthermore, mobile XR demands precise 6-degrees-of-freedom (6-DoF) tracking and robust security for shared virtual spaces.

Research Pillars in XR Signal Processing

The editorial categorizes the contributions into four primary research pillars:

1. XR Streaming and Tracking

Effective XR requires both high-fidelity video reconstruction and precise user tracking. Innovations in this area include the use of Extended Kalman filters for 5G carrier phase-based tracking [A1], ensuring continuous 6-DoF alignment. Additionally, spatiotemporal scene graphs are employed for semantic video reconstruction, allowing the system to understand and prioritize the most relevant visual elements for the user.

2. MIMO Systems and Reconfigurable Surfaces

To boost spectrum efficiency and coverage, researchers are leveraging Cell-Free Massive MIMO and Reconfigurable Intelligent Surfaces (RIS). RIS-assisted designs [A5] effectively balance the trade-off between positioning accuracy and communication throughput. Secure hybrid beamforming techniques [A6] are also introduced to protect sensitive mmWave transmissions from eavesdropping in public or shared environments.

3. AI-Assisted and Semantic Communications

A paradigm shift is occurring from traditional bit-level transmission to Semantic Communication. By transmitting the "meaning" or essential features of data rather than raw pixels, systems can achieve massive bandwidth savings. One proposed semantic coding framework [A8] demonstrated significant bandwidth savings and lower latency compared to the standard VTM + 5G LDPC transmission scheme. Other methods use generative AI to synthesize complex environments, such as vehicular MR, with minimal data overhead.

4. Resource Allocation and Information Freshness

Optimizing network resources requires new metrics beyond simple throughput. The Age of Incorrect Information (AoII) is proposed to measure the freshness and accuracy of data in NOMA-aided XR systems. Joint power allocation and caching strategies, particularly user dynamics-aware edge caching [A16], are essential for reducing frame loss in mobile VR scenarios compared to conventional schemes.

Key Insights and Benchmarking

A major shift is occurring from traditional data transmission to "Semantic Communication." By reconstructing video from scene graphs or using variable-length semantic coding, systems can achieve massive bandwidth savings without losing the perceived quality of the XR experience. Furthermore, the transition toward Integrated Sensing and Communication (ISAC) allows the network to "sense" the user's location, which is a key enabler for mobile XR.

To support these advancements, one significant contribution [A3] introduces a large-scale Quality-of-Experience (QoE) database for adaptive omnidirectional video streaming. This benchmark is unique because it integrates both adaptive bit rate streaming factors and subjective human evaluation, providing a much-needed baseline for testing how users perceive 360-degree video quality under varying network conditions.

Conclusion and Future Directions

While the research in this special issue makes significant strides, achieving ubiquitous, wide-area XR coverage remains a challenge. Future work must focus on reducing the computational complexity of advanced AI-driven models to suit mobile hardware. Furthermore, integrating semantic communication metrics and "Age of Information" standards into future 6G frameworks will be critical for industry-wide adoption. As signal processing continues to evolve, the gap between the physical and virtual worlds will continue to narrow, enabling the next generation of immersive human-to-human communication.

This article is a summary of the Guest Editorial for the IEEE Journal of Selected Topics in Signal Processing Special Issue on Signal Processing for XR Communications and Systems.

Summary updated

Abstract

Future wireless networks are expected to support ubiquitous extended reality (XR) with human-to-human communications. XR is a term that refers to all real-and-virtual combined environments and human-machine interactions generated by computer technology and wearables, where the X represents any current or future spatial computing technology. XR includes augmented reality (AR), mixed reality (MR), and virtual reality (VR) that all are immersive at different levels and entail distinct degrees of sensory inputs. The ultra-high resolution, detailed representation, panoramic scenery, and multi-stimuli of XR provide a unique immersive experience, allowing users to interact within an alternative world. Transmitting XR video, with its ultra-high bit rate and low latency, presents critical challenges to wireless networking.
BibTeX
@article{wu2023guest,
  author = {Yongpeng Wu and Erik G. Larsson and Jing Li and Angel Lozano and Luce Morin and Mai Xu and Chengshan Xiao and Wei Yang},
  title = {{Guest Editorial Signal Processing for XR Communications and Systems}},
  journal = {IEEE Journal of Selected Topics in Signal Processing},
  year = {2023},
  doi = {10.1109/jstsp.2023.3326801},
}

Loading related content...

Figure