Unequal Error Protection Aided Region of Interest Aware Wireless Panoramic Video

Yongkai Huo, Xu Wang, Peichang Zhang, Jianmin Jiang, Lajos HanzoIEEE Access 2019 · DOI

Introduction

The transmission of high-quality 360° (panoramic) video over wireless networks presents significant engineering hurdles. Unlike traditional video, 360° content covers an entire sphere, necessitating immense bandwidth. Furthermore, the inherent volatility of wireless channels often leads to packet loss, resulting in severe visual artifacts that degrade the immersive experience. To address these issues, researchers from Shenzhen University and the University of Southampton have proposed a Region of Interest Aware Unequal Error Protection (ROI-UEP) scheme, published in IEEE Access.

The Challenge of Panoramic Video Transmission

A primary inefficiency in current 360° video streaming is that standard protocols treat the entire spherical frame as equally important. However, a viewer using a Head-Mounted Display (HMD) typically only perceives a "viewport" of approximately 120°. Consequently, resources spent protecting pixels outside this viewport—such as those directly behind the viewer—are largely wasted if the channel is constrained.

Spherical representation of a 360° video frame from the RaceVR dataset.

Additionally, 360° video is commonly stored using Equirectangular Projection (ERP). This mapping stretches pixels near the poles, meaning that a pixel's importance in the flat representation does not linearly correspond to its visual significance on the sphere. Traditional quality metrics like Peak Signal-to-Noise Ratio (PSNR) fail to account for this geometric distortion.

Equirectangular projection (ERP) of the 360° frame, showing the stretching effect at the poles.

Proposed Methodology: ROI-UEP

The proposed ROI-UEP system optimizes transmission by prioritizing the viewer's gaze and accounting for spherical geometry through several key components:

  • ROI Estimation: The system tracks the viewing trajectory from the HMD to identify the 120° field of view in real-time.
  • Weighted PSNR (WPSNR): A new metric was developed to assign weights to pixels based on their actual area on the sphere and their proximity to the viewer's ROI.
  • Unequal Error Protection (UEP): The video frame is partitioned into blocks, each assigned an "importance" value. Using a Lagrange Multiplier and Newton’s down-hill method, the system solves an optimization problem to determine the ideal Forward Error Correction (FEC) coding rate for each block.

By applying "heavy-duty" protection to blocks within the ROI and lighter protection to the periphery, the system ensures that the most critical visual information remains intact even in noisy environments.

Experimental Results

The ROI-UEP scheme was evaluated using two 4K sequences: RaceVR (high-motion) and RollerCoaster (complex textures), both encoded with HEVC (H.265).

The results demonstrate significant performance gains over traditional Equal Error Protection (EEP) methods:

  • Quality Improvement: At a channel Signal-to-Noise Ratio (SNR) of 6 dB, the ROI-UEP scheme achieved a 9.4 dB improvement in WPSNR compared to a standard ROI-based EEP approach.
  • Efficiency: Compared to baseline EEP (which protects the entire frame uniformly), the improvement exceeded 10 dB.
  • Visual Fidelity: Qualitative analysis confirmed that the viewer's ROI remained clear under high-noise conditions, while errors were effectively relegated to the non-visible portions of the sphere.

Conclusion and Future Directions

The ROI-UEP approach demonstrates that perceptual awareness can significantly enhance the efficiency of wireless 360° video transmission. By intentionally allowing errors in the periphery, the system provides a "perceptual hardening" of the viewer's actual field of view. Future research will focus on ROI prediction to mitigate latency between head movement and transmission adjustment, as well as the integration of Reinforcement Learning for real-time parameter optimization in dynamic network environments.

Summary updated

BibTeX
@article{huo2019unequal,
  author = {Yongkai Huo and Xu Wang and Peichang Zhang and Jianmin Jiang and Lajos Hanzo},
  title = {{Unequal Error Protection Aided Region of Interest Aware Wireless Panoramic Video}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2921880},
}
Figure