Macroblock Importance Based Method for Stereo Video Congestion Control

Xiaodong Wang, Ting Yao, Shaobo Li, Lianjun Zhang, Chaolin WangCybernetics 2015 · DOI

Abstract

Transporting a stereo video stream over an unreliable channel can degrade severely the video quality. Network congestion is one of the important factors affecting the video quality. To improve the stereo video quality, a stereo video congestion control method, based on the macroblock importance, is proposed. Firstly, the importance of every macroblock is calculated according to its influence on the current frame and future frames. Then, combined with the importance information, all macroblocks of a frame are divided into three levels. Finally, the packets are transported with the help of RTP/RTCP protocol. We discard some not so important packets actively according to the importance levels when a congestion occurs. The results from NS2 simulation platform show that the stereo video quality is improved with the method proposed.

Introduction

Stereo (3D) video data is significantly larger than traditional video, making it highly susceptible to network congestion over unreliable channels. When congestion occurs, standard network protocols often drop packets randomly. This paper addresses a major gap in existing solutions: most congestion control strategies are either content-agnostic (like standard TCP) or operate at the "frame level" (dropping entire B or P frames).

However, not all parts of a frame are created equal. Some macroblocks (MBs) are critical because they are referenced by many subsequent frames for motion compensation or are used to predict the other viewpoint in a stereo pair. Dropping a "high-importance" macroblock causes error propagation that can ruin the quality of multiple seconds of video. The researchers propose a more granular approach that evaluates the importance of individual macroblocks to maintain higher visual quality even under deteriorating network conditions.

Proposed Methodology

The core of the proposed method is a two-part strategy: Importance Calculation and Active Discarding.

Macroblock Importance Calculation

The importance of an MB is determined by its "Impact Factor," which combines three primary metrics:

  1. Current Frame Impact: The distortion caused if the MB is lost and replaced by a simple error concealment technique (like copying from the previous frame).
  2. Future Frame Impact: A count of how many pixels in subsequent frames reference this specific MB for motion compensation.
  3. Inter-view Dependency: For the left viewpoint, the authors specifically account for how much the right viewpoint relies on it for inter-view prediction.
This diagram illustrates how a macroblock in the current frame (t) impacts future frames (t+1) through motion compensation references.

Classification and Active Congestion Control

MBs are categorized into three levels (Level 1 = Most Important, Level 3 = Least Important) based on a threshold derived from the maximum importance value in the sequence. Using the RTP/RTCP protocol, the sender monitors packet loss feedback ($P_{lost}$) in real-time.

During light congestion, only Level 3 packets are dropped. In cases of serious congestion, Level 2 and Level 3 packets are dropped proportionally. Under critical congestion, packets from all three levels are dropped, but the system still prioritizes the preservation of Level 1 data. This proactive approach ensures that the limited available bandwidth is spent on the most valuable data.

Experimental Results

The authors evaluated their method using several standard video test sequences: Book, Crowd, Akko & Exit, and Xmas. These were simulated using the NS2 (Network Simulator 2) platform to mimic unreliable network environments with packet loss rates ranging from 1% to 10%.

The graphs show the PSNR (dB) performance of the proposed method versus random packet loss across different loss rates for two test sequences.

The proposed method consistently outperformed random packet loss across all test sequences, particularly as congestion worsened:

  • Low Congestion (1% loss): Performance was similar to random loss because the network could still handle most data.
  • High Congestion (3%–10% loss): The proposed method showed a clear advantage. For the Akko sequence at 5% loss, the proposed method achieved a PSNR of 27.17 dB compared to 26.50 dB for random loss—a gain of 0.67 dB.
  • Average Improvement: Across various sequences, the method provided an average PSNR increase of approximately 0.5 dB. In video quality terms, this represents a noticeable reduction in flickering and blocky artifacts caused by error propagation.

Conclusion and Future Work

By moving from frame-level to macroblock-level importance, the network can "save" the most critical parts of a frame even when it cannot transmit the entire stream. While effective, the current approach has limitations, including high computational complexity for real-time encoding and the use of fixed thresholds for importance levels. Future work will focus on validating this approach for modern codecs like H.265 (HEVC) or AV1 and developing more dynamic, sliding-window thresholds for real-world broadcasting scenarios.

Summary updated

Abstract

Abstract Transporting a stereo video stream over an unreliable channel can degrade severely the video quality. Network congestion is one of the important factors affecting the video quality. To improve the stereo video quality, a stereo video congestion control method, based on the macroblock importance, is proposed. Firstly, the importance of every macroblock is calculated according to its influence on the current frame and future frames. Then, combined with the importance information, all macroblocks of a frame are divided into three levels. Finally, the packets are transported with the help of RTP/RTCP protocol. We discard some not so important packets actively according to the importance levels when a congestion occurs. The results from NS2 simulation platform show that the stereo video quality is improved with the method proposed.
BibTeX
@inproceedings{wang2015macroblock,
  author = {Xiaodong Wang and Ting Yao and Shaobo Li and Lianjun Zhang and Chaolin Wang},
  title = {{Macroblock Importance Based Method for Stereo Video Congestion Control}},
  booktitle = {Cybernetics},
  year = {2015},
  doi = {10.1515/cait-2015-0069},
}
Figure