Decoding order recovery for multi flow transmission of scalable video coding (SVC) over mobile IP channels

Thomas Schierl, Yago Sánchez de la Fuente, Cornelius Hellge, Thomas Wiegand

Abstract

Scalable Video Coding (SVC) enables adaptive video delivery by splitting content into base and enhancement layers. When these layers are transmitted over separate IP flows in mobile networks, receivers face the challenge of cross-flow decoding order recovery (DOR). This paper evaluates two standardized DOR methods: the simple Timestamp-based (TS-based) approach and the more complex Cross-Session Decoding Order Number (CS-DON) approach. Through simulations of bursty mobile channels, we demonstrate that the TS-based method achieves nearly identical video quality to CS-DON while avoiding additional signaling overhead.

Introduction

Scalable Video Coding (SVC) is a sophisticated encoding standard that partitions video into a "base layer" (providing basic quality) and one or more "enhancement layers" (improving resolution, frame rate, or quality). In modern mobile environments, these layers are often transmitted over distinct IP flows. This multi-flow approach allows network operators to prioritize the base layer, ensuring a minimum level of service even under poor channel conditions.

However, this architecture introduces a significant challenge at the receiver: the "jigsaw puzzle" problem. Because IP packets in different flows may experience varying delays or paths, they often arrive out of order. To successfully decode the video, the receiver must reassemble these packets into their exact decoding order. While the Real-time Transport Protocol (RTP) handles reordering within a single flow via sequence numbers, it lacks a native mechanism to synchronize decoding order across multiple sessions.

Decoding Order Recovery (DOR) Mechanisms

To address cross-flow synchronization, the industry has standardized two primary Decoding Order Recovery (DOR) methods:

  1. Timestamp-based (TS-based) DOR: This method utilizes existing RTP presentation timestamps to align flows. It operates on the assumption that if a specific timestamp exists in an enhancement layer, the corresponding data must also be present in the base layer. It is computationally simpler and requires no additional header overhead.
  2. Cross-Session Decoding Order Number (CS-DON): This is a more explicit approach where every packet is tagged with a global decoding number. While this makes reassembly straightforward, it adds data overhead to every packet, which can be costly in bandwidth-constrained mobile links.
Mechanism of Timestamp-based (TS-based) Decoding Order Recovery across multiple RTP flows.

Critics of the TS-based method argued that it would fail during packet loss because video timestamps do not always increase linearly (e.g., due to B-frames). This study investigates whether this theoretical vulnerability impacts real-world performance.

Experimental Evaluation

The researchers developed a simulation testbed to compare these methods in a DVB-H (Digital Video Broadcasting - Handheld) environment. They employed a Gilbert-Elliot model to simulate the "bursty" packet losses typical of mobile channels, where interference or physical obstructions cause data to be lost in contiguous chunks.

The study utilized a 40-second video sequence composed of standard test sequences (City, Crew, Foreman, and Soccer). The video was encoded in SVC with two flows: a base layer at QVGA resolution (15 fps) and an enhancement layer providing temporal (30 fps) and quality (MGS) improvements.

Results and Discussion

The evaluation revealed a critical distinction between burst loss and random loss scenarios. While the TS-based method struggles with random packet loss—which creates numerous small, ambiguous gaps—it performs exceptionally well under burst loss conditions common in mobile networks.

In a burst loss event, the receiver typically loses enough data that it simply reverts to the base layer until the next synchronization point is reached. This effectively bypasses the reordering puzzle during the period of high loss. As shown in the performance analysis, the difference in video quality (measured in PSNR) between the simple TS-based method and the complex CS-DON method was less than 0.01 dB in realistic bursty conditions.

Average PSNR comparison between CS-DON and TS-based methods under burst and random loss conditions.

Significant degradation for the TS-based method only occurred in unrealistic random loss scenarios. However, at the loss rates where this gap appeared (above 2.5%), the overall video quality was already so poor that the choice of DOR method became practically irrelevant.

Conclusion

The findings suggest that the theoretical concerns regarding timestamp-based synchronization are largely mitigated by the nature of mobile channel losses. The TS-based DOR method provides a robust, efficient alternative to CS-DON, offering comparable video quality without the need for additional signaling overhead. For mobile broadcast applications, the simpler approach is not only sufficient but preferable for maximizing bandwidth efficiency.

Summary updated

Abstract

Recent standardization activities for the transport of Scalable Video Coding over multimedia networking channels have introduced means to differentiate layers of a scalable media in transport. Therefore in RTP, those layers are transported in different RTP flows respectively RTP sessions. To decode such layers transported on different IP network flows, a decoding order recovery mechanism is required that reorders the media data of the layers to decoding order at the receiver or at an intelligent network element. Since today's multimedia networking channels typically provide presentation timestamps on the transport layer, it was obvious to rely on those data for decoding order recovery. Due to the fact that presentation timestamps do not appear in the media stream in increasing order, an algorithm purely based on presentation timestamps may fail in some scenarios where temporal scalability is used. Another approach to solve the problem is to use a numbering mechanism in the bit stream giving the decoding order. In this paper, we highlight both mechanisms as defined in the RTP payload format for SVC video and compare them in loss scenarios typically present in mobile broadcast channels. We show that the presentation timestamp based solution gives over a wide range of loss scenarios almost the same results in terms of decoded video quality as the numbering based approach.
BibTeX
@inproceedings{schierl2009decoding,
  author = {Thomas Schierl and Yago Sánchez de la Fuente and Cornelius Hellge and Thomas Wiegand},
  title = {{Decoding order recovery for multi flow transmission of scalable video coding (SVC) over mobile IP channels}},
  booktitle = {MobiMedia},
  year = {2009},
  doi = {10.4108/icst.mobimedia2009.7480},
}
Figure