TL;DR
This paper tackles the challenge of delivering constant quality video over fluctuating network conditions using a leaky prediction-based Fine Granularity Scalability (L-FGS) video codec. They introduce a novel Rate-Distortion (R-D) model that accounts for frame dependencies and correlations, leading to a more accurate bit allocation strategy that significantly reduces video quality variations compared to traditional methods.
What Problem Does This Paper Address?
The internet is a heterogeneous environment with varying network bandwidths and diverse user requirements. Delivering high-quality video streaming in such an environment is difficult. Fine Granularity Scalability (FGS) video coding offers flexibility by allowing bitstream truncation to adapt to available bandwidth. However, a specific type of FGS, Leaky Prediction FGS (L-FGS), which uses information from previous frames to improve coding efficiency, introduces temporal dependencies that complicate rate adaptation. Specifically, accurately predicting the Rate-Distortion (R-D) behavior of L-FGS frames is challenging because the distortion in a current frame depends not only on its own bit allocation but also on the distortion propagated from previously adapted frames. This makes it hard to achieve "constant quality" (i.e., consistent visual quality across frames) while staying within bandwidth constraints.
Methods
The core of this work lies in two main contributions:
- Accurate R-D Model for L-FGS: The authors propose a new R-D model that goes beyond simply considering the distortion in the current frame and the distortion propagated from the reference frame. Crucially, it also models the correlation between these two distortion components. This is achieved by building upon previous work on estimating distortion propagation due to channel errors and adapting it to rate adaptation in L-FGS. The model is derived through extensive experimentation and mathematical formulation, incorporating terms that capture the interdependencies.
- Sliding Window Bit Allocation Algorithm: Based on the accurate R-D model, a practical bit allocation algorithm is developed. This algorithm uses a sliding window approach combined with a bisection search. The goal is to allocate bits to frames within the window such that their distortion is as close as possible to a target distortion, while ensuring the total bits do not exceed the available bandwidth. This process is repeated iteratively to find an optimal bit distribution for constant quality.
Datasets
The experiments were conducted on five QCIF format video sequences: Foreman, Akiyo, Mother & Daughter, News, and Coastguard. These sequences are standard benchmarks for video coding research.
Key Insights and Ideas
- Inter-frame Distortion Correlation is Crucial: The most significant insight is that the independence assumption between current frame distortion and propagated distortion is incorrect for L-FGS. Modeling this correlation is key to achieving accurate R-D prediction and, consequently, effective rate adaptation.
- Linear R-D Holds for L-FGS Bitplanes: Similar to baseline FGS, the R-D relationship within each bitplane of the L-FGS enhancement layer is found to be approximately linear, which simplifies offline R-D curve computation.
- Sliding Window Balances Quality and Rate: The sliding window approach effectively balances the need for consistent quality across a group of frames with the dynamic nature of network bandwidth, allowing for adaptation to both CBR and VBR channels.
- Accuracy Without Constant Re-estimation: A notable property of their R-D model is its accuracy even when applied over long video sequences without updating actual distortion values, indicating robustness and reduced computational overhead during streaming.
Key Results
- Accurate Distortion Estimation: The proposed R-D model demonstrates very accurate distortion estimation, with relative errors typically within a small range (e.g., [-0.04, 0.03] for Foreman).
- Significantly Smoother Video Quality: The proposed bit allocation scheme achieves much smoother video quality, as evidenced by a substantial reduction in the standard deviation of frame distortions (Distortion STD). For example, in Table 3, the STD is reduced from over 11 dB to as low as -1.00 dB for Foreman at 32+96 kbps with a window size of 80.
- Competitive Average Distortion: While prioritizing quality consistency, the average distortion achieved by the proposed method is comparable to, or slightly lower than, uniform bit allocation.
- Adaptability to VBR: The scheme effectively adapts to time-varying channels (VBR), maintaining local quality consistency even when bandwidth fluctuates significantly.
Limitations and Future Work
- Decoder Buffer Size and Pre-loading Time: The proposed scheme requires a larger decoder buffer size and longer pre-loading time to prevent underflow and overflow, which is a trade-off for improved distortion STD.
- Frame-Level Bit Allocation: The current algorithm focuses on optimal bit allocation among P-frames, with the I-frame receiving the remaining bandwidth. Future work could explore optimal frame-level bit allocation for single-layer video coding.
- Adaptive Leaky Prediction Factor: The paper assumes a fixed leaky prediction factor. Investigating how to adaptively choose this factor could further enhance performance.
- More Practical Issues: Future research will consider more practical issues like bit allocation under stricter constraints on pre-loading time and buffer size.
Summary updated