A Novel Content-Based Video Streaming Algorithm for Fine Granular Scalable Coding

Li Zhao, Qi Wang, Yuwen He, Shiqiang Yang, Yuzhuo ZhongMMNS 2001 · OpenReview

Abstract

Most current video streaming technologies protect the video stream by rate allocation or CRC means according to network bandwidth and expected video quality, regardless of content. This paper proposes a new content-based video streaming method based on video content analysis. The method discusses the rate distortion function of Fine Granular Scalable (FGS) enhancement layer bitstream based on video content and allocates suitable bits for video units by the importance of video content. Experimental results indicate that this content-based rate allocation method protects video transmission efficiently, improves subjective quality, and maximizes network bandwidth utility.

Introduction

Streaming video over networks without Quality-of-Service (QoS) guarantees, such as the early 2000s Internet, presented significant challenges due to fluctuating bandwidth, delay, and packet loss. Fine Granular Scalable (FGS) coding was developed to address this by allowing video to be "scaled" to fit available bandwidth by trimming an enhancement layer. However, traditional FGS methods were "content-blind," treating every frame with equal importance.

This paper introduces a content-aware bit allocation strategy for FGS video streaming. By prioritizing high-motion segments through a weighted rate-distortion model, the algorithm ensures that the most "important" parts of a video maintain higher quality even when network bandwidth is limited. Human viewers perceive quality differently depending on content; for instance, high-action sequences require more bits than static backgrounds to maintain acceptable subjective quality.

Methodology

The proposed algorithm integrates video content analysis with FGS rate control through three primary steps:

1. Content Analysis and Importance Weighting

The video is segmented into clips, and the "importance" ($w$) of each clip is determined by its motion activity. This is calculated using Global Motion Estimation (GME). The assumption is that higher motion correlates with higher visual importance to the viewer.

Global Motion Estimation (GME) for the "Stefan" sequence, used to determine segment importance based on motion activity.

2. Weighted Rate-Distortion (R-D) Model

The standard R-D function is modified to include the weight coefficient ($w$): $$D(R) = w \cdot \sigma^2 \cdot e^{-aR}$$ where $D$ is distortion, $R$ is the bit count, and $\sigma^2$ is the signal variance.

3. Optimization and Bit Allocation

To minimize total distortion across all clips given a fixed bit budget, the authors employ the Lagrange multiplier method. To ensure the algorithm is practical for real-time streaming, they derived a simplified bit allocation formula that avoids computationally expensive logarithmic operations. This allows bit allocation to be performed on the FGS enhancement layer after encoding but before transmission, based on real-time channel feedback.

Experimental Results

The algorithm was evaluated using the "Stefan" sequence, a standard MPEG-4 test sequence (SIF format, 352x240 resolution, 30 fps) known for its high-motion tennis action.

Comparison of PSNR results between the proposed content-based method and standard average bit allocation.

Key findings include:

  • PSNR Gains: The proposed method improved the Peak Signal-to-Noise Ratio (PSNR) by 0.5 to 1.0 dB during high-motion segments.
  • Subjective Quality: In the "Stefan" sequence (specifically frames 175–200), which features intense motion, the PSNR was a full 1.0 dB higher than the baseline, resulting in visibly clearer action.
  • Bandwidth Efficiency: The method demonstrated more efficient use of network bandwidth by "spending" bits where they provide the most subjective value.

Conclusion and Future Work

The core innovation of this work is using motion activity as a proxy for bit priority, decoupling encoding from allocation to allow for real-time adjustments. While effective, the current model relies solely on motion. The authors acknowledge that "importance" is multifaceted and suggest that future versions should incorporate video structure analysis and objective retrieval. Future research will also aim to expand this into a comprehensive architecture including error resilience and concealment for a full content-based delivery system.

Summary updated

Abstract

Most of current video streaming technologies, which protect the video stream by rate allocation or CRC means according to the bandwidth of network and the video quality user expected, is unrelated with the content of video. This paper proposes a new content-based video streaming method, which is based on video content analysis. The method discusses the rate distortion function of Fine Granular Scalable (FGS) enhancement layer bitstream based on the video content, and allocates the suitable bits for video units by the importance of video content. The experimental results indicate that our contentbased rate allocation method can protect the transmission of video efficiently, improve the subjective quality of video and use the network bandwidth in maximum utility.
BibTeX
@inproceedings{zhao2001novel,
  author = {Li Zhao and Qi Wang and Yuwen He and Shiqiang Yang and Yuzhuo Zhong},
  title = {{A Novel Content-Based Video Streaming Algorithm for Fine Granular Scalable Coding}},
  booktitle = {MMNS},
  year = {2001},
  url = {https://openreview.net/forum?id=kanwepeV2k},
}
Figure