Abstract
Most current video streaming technologies protect the video stream by rate allocation or CRC means according to network bandwidth and expected video quality, regardless of content. This paper proposes a new content-based video streaming method based on video content analysis. The method discusses the rate distortion function of Fine Granular Scalable (FGS) enhancement layer bitstream based on video content and allocates suitable bits for video units by the importance of video content. Experimental results indicate that this content-based rate allocation method protects video transmission efficiently, improves subjective quality, and maximizes network bandwidth utility.
Introduction
Streaming video over networks without Quality-of-Service (QoS) guarantees, such as the early 2000s Internet, presented significant challenges due to fluctuating bandwidth, delay, and packet loss. Fine Granular Scalable (FGS) coding was developed to address this by allowing video to be "scaled" to fit available bandwidth by trimming an enhancement layer. However, traditional FGS methods were "content-blind," treating every frame with equal importance.
This paper introduces a content-aware bit allocation strategy for FGS video streaming. By prioritizing high-motion segments through a weighted rate-distortion model, the algorithm ensures that the most "important" parts of a video maintain higher quality even when network bandwidth is limited. Human viewers perceive quality differently depending on content; for instance, high-action sequences require more bits than static backgrounds to maintain acceptable subjective quality.
Methodology
The proposed algorithm integrates video content analysis with FGS rate control through three primary steps:
1. Content Analysis and Importance Weighting
The video is segmented into clips, and the "importance" ($w$) of each clip is determined by its motion activity. This is calculated using Global Motion Estimation (GME). The assumption is that higher motion correlates with higher visual importance to the viewer.
2. Weighted Rate-Distortion (R-D) Model
The standard R-D function is modified to include the weight coefficient ($w$): $$D(R) = w \cdot \sigma^2 \cdot e^{-aR}$$ where $D$ is distortion, $R$ is the bit count, and $\sigma^2$ is the signal variance.
3. Optimization and Bit Allocation
To minimize total distortion across all clips given a fixed bit budget, the authors employ the Lagrange multiplier method. To ensure the algorithm is practical for real-time streaming, they derived a simplified bit allocation formula that avoids computationally expensive logarithmic operations. This allows bit allocation to be performed on the FGS enhancement layer after encoding but before transmission, based on real-time channel feedback.
Experimental Results
The algorithm was evaluated using the "Stefan" sequence, a standard MPEG-4 test sequence (SIF format, 352x240 resolution, 30 fps) known for its high-motion tennis action.
Key findings include:
- PSNR Gains: The proposed method improved the Peak Signal-to-Noise Ratio (PSNR) by 0.5 to 1.0 dB during high-motion segments.
- Subjective Quality: In the "Stefan" sequence (specifically frames 175–200), which features intense motion, the PSNR was a full 1.0 dB higher than the baseline, resulting in visibly clearer action.
- Bandwidth Efficiency: The method demonstrated more efficient use of network bandwidth by "spending" bits where they provide the most subjective value.
Conclusion and Future Work
The core innovation of this work is using motion activity as a proxy for bit priority, decoupling encoding from allocation to allow for real-time adjustments. While effective, the current model relies solely on motion. The authors acknowledge that "importance" is multifaceted and suggest that future versions should incorporate video structure analysis and objective retrieval. Future research will also aim to expand this into a comprehensive architecture including error resilience and concealment for a full content-based delivery system.
Summary updated