TL;DR
This paper proposes a novel video compression technique for video-telephony that prioritizes visually salient regions (ROIs) identified by a visual attention (VA) model. By blurring non-ROI areas, they achieve significant bitrate reduction with comparable or improved perceived quality compared to standard MPEG-1, outperforming a previous VA-based method.
What Problem Does This Paper Address?
Efficiently compressing video streams for applications like video-telephony is crucial, especially given bandwidth limitations. Traditional methods often apply uniform compression across the entire frame, which is inefficient because human vision is not uniform. Our fovea, the central part of our vision, has high acuity, while peripheral vision has much lower resolution. This means we only perceive a small portion of a scene in high detail at any given moment. Therefore, encoding every part of a video frame with the same quality is wasteful. The challenge is to intelligently identify these "regions of interest" (ROIs) that capture human attention and allocate more compression resources to them, while less important areas can be compressed more aggressively without significantly degrading the perceived quality. Existing ROI estimation methods often lack a strong scientific basis for identifying what humans actually find interesting.
Methods
The core of this work is a visual attention (VA) model designed to identify salient regions in video frames. The authors build upon the influential Itti and Koch model, which uses a bottom-up approach to compute "conspicuity maps" based on intensity, color, and orientation.
Key innovations include:
- Enhanced Top-Down Channel: They incorporate a top-down channel that specifically models the human tendency to search for faces. This is particularly relevant for video-telephony, where faces are almost always present and are a primary focus of attention. A multiresolution skin detector is used to generate a "face map."
- Wavelet-based Multiscale Analysis: Instead of Gaussian pyramids, they employ wavelet transforms for multiscale analysis. This is claimed to be computationally more efficient and provides a different way to capture center-surround differences, mimicking visual receptive fields.
- YCbCr Color Space: They utilize the YCbCr color space, which is beneficial for face detection and allows for better separation of luminance (Y) and chrominance (Cb, Cr) information.
- Saliency Map Combination: A sigmoid (saturate) function is used to combine the individual conspicuity maps (intensity, orientation, color, and face). This approach is designed to preserve the distinctiveness of each feature map, preventing strong stimuli in one modality from being diluted by weaker stimuli in others.
- Priority Encoding: For experimental purposes, non-ROI areas are blurred using a smoothing filter before being passed to the video encoder. This simple approach aims to reduce the number of non-zero DCT coefficients in Intra-coded frames and lower prediction errors in Inter-coded frames, leading to better compression.
Datasets
The paper does not explicitly mention training datasets. For evaluation, they used ten short video clips (e.g., "eye_witness," "fashion," "grandma," "justice," "lecturer," "news_cast1," "news_cast2," "night_interview," "old_man," "soldier"). These clips were chosen for their varied content, including humans and objects, and were representative of news reports suitable for 3G video telephony, though not actual 3G video telephony clips.
Key Insights and Ideas
- Human Face as a Strong Attractor: Explicitly modeling the human search for faces significantly improves the identification of salient regions in video-telephony, as faces are a primary focus of attention.
- Wavelets for Efficiency: Using wavelets for multiscale analysis offers a computationally efficient alternative to Gaussian pyramids for building conspicuity maps.
- Saliency Map Combination Matters: A simple summation of feature maps can dilute important information. A saturate function helps preserve the impact of strong visual cues from individual modalities.
- Blurring Non-ROIs is Effective: Even a simple blurring of non-ROI areas can lead to substantial bitrate gains without a noticeable degradation in perceived quality, demonstrating the effectiveness of attention-based prioritization.
- Beyond Static Images: Extending visual attention models from static images to video, especially for specific applications like video-telephony, offers practical benefits.
Key Results
- Bitrate Gain: The proposed VA-ROI based encoding achieved an average bitrate gain of 28.3% compared to standard MPEG-1.
- Quality Comparison: In visual trials, the proposed VA-ROI method was preferred 37.5% of the time, compared to standard MPEG-1 at 44% and Itti's ROI method at 18.5%. The authors note that the difference between their method and standard MPEG-1 is small enough to suggest no significant quality deterioration.
- Outperforming Itti's Method: The proposed method consistently outperformed Itti's ROI-based encoding in both compression ratio and perceived quality across the tested video sequences.
- Compression Ratio: For a specific example, the compression ratio achieved was about 1.44:1 compared to standard JPEG.
Limitations and Future Work
- Limitations:
- The current priority encoding method (blurring non-ROI areas) is a simple approach and not necessarily optimal for maximizing encoding gain.
- The video clips used for testing were not actual 3G video telephony clips, though they were chosen to be representative.
- The presence of TV channel logos, which might attract attention, is considered unusual in typical 3G video telephony scenarios, potentially limiting the generalizability of the face-detection bias.
- Future Work:
- MPEG-4 Integration: Test the proposed method's efficiency within the MPEG-4 framework.
- Adaptive Quantization: Explore incorporating priority encoding by dynamically varying the quality factor of the DCT quantization table for ROI and non-ROI blocks.
- Further Experimental Validation: Conduct experiments with actual 3G video telephony content.
Summary updated