TL;DR
This paper introduces an "edge-directed geometric partitioning" (E-GEO) method to improve video compression efficiency in the upcoming Versatile Video Coding (VVC) standard. By intelligently predicting the best geometric partition modes based on object edges, the method significantly reduces the overhead associated with signaling these modes, leading to better compression gains and improved visual quality at object boundaries.
What Problem Does This Paper Address?
Modern video coding standards like H.264/AVC and H.265/HEVC rely on block-based partitioning to compress video data. While flexible partitioning schemes like quadtrees and multi-type trees (QT-MTT) have been developed, they still struggle to accurately represent the arbitrary shapes of objects within rectangular coding blocks. This limitation can lead to inefficiencies in motion estimation and compensation, particularly around object edges, potentially causing visual artifacts. The Versatile Video Coding (VVC) standard introduces a geometric partition (GEO) scheme to address this, offering up to 140 partition candidates. However, the overhead of signaling the index of the chosen GEO mode can be substantial, negating some of the compression benefits. This paper aims to reduce this signaling overhead and improve the overall coding efficiency of GEO by predicting the most probable GEO modes.
Methods
The core of this paper is the proposed Edge-Directed Geometric Partitioning (E-GEO) scheme. The authors leverage the observation that geometric partitions are most effective at object boundaries. Their approach involves:
- Most Probable Mode (MPM) List Construction: Instead of signaling the full GEO index, E-GEO constructs a smaller list of "most probable modes" (MPM) for each coding unit (CU). This significantly reduces the bits needed to signal the chosen mode.
- Spatio-Temporal Edge Detection: To build the MPM list, E-GEO first estimates an edge map of the current CU. This is done by combining information from both the current frame (using a guided filter and Canny edge detector) and temporally collocated blocks in reference frames. This edge information is crucial for guiding the partition selection.
- Reference Block Derivation: Before edge detection, a relevant reference block is identified using motion vectors and a similarity metric (Mean of Squared Difference) between template areas.
- MPM List Generation: The edge map and angle information are then used to construct the MPM list. The most probable modes are selected based on the detected edge characteristics and their correlation with spatial and temporal neighbors. A default mode is included if the predicted modes don't fill the list.
The E-GEO scheme is applied only for merge mode in VVC, which helps to manage complexity by avoiding explicit motion estimation for geometric partitions.
Datasets
The paper uses standard video sequences from the common test conditions for VVC, specifically mentioning:
- Test Sequences: Tango2, FoodMarket4, Campfire, Catrobot, DaylightRoad2, ParkRunning3, Marketplace, RitualDance, Cactus, BasketballDrive, BQTerrace, BasketballDrill, BQMall, Partyscene, RaceHorsesC, Four People, BQJohny, KristenAndSara.
- Configurations: Random Access (RA) and Low Delay B (LDB).
- Quantization Parameters (QPs): Used for training and evaluation, including typical values like 22, 27, 32, and 37.
The authors also mention using four specific sequences (BasketballPass, BQSquare, BlowingBubbles, and RaceHorses) for parameter training.
Key Insights and Ideas
- Object Edges are Key: The most significant insight is the strong correlation between geometric partition modes and object boundaries. This observation is the foundation for predicting GEO modes effectively.
- MPM for Overhead Reduction: Using an MPM list for GEO mode selection is a novel approach to drastically cut down the signaling overhead, which was a major bottleneck for GEO's efficiency.
- Spatio-Temporal Edge Information: Combining edge information from both spatial neighbors and temporal references provides a more robust edge map, leading to better MPM prediction.
- Guided Filter for Edge Detection: Employing a guided filter in the edge detection process helps preserve object boundaries while reducing quantization noise, which is crucial for accurate edge-based prediction.
Key Results
The proposed E-GEO method demonstrates significant improvements:
- BD-rate Reduction: Achieves an average BD-rate gain of 0.58% for RA and 1.00% for LDB configurations compared to the VTM-6.0 anchor. This represents a substantial improvement over the original GEO method, with gains of 41% and 43% respectively.
- Improved Subjective Quality: E-GEO visibly enhances the subjective quality of video, particularly around the boundaries of moving objects, by better approximating their shapes.
- Effective MPM Prediction: The paper shows that the proposed method can predict GEO partition modes effectively, with a high percentage of selected GEO modes falling within the MPM list.
Limitations and Future Work
- Complexity Management: While E-GEO reduces signaling overhead, the initial edge detection and MPM list construction still add some computational complexity. The authors mention that the complexity remains comparable to existing methods, but further optimization might be explored.
- Further Optimization: The authors acknowledge that there is still "space for optimization" in their prediction mechanism. They suggest investigating "other GEO mode prediction methods" in future work.
- Parameter Training: The paper details a parameter training process for the edge detection filters. While effective, the specific values chosen might be further refined or adapted for different content types or coding conditions.
Summary updated