Edge-Directed Geometric Partitioning For Versatile Video Coding

Xuewei Meng, Xinfeng Zhang, Chuanmin Jia, Xia Li, Shanshe Wang, Siwei Ma

TL;DR

This paper introduces an "edge-directed geometric partitioning" (E-GEO) method to improve video compression efficiency in the upcoming Versatile Video Coding (VVC) standard. By intelligently predicting the best geometric partition modes based on object edges, the method significantly reduces the overhead associated with signaling these modes, leading to better compression gains and improved visual quality at object boundaries.

What Problem Does This Paper Address?

Modern video coding standards like H.264/AVC and H.265/HEVC rely on block-based partitioning to compress video data. While flexible partitioning schemes like quadtrees and multi-type trees (QT-MTT) have been developed, they still struggle to accurately represent the arbitrary shapes of objects within rectangular coding blocks. This limitation can lead to inefficiencies in motion estimation and compensation, particularly around object edges, potentially causing visual artifacts. The Versatile Video Coding (VVC) standard introduces a geometric partition (GEO) scheme to address this, offering up to 140 partition candidates. However, the overhead of signaling the index of the chosen GEO mode can be substantial, negating some of the compression benefits. This paper aims to reduce this signaling overhead and improve the overall coding efficiency of GEO by predicting the most probable GEO modes.

Methods

The core of this paper is the proposed Edge-Directed Geometric Partitioning (E-GEO) scheme. The authors leverage the observation that geometric partitions are most effective at object boundaries. Their approach involves:

  • Most Probable Mode (MPM) List Construction: Instead of signaling the full GEO index, E-GEO constructs a smaller list of "most probable modes" (MPM) for each coding unit (CU). This significantly reduces the bits needed to signal the chosen mode.
  • Spatio-Temporal Edge Detection: To build the MPM list, E-GEO first estimates an edge map of the current CU. This is done by combining information from both the current frame (using a guided filter and Canny edge detector) and temporally collocated blocks in reference frames. This edge information is crucial for guiding the partition selection.
  • Reference Block Derivation: Before edge detection, a relevant reference block is identified using motion vectors and a similarity metric (Mean of Squared Difference) between template areas.
  • MPM List Generation: The edge map and angle information are then used to construct the MPM list. The most probable modes are selected based on the detected edge characteristics and their correlation with spatial and temporal neighbors. A default mode is included if the predicted modes don't fill the list.

The E-GEO scheme is applied only for merge mode in VVC, which helps to manage complexity by avoiding explicit motion estimation for geometric partitions.

Datasets

The paper uses standard video sequences from the common test conditions for VVC, specifically mentioning:

  • Test Sequences: Tango2, FoodMarket4, Campfire, Catrobot, DaylightRoad2, ParkRunning3, Marketplace, RitualDance, Cactus, BasketballDrive, BQTerrace, BasketballDrill, BQMall, Partyscene, RaceHorsesC, Four People, BQJohny, KristenAndSara.
  • Configurations: Random Access (RA) and Low Delay B (LDB).
  • Quantization Parameters (QPs): Used for training and evaluation, including typical values like 22, 27, 32, and 37.

The authors also mention using four specific sequences (BasketballPass, BQSquare, BlowingBubbles, and RaceHorses) for parameter training.

Key Insights and Ideas

  • Object Edges are Key: The most significant insight is the strong correlation between geometric partition modes and object boundaries. This observation is the foundation for predicting GEO modes effectively.
  • MPM for Overhead Reduction: Using an MPM list for GEO mode selection is a novel approach to drastically cut down the signaling overhead, which was a major bottleneck for GEO's efficiency.
  • Spatio-Temporal Edge Information: Combining edge information from both spatial neighbors and temporal references provides a more robust edge map, leading to better MPM prediction.
  • Guided Filter for Edge Detection: Employing a guided filter in the edge detection process helps preserve object boundaries while reducing quantization noise, which is crucial for accurate edge-based prediction.

Key Results

The proposed E-GEO method demonstrates significant improvements:

  • BD-rate Reduction: Achieves an average BD-rate gain of 0.58% for RA and 1.00% for LDB configurations compared to the VTM-6.0 anchor. This represents a substantial improvement over the original GEO method, with gains of 41% and 43% respectively.
  • Improved Subjective Quality: E-GEO visibly enhances the subjective quality of video, particularly around the boundaries of moving objects, by better approximating their shapes.
  • Effective MPM Prediction: The paper shows that the proposed method can predict GEO partition modes effectively, with a high percentage of selected GEO modes falling within the MPM list.

Limitations and Future Work

  • Complexity Management: While E-GEO reduces signaling overhead, the initial edge detection and MPM list construction still add some computational complexity. The authors mention that the complexity remains comparable to existing methods, but further optimization might be explored.
  • Further Optimization: The authors acknowledge that there is still "space for optimization" in their prediction mechanism. They suggest investigating "other GEO mode prediction methods" in future work.
  • Parameter Training: The paper details a parameter training process for the edge detection filters. While effective, the specific values chosen might be further refined or adapted for different content types or coding conditions.

Summary updated

Abstract

To improve the coding performance, geometric partition (GEO) was proposed for the upcoming VVC standard. GEO provides 140 partition candidates. The index of optimal GEO mode needs to be signaled explicitly. Considering different structural characteristics of different CUs and the correlation between spatial adjacent blocks and temporal collocated blocks, we propose a GEO mode prediction strategy by constructing a Most Probable Mode (MPM) list to reduce the overhead of GEO index and improve coding efficiency. Based on the observation of the high correlation between the partition mode and object boundaries, an edge-directed geometric partition scheme is proposed to construct the MPM list according to spatio-temporal edge information. The proposed method provides an objective BD-rate gain of 0.58% and 1.00% on average for RA and LDB configurations compared to VTM-6.0. Besides, it also promotes the visual quality of object boundaries.
BibTeX
@inproceedings{meng2020edge,
  author = {Xuewei Meng and Xinfeng Zhang and Chuanmin Jia and Xia Li and Shanshe Wang and Siwei Ma},
  title = {{Edge-Directed Geometric Partitioning For Versatile Video Coding}},
  booktitle = {ICME},
  year = {2020},
  doi = {10.1109/icme46284.2020.9102781},
}
Figure