Compensating for Motion Estimation Inaccuracies in DVC

Jürgen Slowack, Jozef Skorupa, Stefaan Mys, Nikos Deligiannis, Peter Lambert, Adrian Munteanu, Rik Van de Walle

Abstract

Distributed Video Coding (DVC) shifts the computational burden of motion estimation from the encoder to the decoder, making it ideal for low-power applications. However, decoder-side motion estimation often suffers from inaccuracies due to non-linear motion. This paper proposes a multi-predictor ensemble approach that adaptively corrects these errors, achieving quality improvements of up to 1.4 dB over standard benchmarks.

Introduction: The Challenge of Decoder-Side Motion Estimation

In traditional video compression standards like H.264/AVC, the encoder performs intensive motion estimation to identify temporal redundancies. While effective, this process is computationally demanding, posing a challenge for power-constrained devices such as wireless sensor nodes or mobile cameras.

Distributed Video Coding (DVC) addresses this by moving complexity to the decoder. The decoder generates an estimate of the current frame, known as Side Information (SI), typically by interpolating between past and future reference frames. A significant limitation of standard DVC is its reliance on the assumption of linear motion—the idea that objects move at constant speeds in straight lines. In real-world scenarios involving camera shake, acceleration, or occlusions, this assumption fails, resulting in poor SI quality and requiring more bits for error correction.

Side Information (SI) generated at the decoder often contains significant artifacts due to motion estimation errors.

Proposed Methodology

The authors introduce a "safety net" for non-linear motion by replacing the single-predictor model with a weighted ensemble of spatial predictors. The method involves three core stages:

  1. Multi-Predictor Generation: Instead of a single interpolated block, the decoder generates nine candidate predictors: the standard co-located block in the SI and its eight immediate spatial neighbors ($x \pm 1, y \pm 1$).
  2. Online Weight Calculation: Predictors are weighted based on their likelihood of being the optimal match. This is determined through an online procedure: after decoding a frame, the system compares the result against the predictors using Mean Absolute Difference (MAD) to update a global statistics table for subsequent frames.
  3. Weighted Correlation Model: The traditional Laplace distribution used to model "noise" between the original frame and SI is replaced with a mixture of weighted Laplace distributions. This provides the turbo decoder with a more sophisticated statistical model for error correction.
The DVC decoding process: Reference frames are used to generate Side Information, which is then refined into the final Decoded frame using a Residual calculated via parity bits.

Experimental Results

The proposed codec was evaluated using standard CIF sequences at 30 fps (GOP size of 8):

  • Foreman: Moderate motion and panning.
  • Football: Complex, non-linear athletic motion.
  • Coastguard: Simple, linear motion.

Key Findings

  • Quality Gains: The multi-predictor method outperformed the DISCOVER codec by up to 1.4 dB in PSNR for high-motion sequences (Football and Foreman).
  • Incremental Improvement: The approach provided a 0.4 dB boost over the authors' previous noise-model improvements.
  • Adaptive Efficiency: Gains were minimal (0.1 dB) for the linear Coastguard sequence, demonstrating that the system correctly prioritizes linear predictors when they are valid.

Discussion and Future Work

The core innovation lies in the "neighborhood safety net"—the realization that if a linear motion vector is slightly inaccurate, the correct block is likely nearby. To avoid noise interference, a neighbor is only prioritized if its MAD is at least 10% lower than the linear default.

Future improvements could include:

  • Adaptive Learning Rates: Replacing the fixed update parameter ($K=0.8$) with an adaptive one to better track sudden motion changes.
  • Occlusion Handling: Incorporating unidirectional predictors for appearing or disappearing objects.
  • Spatial Adaptivity: Calculating weights locally rather than globally to better distinguish between moving foregrounds and static backgrounds.

Summary updated

Abstract

Distributed video coding is a relatively new video coding approach, where compression is achieved by performing motion estimation at the decoder. Current techniques for decoder-side motion estimation make use of assumptions such as linear motion between the reference frames. It is only after the frame is partially decoded that some of the errors are corrected. In this paper, we propose a new approach with multiple predictors, accounting for inaccuracies in the decoder-side motion estimation process during the decoding. Each of the predictors is assigned a weight, and the correlation between the original frame at the encoder and the set of predictors at the decoder is modeled at the decoder. This correlation information is then used during the decoding process. Results indicate average quality gains up to 0.4 dB.
BibTeX
@inproceedings{slowack2010compensating,
  author = {Jürgen Slowack and Jozef Skorupa and Stefaan Mys and Nikos Deligiannis and Peter Lambert and Adrian Munteanu and Rik Van de Walle},
  title = {{Compensating for Motion Estimation Inaccuracies in DVC}},
  booktitle = {ICISP},
  year = {2010},
  doi = {10.1007/978-3-642-13681-8\_38},
}
Figure