A Statistical Confidence Measure for Optical Flows

Claudia Kondermann, Rudolf Mester, Christoph S. GarbeECCV 2008 · DOI

TL;DR

This paper introduces a novel, statistically-grounded confidence measure for optical flow estimation that is universally applicable across different flow computation methods. By leveraging natural motion statistics and hypothesis testing, their method provides a more reliable assessment of flow vector accuracy, outperforming existing techniques in experimental evaluations.

What Problem Does This Paper Address?

Estimating optical flow, which describes the motion of objects in a sequence of images, is a fundamental task in computer vision. However, not all estimated flow vectors are equally accurate. To effectively use optical flow in downstream applications like tracking, recognition, or image manipulation, it's crucial to know which flow vectors are reliable and which are not. While confidence measures aim to provide this quality assessment, existing methods have several drawbacks: they often rely solely on image structure without considering the computed flow, are tied to specific flow estimation algorithms, or lack a solid statistical foundation. This leaves a gap for a general, statistically motivated, and robust confidence measure.

Methods

The core of this paper is a statistically motivated confidence measure derived from natural motion statistics. The approach involves:

  • Learning Motion Statistics: A probabilistic motion model is learned from training data (ground truth, synthetic, or even computed flow fields). This model captures the typical spatial and temporal patterns of flow vectors within image patches. The empirical mean and covariance of these flow patches are computed.
  • Hypothesis Testing: A hypothesis test is formulated: "The central flow vector of a given flow field patch follows the underlying conditional distribution given the remaining flow vectors of the patch." This is tested against a multivariate normal distribution assumption for the flow vectors.
  • Mahalanobis Distance: The squared Mahalanobis distance is used as the test statistic. This measures how far a given flow vector deviates from the expected distribution, considering the learned statistics.
  • P-value as Confidence: Instead of a binary rejection/acceptance, the paper adopts the concept of p-values. The p-value for a given flow vector is defined as the minimum significance level at which the hypothesis would be rejected. This p-value directly translates into a continuous confidence measure between 0 and 1, where a higher value indicates higher confidence.

A key aspect is that this method is designed to be independent of the specific optical flow computation method used, making it broadly applicable.

Datasets

The paper uses several standard optical flow benchmark sequences for evaluation:

  • Yosemite
  • Marble
  • Dimetrodon
  • RubberWhale

These sequences are often used with ground truth flow data. The authors mention that the training data can be ground truth, synthetic, computed flow fields, or even data from the same sequence for which confidence is being computed.

Key Insights and Ideas

  • Statistical Justification: The paper argues that their method is the first confidence measure for optical flow that is "statistically justified," moving beyond heuristic or purely image-structure-based approaches.
  • Universality: The ability to apply the confidence measure to any optical flow estimation method is a significant advantage, allowing for flexible integration into existing pipelines.
  • Adaptability: The method can be adapted to specific application domains by choosing training data that reflects typical motion patterns, potentially leading to even better performance than shown in the general case.
  • Fast Methods + Confidence = Global Quality: A striking observation is that by using their confidence measure to sparsify the output of fast, local optical flow methods (like the structure tensor method), they can achieve results comparable in quality to slower, global methods, but with significantly reduced computation time.

Key Results

The paper presents quantitative results using "sparsification" plots, which show the average error of the remaining flow field as a fraction of vectors are removed based on their confidence score.

  • Superior Performance: In most experimental comparisons, the proposed confidence measure (dubbed "pVal") consistently resulted in the lowest average error for the remaining flow vectors across various optical flow methods and test sequences.
  • Comparison to Optimal: The proposed method's sparsification order was often found to be closest to an "optimal confidence" benchmark, which represents the ideal ordering of flow vectors by error.
  • Efficiency Gain: For the structure tensor method, achieving a 90% dense flow field of quality comparable to a 100% dense CLG method was demonstrated, highlighting the practical benefits of combining fast local methods with their confidence measure.
  • Robustness to Training Data: Figure 3 suggests that the method's performance is not overly sensitive to the choice of training data, although using ground truth data yields the best results.

Limitations and Future Work

  • Normal Distribution Assumption: While the Mahalanobis distance is optimal for a normal distribution, the paper acknowledges that the true distribution of flow vectors might not be perfectly normal. This means the test statistic is valid but not necessarily the "uniformly most powerful" test.
  • Patch Size and Training Data: While the paper shows the influence of patch size and training data choice is "rather negligible" (Figure 2 & 3), further optimization or exploration of these parameters could potentially yield even better results.
  • Future Work: The paper implicitly suggests that exploring different statistical models beyond the multivariate normal distribution could be a direction for future research. Additionally, further investigation into adapting the training data for specific applications is a clear avenue for improvement.

Summary updated

BibTeX
@inproceedings{kondermann2008statistical,
  author = {Claudia Kondermann and Rudolf Mester and Christoph S. Garbe},
  title = {{A Statistical Confidence Measure for Optical Flows}},
  booktitle = {European Conference on Computer Vision},
  year = {2008},
  doi = {10.1007/978-3-540-88690-7\_22},
}
Figure