Conservative Wasserstein Training for Pose Estimation

Xiaofeng Liu, Yang Zou, Tong Che, Peng Ding, Ping Jia, Jane You, B. V. K. Vijaya Kumar

Pose estimation is a fundamental task in computer vision, but it has a tricky mathematical property: its labels are both discrete (e.g., binned into 5-degree intervals) and periodic (e.g., 359° is right next to 0°). Standard deep learning loss functions often fail to capture this circular reality. This paper introduces a "Conservative Wasserstein" training framework that explicitly models these inter-class relationships and accounts for label noise, leading to state-of-the-art accuracy across multiple benchmarks.

TL;DR

The authors propose a novel loss function for pose estimation based on the Wasserstein distance (Earth Mover's Distance) that respects the periodic nature of angular data. By using "conservative" target labels that model both inlier and outlier noise, they achieve significant performance gains and faster convergence compared to traditional cross-entropy or regression losses.

What Problem Does This Paper Address?

Deep learning models for pose estimation typically treat the problem as either classification or regression.

  • Classification using Cross-Entropy (CE) loss assumes all classes are independent. It fails to recognize that adjacent angles are similar; a model predicting 10° when the truth is 0° is penalized just as heavily as if it predicted 180°.
  • Regression understands numerical distance but often struggles with discrete labels and the "wrap-around" nature of periodic data.

Furthermore, real-world pose datasets are plagued by label noise: "inlier noise" (slight mislabeling of nearby angles) and "outlier noise" (completely wrong labels). Standard losses are not designed to handle this specific noise structure in angular space.

The limitation of CE loss: it doesn't care if a prediction is "close" to the ground truth, only if it hits the exact class.

Methods

The authors propose using the Wasserstein distance as a loss function. Unlike CE, Wasserstein distance incorporates a ground metric—a cost function that defines how "far" one class is from another.

  1. Periodic Ground Metrics: They define the ground metric based on the arc length of a circle, ensuring the model understands the wrap-around at 0/360°. They explore linear, convex, and concave functions of this arc length.
  2. Conservative Target Labels: Instead of using "one-hot" labels (where only one class has a probability of 1), they smooth the target distribution using a wrapped unimodal-uniform mixture. This models inlier noise with a unimodal distribution (like a Binomial) and outlier noise with a uniform distribution.
  3. Fast Optimization: A major hurdle for Wasserstein distance is its $O(N^3)$ computational cost. The authors derive efficient, closed-form solutions ($O(N)$ complexity) specifically for periodic data, making it practical for large-scale deep learning.
Figure 2. Left: The only possible transport plan in one-hot target case. Right: the ground matrix using arc length as ground metric.

Datasets

The method was evaluated on four diverse pose estimation benchmarks:

  • CAVIAR: Head pose estimation in occluded scenarios.
  • TUD Multi-view Pedestrians: Estimating the orientation of people walking.
  • EPFL: Vehicle orientation estimation from image sequences.
  • PASCAL3D+: A challenging benchmark for 3D object pose estimation across 12 categories.

Key Insights and Ideas

  • Convexity is Key: The best results consistently came from using a convex ground metric (e.g., squared arc length). This strongly penalizes predictions far from the ground truth while being more tolerant of slight offsets.
  • Smoothing Periodic Labels: Standard label smoothing doesn't work for angles because it ignores their order. The proposed "wrapped" smoothing correctly distributes probability to neighboring angles on the circle.
  • Adaptive Learning: In some cases, learning the ground metric from the data itself (adaptive ground metric) can improve performance by capturing dataset-specific similarities that a simple arc-length doesn't account for.
Figure 3. Left: the wrapping operation with a Binomial distribu- tion (K + 1 is the number of involved classes of unimodal distri- bution). Right: the distribution of conservative target label.

Key Results

The "Conservative Wasserstein" approach achieved state-of-the-art results across the board:

  • Head Pose (CAVIAR): Reduced Mean Absolute Angular Deviation (MAAD) from 23.7° (previous best) to 17.1°.
  • Vehicle Orientation (EPFL): Achieved a 27.8% reduction in Mean Absolute Error compared to the previous state-of-the-art.
  • Efficiency: The proposed exact closed-form solution is 5x faster to train than approximate Wasserstein methods and 3x faster than conventional regression-based approaches.

Limitations and Future Work

The authors note that their adaptive ground metric learning can become unstable when the number of classes is large (e.g., 36 or more). Future research will focus on stabilizing this for high-resolution pose estimation. Additionally, they aim to automate the selection of hyperparameters for the conservative target distribution, which currently requires manual tuning via grid search.

Summary updated

Abstract

This paper targets the task with discrete and periodic class labels ($e.g.,$ pose/orientation estimation) in the context of deep learning. The commonly used cross-entropy or regression loss is not well matched to this problem as they ignore the periodic nature of the labels and the class similarity, or assume labels are continuous value. We propose to incorporate inter-class correlations in a Wasserstein training framework by pre-defining ($i.e.,$ using arc length of a circle) or adaptively learning the ground metric. We extend the ground metric as a linear, convex or concave increasing function $w.r.t.$ arc length from an optimization perspective. We also propose to construct the conservative target labels which model the inlier and outlier noises using a wrapped unimodal-uniform mixture distribution. Unlike the one-hot setting, the conservative label makes the computation of Wasserstein distance more challenging. We systematically conclude the practical closed-form solution of Wasserstein distance for pose data with either one-hot or conservative target label. We evaluate our method on head, body, vehicle and 3D object pose benchmarks with exhaustive ablation studies. The Wasserstein loss obtaining superior performance over the current methods, especially using convex mapping function for ground metric, conservative label, and closed-form solution.
BibTeX
@misc{liu2019conservative,
  author = {Xiaofeng Liu and Yang Zou and Tong Che and Peng Ding and Ping Jia and Jane You and B. V. K. Vijaya Kumar},
  title = {{Conservative Wasserstein Training for Pose Estimation}},
  year = {2019},
  eprint = {1911.00962},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.00962},
}
Figure