Conservative Wasserstein Training for Pose Estimation

Xiaofeng Liu, Yang Zou, Tong Che, Ping Jia, Peng Ding, Jane You, B. V. K. Vijaya Kumar

Abstract: This paper targets the task with discrete and periodic class labels (e.g., pose/orientation estimation) in the context of deep learning. The commonly used cross-entropy or regression loss is not well matched to this problem as they ignore the periodic nature of the labels and the class similarity, or assume labels are continuous value. We propose to incorporate inter-class correlations in a Wasserstein training framework by pre-defining (i.e., using arc length of a circle) or adaptively learning the ground metric. We extend the ground metric as a linear, convex or concave increasing function w.r.t. arc length from an optimization perspective. We also propose to construct the conservative target labels which model the inlier and outlier noises using a wrapped unimodal-uniform mixture distribution. Unlike the one-hot setting, the conservative label makes the computation of Wasserstein distance more challenging. We systematically conclude the practical closed-form solution of Wasserstein distance for pose data with either one-hot or conservative target label. We evaluate our method on head, body, vehicle and 3D object pose benchmarks with exhaustive ablation studies. The Wasserstein loss obtains superior performance over the current methods, especially using convex mapping function for ground metric, conservative label, and closed-form solution.

Introduction

Estimating the pose or orientation of an object—such as a vehicle or a human head—presents unique challenges for standard deep learning loss functions. Traditional approaches typically fall into two categories: classification and regression. Classification using Cross-Entropy (CE) loss treats each angular bin as an independent category, failing to account for the proximity of adjacent bins (e.g., 359° and 1°). Regression, while continuous, often struggles with the periodic "wrap-around" nature of circular data and can overfit to discretized training labels.

Comparison of probability distributions with identical cross-entropy loss.

As illustrated in Figure 1, standard CE loss cannot distinguish between a "near-miss" distribution and a highly scattered one if the probability at the ground-truth index is the same. This motivates the need for a loss function that understands the underlying geometry of the label space.

Methodology

The core contribution of this work is the application of the Wasserstein distance (Earth Mover’s Distance) to pose estimation. This metric measures the minimum "work" required to transform the predicted probability distribution into the target distribution, naturally incorporating the distance between classes.

Circular Ground Metric

To handle periodicity, the authors define the ground metric—the cost of moving probability mass—based on the arc length of a circle. They extend this by evaluating linear, convex, and concave mapping functions. Empirically, convex functions (which penalize large angular errors more heavily) were found to provide the best performance.

Conservative Target Labels

Standard "one-hot" labels assume perfect ground truth, which is rarely the case in human-annotated pose datasets. The authors propose conservative target labels, which model label noise using a mixture of a unimodal distribution (capturing near-miss inlier noise) and a uniform distribution (capturing random outlier noise).

Construction of conservative target labels using a mixture distribution.

As shown in Figure 3, these labels provide a "softened" target that better reflects the uncertainty inherent in orientation estimation, leading to more robust model training.

Efficient Closed-Form Solutions

While calculating Wasserstein distance is generally computationally expensive ($O(N^3)$), the authors derived efficient, closed-form solutions specifically for circular, discrete data. This reduces the complexity to $O(N)$, making it feasible to use as a loss function in modern deep neural networks.

Experimental Results

The proposed method was evaluated on several major benchmarks, consistently outperforming both Cross-Entropy and standard regression baselines:

  • Head Pose (CAVIAR): Achieved a Mean Absolute Angular Deviation (MAAD) of 17.1°, a significant improvement over the previous state-of-the-art of 23.7°.
  • Vehicle Orientation (EPFL): Reduced Mean Absolute Error (MAE) from 9.86° to 6.04°, a 27.8% improvement.
  • 3D Object Pose (PASCAL3D+): Outperformed complex joint classification-regression frameworks across 12 object categories.

Beyond accuracy, the exact Wasserstein loss was found to converge 5x faster than approximate versions and 3x faster than conventional regression methods.

Conclusion

By integrating the circular topology of pose data into the training objective and accounting for label noise through conservative targets, this framework offers a theoretically sound and practically efficient alternative to standard losses. Future work will explore stabilizing adaptive metric learning for higher-resolution label spaces and automating the hyperparameter tuning for noise distributions.

Summary updated

Abstract

This paper targets the task with discrete and periodic class labels (e.g., pose/orientation estimation) in the context of deep learning. The commonly used cross-entropy or regression loss is not well matched to this problem as they ignore the periodic nature of the labels and the class similarity, or assume labels are continuous value. We propose to incorporate inter-class correlations in a Wasserstein training framework by pre-defining (i.e., using arc length of a circle) or adaptively learning the ground metric. We extend the ground metric as a linear, convex or concave increasing function w.r.t. arc length from an optimization perspective. We also propose to construct the conservative target labels which model the inlier and outlier noises using a wrapped unimodal-uniform mixture distribution. Unlike the one-hot setting, the conservative label makes the computation of Wasserstein distance more challenging. We systematically conclude the practical closed-form solution of Wasserstein distance for pose data with either one-hot or conservative target label. We evaluate our method on head, body, vehicle and 3D object pose benchmarks with exhaustive ablation studies. The Wasserstein loss obtaining superior performance over the current methods, especially using convex mapping function for ground metric, conservative label, and closed-form solution.
BibTeX
@misc{liu2019conservative,
  author = {Xiaofeng Liu and Yang Zou and Tong Che and Ping Jia and Peng Ding and Jane You and B. V. K. Vijaya Kumar},
  title = {{Conservative Wasserstein Training for Pose Estimation}},
  year = {2019},
  eprint = {1911.00962},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.00962},
}
Figure