Wasserstein Loss based Deep Object Detection

Yuzhuo Han, Xiaofeng Liu, Zhenfei Sheng, Yutao Ren, Xu Han, Jane You, Risheng Liu, Zhongxuan LuoWorkshops @ CVPR 2020 · OpenReview

Introduction

In standard deep learning-based object detection, classification is typically treated as a "one-hot" problem optimized via Cross-Entropy (CE) or Binary Cross-Entropy (BCE) loss. These loss functions are fundamentally "attribute-blind": they penalize all incorrect predictions equally, regardless of the semantic similarity between the predicted and ground-truth classes.

For safety-critical applications like autonomous driving, however, not all mistakes are equal. If a detector misidentifies a Bike as a Motorcycle, the resulting driving logic might still be relatively safe due to shared size and movement patterns. If that same Bike is misidentified as a Car, the path-planning algorithm might make a catastrophic error. Current models lack "attribute perception"—the ability to understand the severity of different types of misclassifications.

Figure 1: Motivation for Wasserstein Loss. Standard losses (CE/BCE) treat the 'Motor' and 'Car' misclassifications as equally wrong, whereas the proposed method distinguishes between them based on class similarity.

Methodology: Wasserstein Loss for Object Detection (WLOD)

The authors propose a novel loss formulation based on the Wasserstein distance (also known as the Earth Mover's Distance) to incorporate a "ground metric" that defines similarity between classes.

1. The Ground Distance Matrix ($W$)

The core innovation is the introduction of a pre-defined matrix $W$ that encodes the "distance" or cost of confusing one class for another. For example, the distance between 'Bike' and 'Motorcycle' is set to be small, while the distance between 'Bike' and 'Car' is large. This matrix is constructed using prior knowledge, grouping objects into hierarchies (e.g., all vehicles in one group, pedestrians/riders in another).

Figure 4: Visualization of the Ground Distance Matrix $W$. The heatmap shows lower costs (blue) for confusing semantically similar classes and higher costs (yellow) for dissimilar ones.

2. Loss Formulation

The Wasserstein distance measures the minimum "cost" to transform the predicted probability distribution into the ground-truth distribution. While calculating the general Wasserstein distance can be computationally expensive, the authors prove that in the case of a one-hot ground truth label, the loss simplifies into a much faster soft-attention scheme.

The final multi-task loss function combines standard detection losses with the new Wasserstein term: $$L = L_{locat} + L_{conf} + L_{class} + \lambda L_{WD}$$ Where $L_{WD}$ is the Wasserstein loss and $\lambda$ is a hyperparameter (tested at 0.5 and 1.0) that controls the model's sensitivity to the severity of misclassifications.

Experimental Results

The model was evaluated on two major autonomous driving benchmarks: BDD100K (100k images in diverse conditions) and KITTI (a standard robotics dataset). The inclusion of Wasserstein loss led to consistent improvements across different architectures:

  • BDD100K: Using a YOLOv3 backbone, the mAP increased from 25.8 to 28.7.
  • KITTI: The mAP improved from 68.3% to 69.4% for YOLOv3 and from 72.8% to 74.7% for SSD.

Qualitative results showed that the WLOD model was better at distinguishing between overlapping classes (like a "Rider" on a "Bike") and was less likely to produce multiple conflicting labels for the same object (e.g., labeling a bus as both a "Bus" and a "Truck").

Conclusion

By incorporating a ground metric, the network is forced to learn features that distinguish between "similar" classes more finely, effectively guiding the deep network to learn essential attributes. While the current ground distance matrix $W$ relies on manual grouping, the approach demonstrates that "soft" attention to the entire probability distribution can significantly improve detection robustness in complex environments.

Summary updated

BibTeX
@inproceedings{han2020wasserstein,
  author = {Yuzhuo Han and Xiaofeng Liu and Zhenfei Sheng and Yutao Ren and Xu Han and Jane You and Risheng Liu and Zhongxuan Luo},
  title = {{Wasserstein Loss based Deep Object Detection}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=3u0QZcgVHb},
}
Figure