Rethinking Person Re-Identification with Confidence

George Adaimi, Sven Kreiss, Alexandre AlahiarXiv 2019 · OpenReview

Abstract

Transportation systems often rely on understanding the flow of vehicles or pedestrians. From traffic monitoring at the city scale to commuters in train terminals, recent progress in sensing technology makes it possible to use cameras to better track moving agents (e.g., vehicles and pedestrians). Whether the cameras are mounted on drones, vehicles, or fixed in the built environment, they inevitably remain scattered. We need to develop the technology to re-identify the same agents across images captured from non-overlapping fields-of-view, referred to as the visual re-identification (Re-ID) task. State-of-the-art methods learn a neural network-based representation trained with the cross-entropy loss function. We argue that such a loss function is not suited for the visual Re-ID task and propose to model confidence in the representation learning framework. We show the impact of our confidence-based learning framework with three methods: label smoothing, confidence penalty, and deep variational information bottleneck. Our contribution is generic to any agent of interest, i.e., vehicles or pedestrians, and outperforms highly specialized state-of-the-art methods across 5 datasets. The source code and models are shared towards an open science mission.

Introduction: The Challenge of Visual Re-ID

Visual Re-identification (Re-ID) is the task of identifying the same "agent"—be it a person or a vehicle—across different, non-overlapping camera views. This is a critical component for intelligent transportation systems, city-scale traffic monitoring, and pedestrian tracking in complex environments. As agents move through a city, they are captured by a variety of sensors, from fixed street cameras to mobile drones.

Overview of the visual Re-ID task across a city environment using drones and fixed cameras.

The Pitfalls of Overconfidence

Standard Re-ID models typically treat the task as a classification problem, optimized using cross-entropy loss. This approach forces the model toward a "one-hot" target distribution, demanding 100% confidence in the correct ID and 0% in all others. However, in real-world scenarios, different individuals or vehicles often appear nearly identical due to low resolution, similar clothing, or common car models.

Example of a pedestrian crop from a Re-ID dataset, showing the limited visual information available for identification.

When forced to be absolutely certain, models tend to overfit on "destructive variations"—transient features like specific shadows, background noise, or poses—rather than learning robust, generalizable representations. This results in brittle models that struggle to generalize to new, unseen images. By forcing a distinction between visually identical individuals, the model essentially "hallucinates" differences based on noise.

Proposed Methods: Inducing Doubt

To address this, we propose a framework that regularizes standard cross-entropy loss by "inducing doubt" through confidence-based learning. By penalizing overconfidence, the model is encouraged to learn more robust features. We explore three specific implementations:

  1. Label Smoothing (LS): Instead of a hard binary target, LS assigns a slightly lower probability (e.g., 0.9) to the ground truth and distributes the remainder across all other classes.
  2. Confidence Penalty (CP): This method penalizes low-entropy output distributions, preventing the model from concentrating all probability mass on a single class. Unlike LS, CP is adaptive to the model's current predictions.
  3. Deep Variational Information Bottleneck (VIB): VIB acts on the internal latent representation, forcing the model to compress the input and discard irrelevant noise while retaining only essential identification features.

Experimental Evaluation

We evaluated our framework across five diverse datasets, covering both person and vehicle Re-ID:

  • Person Re-ID: Market-1501, MSMT17, and DukeMTMC-reID.
  • Vehicle Re-ID: VERI-Wild and a new aerial dataset, EPFL Roundabout.
  • Ego-centric Re-ID: nuScenes-ReID, testing pedestrian identification from the perspective of moving autonomous vehicles.

Key Results

Our confidence-based framework consistently outperformed standard baselines and highly specialized state-of-the-art (SOTA) methods:

  • Performance Gains: We observed a 6–7% boost in Mean Average Precision (mAP) across all datasets compared to standard baselines.
  • SOTA Performance: On the Market-1501 dataset, our approach achieved 90.1% mAP when combined with standard re-ranking, outperforming much more complex architectures.
  • Aerial Surveillance: On the EPFL Roundabout dataset, Label Smoothing improved the baseline mAP by 14%, demonstrating significant utility in drone-based monitoring scenarios.
  • Efficiency: The VIB method achieved competitive results with a reduced representation dimension (1024 vs. 2048), highlighting its ability to learn compact, informative features.

Discussion and Insights

The primary advantage of this approach lies in its simplicity. While many SOTA methods rely on complex additions like pose estimation or semantic segmentation, we show that refining the loss function can yield superior results. By accepting that different IDs can look similar, the model stops chasing noise and focuses on robust features.

Interestingly, while LS and CP are "plug-and-play," VIB proved particularly effective for vehicle Re-ID, where "near-identical" appearances (same make and model) are frequent. However, ablation studies showed that combining these methods often leads to diminishing returns as they may over-restrict the model's learning capacity.

Conclusion

Modeling confidence in representation learning provides a generic and powerful tool for visual Re-ID. By intentionally inducing doubt during training, we create models that are more robust, less prone to overfitting, and capable of outperforming complex, specialized architectures across both human and vehicle domains.

Summary updated

Abstract

Transportation systems often rely on understanding the flow of vehicles or pedestrian. From traffic monitoring at the city scale, to commuters in train terminals, recent progress in sensing technology make it possible to use cameras to better understand the demand, i.e., better track moving agents (e.g., vehicles and pedestrians). Whether the cameras are mounted on drones, vehicles, or fixed in the built environments, they inevitably remain scatter. We need to develop the technology to re-identify the same agents across images captured from non-overlapping field-of-views, referred to as the visual re-identification task. State-of-the-art methods learn a neural network based representation trained with the cross-entropy loss function. We argue that such loss function is not suited for the visual re-identification task hence propose to model confidence in the representation learning framework. We show the impact of our confidence-based learning framework with three methods: label smoothing, confidence penalty, and deep variational information bottleneck. They all show a boost in performance validating our claim. Our contribution is generic to any agent of interest, i.e., vehicles or pedestrians, and outperform highly specialized state-of-the-art methods across 5 datasets. The source code and models are shared towards an open science mission.
BibTeX
@misc{adaimi2019rethinking,
  author = {George Adaimi and Sven Kreiss and Alexandre Alahi},
  title = {{Rethinking Person Re-Identification with Confidence}},
  year = {2019},
  eprint = {1906.04692},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.04692},
}
Figure