TL;DR
This paper introduces Diverse Representations Embedding (DRE), a novel lifelong person re-identification (LReID) framework that leverages transformers to learn rich, multi-faceted representations of individuals. DRE effectively balances preserving past knowledge with adapting to new data streams, outperforming existing methods on challenging datasets.
What Problem Does This Paper Address?
Person re-identification (ReID) is crucial for tasks like surveillance and multi-camera tracking, aiming to match individuals across different camera views. However, real-world ReID systems face a continuous stream of new data from different environments and tasks. This "lifelong learning" scenario presents a significant challenge: how can a model learn new information without forgetting what it already knows? Existing methods, often relying on CNNs, struggle to capture diverse representations of each person, limiting their performance, especially when dealing with limited data from previous tasks and domain shifts between tasks. This paper tackles this by proposing a transformer-based approach that explicitly learns multiple, complementary representations for each individual.
Methods
The core of the proposed DRE framework is a pure transformer backbone, a departure from the CNNs common in prior LReID work. This transformer is designed to generate diverse representations for each instance.
Here's a breakdown of the key components:
- Transformer Backbone: Instead of a standard transformer, DRE utilizes an "adjustment model" and a "learner model," both employing enhanced transformer structures. These models process input images by splitting them into patches, incorporating "maximum embedding" (ME) to focus on critical regions, and crucially, using multiple class tokens. These multiple class tokens generate overlapping representations that capture different facets of an instance.
- Adaptive Constraint Module (ACM): This module is designed to manage the multiple representations generated by the transformer. It implements "integration" (combining representations) and "push away" (minimizing overlap between auxiliary representations) operations. This ensures that the primary representation captures rich body information, while auxiliary representations maintain discriminative power.
- Knowledge Update (KU) Strategy: This strategy focuses on enabling the learner model to adapt to new information. It comprises:
- Adaptive Learning: Using cross-entropy and triplet losses to train the learner model on new instances.
- Logit-level Distillation (LLD): Minimizing the difference in output logits between the adjustment and learner models to ensure consistency.
- Slow Update: Employing an exponential moving average (EMA) to gradually update the adjustment model's parameters, preventing it from becoming too rigid and allowing for better adaptation.
- Knowledge Preservation (KP) Strategy: This strategy is crucial for preventing catastrophic forgetting of old knowledge. It includes:
- Representation-level Alignment (RLA): Using triplet loss and an L1 norm-based cosine similarity loss to ensure that the representations learned by the adjustment and learner models for old instances remain consistent.
- Logit-level Supervision (LLS): Using the logits from the adjustment model's primary embedding as a supervisory signal for the learner model to preserve old knowledge.
Datasets
The paper evaluates DRE on a comprehensive set of eleven Re-ID datasets.
- Seen Datasets (for training): Market, CUHK-SYSU, DukeMTMC, MSMT17_V2, and CUHK03. The authors randomly sample 500 identities from each for training.
- Unseen Datasets (for evaluation/generalization): VIPER, GRID, CUHK02, Occ_Duke, Occ_REID, and PRID2011. The authors specifically highlight the inclusion of occluded datasets (Occ_Duke, Occ_REID) to test robustness.
Two training orders are used:
- Training Order-1: Market→CUHK-SYSU→DukeMTMC→MSMT17_V2→CUHK03
- Training Order-2: DukeMTMC→MSMT17_V2→Market→CUHK-SYSU→CUHK03
Key Insights and Ideas
- Transformers are well-suited for diverse representations: The paper argues that the multi-head self-attention mechanism in transformers naturally lends itself to generating multiple, overlapping representations, which is beneficial for LReID.
- Balancing old and new knowledge is key: The core challenge of LReID is this trade-off. DRE's dual strategy of Knowledge Update (KU) and Knowledge Preservation (KP) directly addresses this.
- Instance-level diversity matters: By generating multiple representations for each person (primary and auxiliary), the model can capture richer and more discriminative features, leading to better matching.
- ACM is crucial for managing diverse representations: The Adaptive Constraint Module effectively integrates and refines these multiple representations, preventing redundancy and enhancing their utility.
- Transformer backbones offer advantages over CNNs for LReID: The paper suggests that transformers, with their inherent ability to capture long-range dependencies and generate multiple embeddings, are a more natural fit for the complex demands of lifelong ReID.
Key Results
- Significant performance gains: DRE consistently outperforms state-of-the-art LReID methods across various datasets and training orders.
- Average incremental gains: DRE achieves an average improvement of +6.6% in mAP and +6.5% in Rank-1 on seen datasets compared to other methods.
- Strong generalization: DRE demonstrates robust generalization to unseen datasets, particularly excelling on occluded datasets.
- Transformer advantage: The transformer-based backbone, combined with DRE's strategies, leads to substantial improvements over CNN-based methods like ResNet50. For instance, DRE shows an 8.6% increase in mAP and an 8.2% increase in Rank-1 over a ResNet50 baseline (ResKUP).
- Component effectiveness: Ablation studies show that while Logit-level Distillation (LLD) alone is insufficient, the combination of RLA, LLS, KU, and ACM leads to significant performance improvements.
Limitations and Future Work
- Complexity of multiple representations: While beneficial, managing multiple representations can increase computational complexity. The paper addresses this by setting the number of auxiliary representations to 2, striking a balance between performance and complexity.
- Future work: The paper suggests that further research could explore more advanced transformer architectures and investigate more sophisticated methods for knowledge interaction and adaptation in lifelong learning scenarios. The authors also mention that their code will be made available, facilitating further research in this area.
Summary updated