Learning Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification

Zhenyu Cui, Jiahuan Zhou, Xun Wang, Manyu Zhu, Yuxin Peng

Abstract

Lifelong Person Re-identification (L-ReID) involves training models on a continuous stream of data from diverse environments. A significant yet often overlooked hurdle in L-ReID is the requirement for "re-indexing"—re-processing all historical gallery images with every model update to ensure feature compatibility. This requirement is frequently untenable due to computational costs and strict data privacy regulations, such as GDPR, which may mandate the deletion of raw images. This paper formalizes the Re-indexing Free Lifelong Person Re-identification (RFL-ReID) task and proposes the Continual Compatible Representation ($C^2R$) framework. $C^2R$ enables the transformation of legacy features into a new, compatible feature space without access to original images. Through a combination of a Continual Compatible Transfer (CCT) network and balanced distillation strategies, the framework achieves state-of-the-art performance across multiple benchmarks, effectively mitigating both feature incompatibility and catastrophic forgetting.

The Challenge of Re-indexing and Privacy

Person Re-identification (ReID) is a critical task in computer vision, aimed at matching individuals across non-overlapping camera views. In real-world deployments, systems must adapt to new environments over time, a challenge addressed by Lifelong ReID (L-ReID). However, traditional L-ReID assumes that historical raw data is always available for re-indexing. When privacy constraints or storage limits prevent the retention of raw images, standard L-ReID models fail because new models produce features that are mathematically incompatible with those stored in the old gallery.

Comparison between traditional Lifelong ReID and the proposed Re-indexing Free Lifelong ReID (RFL-ReID) task.

As shown in Figure 2, traditional methods (a) rely on re-indexing to maintain accuracy, whereas RFL-ReID (b) must handle "Old Gallery Features" directly. Without a compatibility mechanism, the "New Model" produces "Query Features" that cannot be accurately compared to the "Old Gallery Features," leading to a significant drop in retrieval performance due to the "Data Privacy" wall.

The $C^2R$ Framework

To bridge this gap, the $C^2R$ framework introduces three core components designed to ensure feature compatibility and prevent knowledge loss without ever needing the original raw pixels.

The architecture of the $C^2R$ framework, illustrating the training, transfer, and testing stages.
  1. Continual Compatible Transfer (CCT): This module acts as a bridge between feature spaces. It consists of a Knowledge Capturing Module (KCM), which learns the domain shift between old and new environments using knowledge prototypes, and a Forward Mapping Module (FMM), which preserves the discriminative identity information from the old features. An adaptive factor balances these streams to output a "transferred" feature that is compatible with the new model.
  2. Balanced Compatible Distillation (BCD): BCD aligns the new model's output with the transferred features. It employs both point-to-point alignment (L2 loss) and relationship distillation, ensuring that the relative distances (similarities) between different identities remain consistent across model versions.
  3. Balanced Anti-forgetting Distillation (BAD): To combat catastrophic forgetting, BAD utilizes the classification logic of the previous model. It constrains the new model to maintain the decision boundaries learned on earlier datasets, ensuring that the ability to recognize "old" people is not lost while learning "new" ones.

Experimental Evaluation

The framework was rigorously tested on five major ReID benchmarks: Market-1501, CUHK-SYSU, DukeMTMC-ReID, MSMT17-V2, and CUHK03. The researchers simulated lifelong learning by presenting these datasets in various sequential orders.

In the "Order-1" sequence, $C^2R$ achieved a mean Average Precision (mAP) of 39.5%, surpassing the previous state-of-the-art (PatchKD) by 3.3%. Notably, the method demonstrated exceptional stability, with an Average Forgetting (AF) score of only 13.9% mAP, significantly lower than traditional lifelong learning methods like LwF. These results indicate that $C^2R$ not only solves the compatibility problem but also serves as a robust regularizer for lifelong learning in general.

Conclusion

The $C^2R$ framework represents a significant step toward deploying privacy-compliant, scalable ReID systems. By treating feature compatibility as a transfer learning problem rather than a data-reprocessing problem, the authors provide a viable path for lifelong learning in environments where raw data retention is prohibited. While the model currently faces challenges in extremely data-scarce or high-noise scenarios, its ability to maintain backward compatibility without re-indexing sets a new standard for practical ReID applications.

Summary updated

Abstract

Lifelong Person Re-identification (L-ReID) aims to learn from sequentially collected data to match a person across different scenes. Once an L-ReID model is updated using new data, all historical images in the gallery are required to be re-calculated to obtain new features for testing, known as “re-indexing”. However, it is infeasible when raw images in the gallery are unavailable due to data privacy concerns, resulting in incompatible retrieval between the query and the gallery features calculated by different models, which causes significant performance degradation. In this pa-per, we focus on a new task called Re-indexing Free Life- long Person Re-identification (RFL-ReID), which requires achieving effective L-ReID without re-indexing raw images in the gallery. To this end, we propose a Continual Com-patible Representation (C 2 R) method, which facilitates the query feature calculated by the continuously updated model to effectively retrieve the gallery feature calculated by the old model in a compatible manner. Specifically, we design a Continual Compatible Transfer (CCT) network to con-tinuously transfer and consolidate the old gallery feature into the new feature space. Besides, a Balanced Compati-ble Distillation module is introduced to achieve compatibil-ity by aligning the transferred feature space with the new feature space. Finally, a Balanced Anti-forgetting Distillation module is proposed to eliminate the accumulated for- getting of old knowledge during the continual compatible transfer. Extensive experiments on several benchmark L- ReID datasets demonstrate the effectiveness of our method against state-of-the-art methods for both RFL-ReID and L- ReID tasks. The source code of this paper is available at https://github.com/PKU-ICST-MIPL/C2R_CVPR2024.
BibTeX
@inproceedings{cui2024learning,
  author = {Zhenyu Cui and Jiahuan Zhou and Xun Wang and Manyu Zhu and Yuxin Peng},
  title = {{Learning Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01572},
}
Figure