Visible-Infrared Person Re-Identification (VI-ReID) is a critical technology for 24/7 surveillance, enabling the tracking of individuals across both visible and infrared spectrums. However, traditional research has largely operated in a "laboratory" setting, assuming centralized access to sensitive image data. This paper introduces L2RW, a new benchmark that shifts the focus toward real-world, privacy-conscious decentralized training, and proposes DPPT, a novel algorithm designed to overcome the inherent challenges of federated learning in this domain.
The Privacy Wall in Person Re-Identification
Standard VI-ReID models typically rely on a centralized training paradigm. In this setting, raw images from various cameras and administrative entities are aggregated onto a single server for model optimization. While effective for performance, this approach is increasingly untenable due to strict data privacy regulations like GDPR and the proprietary nature of surveillance data.
Transitioning to a decentralized (federated) setting, where data remains on local nodes, introduces three significant technical hurdles:
- Modality Incompleteness: Individual cameras may only capture one modality (e.g., only infrared), breaking standard dual-stream models that expect both.
- Identity Missing: A person might appear in only one camera's field of view, preventing other nodes from learning that identity signature.
- Domain Shift: Variations in lighting, angles, and backgrounds across different cameras hinder model generalization.
The L2RW Benchmark
To address these issues, the authors established the Laboratory to Real World (L2RW) benchmark. By re-organizing major datasets like SYSU-MM01, RegDB, and LLCM, the benchmark defines three levels of privacy:
- Camera Independence (CI): The strictest protocol, where no data is shared between individual cameras.
- Entity Independence (EI): Data is shared within a specific entity (e.g., a single building) but not between different entities.
- Entity Sharing (ES): The traditional centralized baseline for comparison.
Decentralized Privacy-Preserved Training (DPPT)
The proposed DPPT framework is engineered to navigate the constraints of the L2RW benchmark. Its architecture moves away from the traditional dual-stream approach in favor of a One-Stream Architecture. This allows the model to process any available modality at a specific node, effectively solving the modality incompleteness problem.
A core innovation of DPPT is the Memory Rectification Bank (MRB). In this system, each client maintains a local memory of feature centers. These centers are uploaded to a server and aggregated into a Global Memory. To ensure robustness, the MRB employs a "Top-K" filtering strategy, averaging only the most representative features to filter out noise from poor lighting or occlusions. This global memory serves as a shared reference, helping all clients align their local optimizations without ever exchanging raw, sensitive images. Additionally, Channel Augmentation (CA) is used to force the network to focus on structural features rather than color-specific ones, bridging the gap between RGB and IR images.
Key Results and Insights
The DPPT method demonstrates that privacy does not have to come at the cost of performance. In evaluations on the SYSU-MM01 dataset under the strict CI protocol, DPPT achieved a 51.27% Rank-1 accuracy, nearly doubling the performance of standard federated learning baselines like FedAvg (~27.5%).
Key takeaways include:
- Robustness through One-Stream: Abandoning dual-stream architectures makes the model significantly more resilient to "modality-poor" environments.
- Feature Anchoring: The success of the MRB proves that sharing mathematical feature centers is a viable alternative to sharing raw data for maintaining global identity consistency.
- Generalization: DPPT showed strong cross-domain performance, maintaining high accuracy even when tested on entirely unseen entities.
Future Directions
While DPPT marks a significant step forward, challenges remain. The communication overhead of frequent weight updates in federated systems can be high. Future research may explore more efficient optimization techniques and the integration of differential privacy to provide even stronger theoretical guarantees against data leakage.
Summary updated