Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach

Shizhou Zhang, Wenlong Luo, De Cheng, Qingchun Yang, Lingyan Ran, Yinghui Xing, Yanning Zhang

Introduction: The Challenge of Cross-Platform Re-Identification

Person Re-Identification (ReID) is a critical task in computer vision, focused on matching individuals across non-overlapping camera views. While traditional ReID research has matured in ground-to-ground scenarios (e.g., matching pedestrians across sidewalk cameras), real-world security applications increasingly demand "cross-platform" capabilities. Specifically, matching a suspect identified by a ground-level camera with footage from an airborne Unmanned Aerial Vehicle (UAV) presents unique challenges.

Comparison of ground-level and UAV-captured pedestrian images at various altitudes, highlighting the drastic viewpoint and resolution shifts.

This cross-platform scenario introduces three primary difficulties:

  1. Drastic Viewpoint Changes: Ground cameras typically capture horizontal views, whereas UAVs provide high-angle, top-down, or oblique perspectives.
  2. Resolution Discrepancy: UAV footage resolution varies significantly based on flight altitude (ranging from 20m to 60m in this study).
  3. Occlusion and Pose Variation: Aerial views often suffer from self-obstruction (e.g., the head obscuring the body) and distinct lighting conditions compared to ground views.

VSLA-CLIP: A Parameter-Efficient Adaptation Approach

To address these challenges, the authors propose VSLA-CLIP (Video Set-Level-Adapter CLIP). Rather than training a model from scratch, this framework adapts the CLIP (Contrastive Language-Image Pre-training) foundation model, leveraging its robust pre-trained visual-semantic knowledge.

The two-stage VSLA-CLIP architecture, featuring visual-semantic alignment and the Video Set-Level-Adapter (VSLA) modules.

The framework introduces several key technical innovations:

  • Visual-Semantic Alignment: Since ReID identities lack natural language descriptions, the authors employ "learnable description tokens." These virtual words allow the model to associate specific IDs with visual features within CLIP’s shared image-text space.
  • Video Set-Level-Adapter (VSLA): This module optimizes feature extraction across video sequences. It includes an Intra-Frame Adapter (IFA) to refine single-image features and a Cross-Frame Attention Adapter (CFAA) that aggregates information across different frames in a tracklet, helping to overcome temporary occlusions.
  • Platform-Bridge Prompt (PBP): These platform-specific learnable prompts guide the model to distinguish between ground and aerial footage, encouraging it to focus on platform-invariant features like clothing color rather than platform-specific noise.

The G2A-VReID Dataset

A major contribution of this work is the introduction of G2A-VReID, the first large-scale benchmark specifically designed for ground-to-aerial video-based ReID. The dataset comprises 185,907 images across 5,576 tracklets, representing 2,788 distinct individuals. Data was collected across nine diverse outdoor scenarios, including campuses and subways, using both stationary ground cameras and consumer-grade UAVs. Compared to existing datasets like MARS or LS-VID, G2A-VReID is unique in its focus on the cross-platform challenge.

Key Insights and Results

The authors treat video as a "disordered set" of frames rather than a strict temporal sequence. In aerial ReID, where views are frequently obstructed, identifying any clear frame is often more valuable than maintaining temporal order. Furthermore, the study demonstrates that foundation models like CLIP can be effectively adapted for extreme viewpoint changes with high parameter efficiency. VSLA-CLIP tunes only approximately 14.5 million parameters—a fraction of the total model size—while achieving superior performance.

On the G2A-VReID benchmark, VSLA-CLIP achieved a 79.70% mAP and 72.55% Rank-1 accuracy, outperforming specialized video ReID methods. The model also showed strong generalization, setting new state-of-the-art records on traditional datasets such as MARS (88.60% mAP) and LS-VID (85.20% mAP).

Conclusion and Future Work

This research bridges a significant gap in multi-platform surveillance. While the G2A-VReID dataset and VSLA-CLIP framework provide a robust foundation, the authors acknowledge ongoing challenges, including ethical considerations regarding pedestrian privacy in public data collection. Future work may explore even more diverse platforms, such as satellite imagery, and further optimize the real-time efficiency of the adapters for deployment in live security environments.

Check out the code and dataset: https://github.com/FHR-L/VSLA-CLIP

Summary updated

BibTeX
@inproceedings{zhang2024cross,
  author = {Shizhou Zhang and Wenlong Luo and De Cheng and Qingchun Yang and Lingyan Ran and Yinghui Xing and Yanning Zhang},
  title = {{Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach}},
  booktitle = {European Conference on Computer Vision},
  year = {2024},
  doi = {10.1007/978-3-031-73383-3\_16},
}
Figure