Abstract
Multi-Target Multi-Camera Tracking (MTMCT) is a critical component of modern smart city infrastructure, yet its development is hindered by the high cost of data annotation and the significant storage requirements of video datasets. This paper presents a semi-automatic annotation system designed to streamline the creation of MTMCT datasets. By integrating deep-learning-based trajectory generation with a metadata-driven recommendation engine, the system reduces human labor and achieves a 99% reduction in storage costs through dynamic visualization. Experimental results on the CityFlow benchmark demonstrate high precision and recall, offering a scalable solution for city-scale vehicle tracking.
1. Introduction
Multi-Target Multi-Camera Tracking (MTMCT) involves tracking specific vehicles across non-overlapping camera views in a city-wide network. While deep learning models have shown promise in this domain, they are frequently limited by the "domain gap"—the phenomenon where a model trained on data from one urban environment fails to generalize to another due to variations in camera angles, lighting, and road topology.
Addressing this gap requires the creation of custom datasets for new deployments. However, manual labeling is prohibitively expensive, often requiring thousands of hours of human effort. Furthermore, the storage of annotated video segments for review poses a significant logistical challenge. This paper proposes a semi-automatic pipeline that leverages automated trajectory generation and intelligent filtering to minimize human intervention and storage overhead.
2. Methodology
The proposed system utilizes a two-stage pipeline to generate high-quality annotations with minimal manual input.
2.1 Automatic Single-Camera Trajectory Generation
The first stage focuses on generating "tracklets" within individual camera views. The system employs YOLOv5 for robust object detection and DeepSort for temporal tracking. To optimize computational efficiency, the system implements a frame-sampling strategy. Rather than processing every frame, the system analyzes "key frames" at set intervals and utilizes linear interpolation to estimate vehicle positions in the intervening frames. This approach maintains high trajectory accuracy while significantly reducing inference time.
2.2 Human-in-the-Loop Cross-Camera Matching
The most labor-intensive part of MTMCT annotation is matching vehicles across different cameras. To simplify this, the system provides a Recommendation List to human labelers. This list is generated by applying "side information" to filter potential candidates:
- Temporal Constraints: The system calculates the travel time between cameras; a vehicle leaving Camera A can only appear in Camera B within a logically determined time window.
- Spatial Constraints: By utilizing the city's road topology, the system excludes cameras that are physically unreachable from a vehicle's last known location.
- Visual Context: Labelers are provided with background scenes and original video clips to help distinguish between vehicles with similar visual characteristics.
3. Engineering and Storage Optimization
To ensure the system is scalable for large teams, the backend is built using Linux Containers and the Advanced Message Queuing Protocol (AMQP). This architecture allows multiple annotators to work simultaneously without experiencing server-side latency.
One of the system's most significant innovations is its approach to data storage. Traditional annotation tools save reviewed clips as large video files (e.g., MP4 or GIF). Instead, this system stores only the trajectory features—the raw coordinates and timestamps. During the review process, the system uses the FFmpeg toolbox to render bounding boxes onto the original video stream on the fly. This strategy reduces storage requirements by approximately 99%, transforming gigabytes of data into mere megabytes.
4. Experimental Results
The system was validated using the CityFlow dataset, a benchmark for city-scale vehicle tracking. The automated trajectory generation achieved a Precision of 86.98% and a Recall of 86.81%.
In terms of storage efficiency, the dynamic visualization strategy proved highly effective. For a standard annotation task, storage requirements dropped from 3.8 GB (using traditional video saving) to just 7.1 MB (using trajectory features). Furthermore, testing indicated that a sampling interval of 2 frames provides the optimal balance between processing speed and the quality of the generated labels.
5. Conclusion and Future Work
The semi-automatic annotation system presented here addresses the primary bottlenecks in MTMCT dataset creation. By combining automated tracking with metadata-driven human-in-the-loop verification, the system achieves high accuracy while drastically reducing storage costs. Future work will focus on further automating the re-identification (ReID) process, potentially using the data generated by this system to train more sophisticated automated matchers that require even less human oversight.
Summary updated