Multi-View, Multi-Target Tracking in Low-Altitude Scenes with UAV Involvement

Pengnian Wu, Yixuan Li, Zhihao Li, Xuqi Yang, Dong XueDrones 2025 · DOI

TL;DR

This paper introduces MVTL-UAV, a novel end-to-end multi-view, multi-target tracking system specifically designed for low-altitude UAV scenarios. By incorporating innovative loss reinforcement, coupled constraints, and temperature coefficient adjustments, MVTL-UAV significantly improves cross-view identity matching accuracy without sacrificing computational efficiency, making it practical for resource-constrained UAV platforms.

What Problem Does This Paper Address?

Tracking multiple objects from the perspective of unmanned aerial vehicles (UAVs) in low-altitude environments presents unique challenges. These include significant variations in target appearance due to scale changes, similar-looking targets, and frequent occlusions. Existing tracking methods struggle to effectively associate target identities across different viewpoints (e.g., from a UAV and handheld cameras), hindering robust tracking performance. This paper aims to bridge this gap by developing a system that can accurately maintain target identities even when they are partially occluded or visually similar, across multiple camera perspectives.

Methods

The proposed MVTL-UAV system builds upon existing end-to-end detection and tracking frameworks. Its novelty lies in three key modules:

  • Loss Reinforcement: This module enhances the detection loss by incorporating positive and negative reinforcements tailored to target scale. For larger targets, it prioritizes offset loss, while for smaller targets, it focuses on size loss. This ensures better detection accuracy across a range of object sizes.
  • Coupled Constraints: This module introduces self-attention reinforcement for re-identification loss, accentuating distinctive target features. It also imposes coupled constraints between detection and tracking losses. This ensures a consistent optimization direction during training, improving overall tracking robustness.
  • Coefficient Improvement: During inference, the system refines the temperature coefficient used in calculating the target identity-matching cost matrix. This refinement considers both the number of cross-view matches and the average target similarity, allowing for better discrimination among visually similar targets.

Datasets

The study utilizes the DIVOTrack dataset for training and evaluation. This dataset is specifically designed for low-altitude multi-view, multi-target tracking scenarios and includes data captured from a single UAV and two handheld cameras, covering 10 scenes.

Key Insights and Ideas

  • Scale-Aware Loss: Recognizing that different target sizes impact detection accuracy differently, the paper introduces a scale-aware loss function that dynamically adjusts focus between offset and size prediction errors. This is crucial for UAVs that often capture targets at varying distances.
  • Self-Attention for Distinctiveness: Employing self-attention mechanisms in the re-identification module helps to amplify the most discriminative features of a target, making it easier to distinguish between visually similar objects, a common problem in crowded scenes.
  • Inference-Time Adaptation: The temperature coefficient adjustment during inference is a clever way to improve performance without requiring additional training data or computational overhead during the learning phase. It allows the system to adapt to varying target densities and similarities on the fly.
  • Efficiency Matters: A significant contribution is achieving improved accuracy without compromising computational efficiency, which is paramount for deployment on resource-limited UAV platforms.

Key Results

The MVTL-UAV model achieved state-of-the-art performance on the DIVOTrack dataset. Specifically, it demonstrated:

  • A 2.19% improvement in Cross-View Matching Accuracy (CVMA).
  • A 1.95% improvement in Cross-View ID F1 metric (CVIDF1) compared to the previous state-of-the-art (CrossMOT).
  • The proposed method showed a 1.43-second reduction in total inference time compared to CrossMOT across five scenes, with only a marginal increase in memory footprint.
  • Qualitative results (visualizations) showed that MVTL-UAV successfully maintained identity associations for targets, even when occluded, outperforming the baseline in challenging scenarios.

Limitations and Future Work

The authors acknowledge that while their method shows modest improvements, future work will focus on:

  • Accelerating Inference: Further optimizing the inference process for seamless integration with cloud servers or ground control stations.
  • Onboard Processing: Implementing single-view visual algorithms directly on each drone to enable more efficient data transmission and processing.
  • Collaborative Systems: Integrating collaborative decision-making and control methods to create a comprehensive perception-decision-control system for multi-target tracking in low-altitude environments.
  • Exploring Diverse Scenarios: Expanding the method to handle a wider range of observation perspectives and more complex application scenarios.

Summary updated

Abstract

Cooperative visual tracking involving unmanned aerial vehicles (UAVs) in low-altitude environments is a dynamic and rapidly evolving domain. Existing models encounter challenges with targets, such as scale variation, appearance similarity, and frequent occlusions, which hinder the effective use of target information for cross-view identity association. To address these challenges, this study introduces a model for multi-view, multi-target tracking in low-altitude scenes involving UAVs (MVTL-UAV), an effective multi-target tracking model specifically designed for low-altitude scenarios involving UAVs. The proposed method is built upon existing end-to-end detection and tracking frameworks, introducing three innovative modules: loss reinforcement, coupled constraints, and coefficient improvement. Collectively, these advancements enhance the accuracy of cross-view target identity matching. Our method is trained using the DIVOTrack dataset, which comprises data collected from a single UAV and two handheld cameras. Empirical results indicate that our approach achieves a 2.19% improvement in cross-view matching accuracy (CVMA) and a 1.95% improvement in the cross-view ID F1 metric (CVIDF1) when compared to current state-of-the-art methodologies. Importantly, the model’s performance is improved without compromising computational efficiency, thereby enhancing its practical value in resource-constrained environments. As a result, our model demonstrates satisfactory performance in various low-altitude target tracking scenarios involving UAVs, establishing a new benchmark in this research area.
BibTeX
@inproceedings{wu2025multi,
  author = {Pengnian Wu and Yixuan Li and Zhihao Li and Xuqi Yang and Dong Xue},
  title = {{Multi-View, Multi-Target Tracking in Low-Altitude Scenes with UAV Involvement}},
  booktitle = {Drones},
  year = {2025},
  doi = {10.3390/drones9020138},
}
Figure