Learning Disentangled Representation with Mutual Information Maximization for Real-Time UAV Tracking

Xucheng Wang, Xiangyang Yang, Hengzhou Ye, Shuiwang Li

TL;DR

This paper introduces DR-MIM, a novel approach for Unmanned Aerial Vehicle (UAV) tracking that leverages disentangled representation learning with mutual information maximization. By separating identity-related and identity-unrelated features, DR-MIM achieves state-of-the-art precision and efficiency, addressing critical limitations in UAV tracking.

What Problem Does This Paper Address?

Tracking objects with Unmanned Aerial Vehicles (UAVs) presents a unique set of challenges. Unlike traditional visual tracking, UAVs operate under severe constraints: limited computational resources, short battery life, and strict weight limits. These factors make it difficult to deploy computationally intensive deep learning (DL) models. While traditional Discriminative Correlation Filter (DCF) trackers are efficient, they often lack the precision needed for complex scenarios like motion blur, occlusion, and extreme viewing angles. Recent attempts to create lightweight DL trackers through model compression have shown promise in efficiency but have struggled to maintain high tracking accuracy. This paper aims to bridge this gap by developing a DL-based tracker that is both highly efficient and precise, specifically for UAV applications.

Methods

The core innovation of this paper is the DR-MIM (Disentangled Representation - Mutual Information Maximization) tracker. It builds upon a Siamese network architecture, similar to P-SiamFC++, but introduces a novel Disentangled Representation (DR) module. This module, implemented within the network's "neck," uses two encoders to split the extracted features into two distinct components:

  • Identity-related features: These capture information specific to the target's identity.
  • Identity-unrelated features: These capture background or other contextual information that is not essential for identifying the target.

The key idea is to maximize the mutual information (MI) between the input features and these disentangled representations. This is achieved through a combination of global and local MI objectives. The global MI encourages the disentangled representation to capture as much information as possible from the original features, while the local MI focuses on information shared across different spatial locations within the feature map.

Crucially, for the downstream tasks of classification and regression (i.e., determining the target's location and class), only the identity-unrelated features are used. The authors hypothesize that by focusing on these features, the model becomes more robust to variations in the target's appearance and background, leading to improved generalization and precision. An additional identity similarity loss is employed to ensure that features from the same target across different frames remain consistent in the identity-related component.

Datasets

The proposed DR-MIM tracker was evaluated on four publicly available UAV tracking benchmarks:

  • UAV123@10fps
  • DTB70
  • UAVDT
  • VisDrone2018

Key Insights and Ideas

  • Disentangling for Robustness: The paper proposes that by explicitly separating identity-related and identity-unrelated features, and then using only the identity-unrelated features for tracking, the model can achieve better generalization and robustness to challenging conditions. This is a counter-intuitive but effective approach.
  • Mutual Information as a Guiding Principle: Leveraging mutual information maximization is shown to be a powerful tool for learning these disentangled representations, ensuring that the extracted features are informative and well-separated.
  • Efficiency Gains from Disentanglement: The disentangled representation not only improves precision but also leads to more compact features, contributing to increased tracking efficiency, which is paramount for UAV applications.
  • Pruning Ratio's Impact: The study highlights that the global pruning ratio (set to 0.5 in their default configuration) plays a significant role in balancing precision and efficiency, with 0.5 yielding optimal results on the DTB70 dataset.

Key Results

  • State-of-the-Art Performance: DR-MIM significantly outperforms existing state-of-the-art UAV tracking methods on the evaluated benchmarks.
  • Precision and Efficiency Boost: Compared to the baseline P-SiamFC++, DR-MIM achieves a reduced model size (64.5% of the original) and a significant speedup, particularly on CPU (nearly doubling the FPS from 46.1 to 92.1).
  • Competitive with DL Trackers: On the VisDrone2018 dataset, DR-MIM demonstrates superior GPU speed (8 times faster than the second-best tracker SparseTT) while maintaining competitive precision.
  • Qualitative Improvements: Visual results show DR-MIM's superior robustness in challenging scenarios like background clutter and pose changes, where other trackers fail.

Limitations and Future Work

  • Pruning Ratio Sensitivity: While the paper identifies an optimal pruning ratio (0.5) for the DTB70 dataset, the results in Table IV suggest that precision can vary with different ratios, indicating a potential area for further tuning or adaptive strategies.
  • Future Work: The authors believe their work opens avenues for developing more lightweight DL-based trackers for UAVs by further exploring disentangled representation learning. The general applicability of the InfoMax principle to visual tracking is also highlighted as a promising direction.

Summary updated

Abstract

Efficiency has been a critical problem in UAV tracking due to limitations in computation resources, battery capacity, and unmanned aerial vehicle maximum load. Although discriminative correlation filters (DCF)-based trackers prevail in this field for their favorable efficiency, some recently proposed lightweight deep learning (DL)-based trackers using model compression demonstrated quite remarkable CPU efficiency as well as precision. Unfortunately, the model compression methods utilized by these works, though simple, are still unable to achieve satisfying tracking precision with higher compression rates. This paper aims to exploit disentangled representation learning with mutual information maximization (DR-MIM) to further improve DL-based trackers' precision and efficiency for UAV tracking. The proposed disentangled representation separates the feature into an identity-related and an identity-unrelated features. Only the latter is used, which enhances the effectiveness of the feature representation for subsequent classification and regression tasks. Extensive experiments on four UAV benchmarks, including UAV123@10fps, DTB70, UAVDT and VisDrone2018, show that our DR-MIM tracker significantly outperforms state-of-the-art UAV tracking methods.
BibTeX
@inproceedings{wang2023learning,
  author = {Xucheng Wang and Xiangyang Yang and Hengzhou Ye and Shuiwang Li},
  title = {{Learning Disentangled Representation with Mutual Information Maximization for Real-Time UAV Tracking}},
  booktitle = {ICME},
  year = {2023},
  url = {https://openreview.net/forum?id=KhiAqJcQTL},
}
Figure