Few-shot Image Recognition for UAV Sports Cinematography

Emmanouil Patsiouras, Anastasios Tefas, Ioannis PitasWorkshops @ CVPR 2020 · DOI

Introduction: The Challenge of Subclass Confusion

Deep learning models typically require vast amounts of labeled data to achieve high accuracy. In the context of Unmanned Aerial Vehicle (UAV) sports cinematography, this "data hunger" presents a significant hurdle. For instance, a drone tracking the leader of a cycling race may only have a few seconds of footage to learn the specific appearance of that athlete.

While Few-Shot Learning (FSL) aims to address this by enabling models to learn from just 1–5 examples, a specific problem known as subclass confusion often arises. When a model is trained to recognize a general "base class" (e.g., cyclists) and is then tasked with identifying a "novel subclass" (e.g., a leader in a pink jersey), standard FSL methods tend to over-specialize. This results in a bias where the model either misidentifies all cyclists as the leader or fails to distinguish the leader from the pack.

Cluster Few-Shot (CFS) Methodology

To overcome this, the authors propose Cluster Few-Shot (CFS) classification. This technique builds upon existing FSL frameworks that utilize cosine similarity and attention-based weight generators. The core innovation lies in "shattering" the base category during training to prevent over-specialization.

  1. Base Class Partitioning: Instead of treating "cyclist" as a monolithic category, the training images are split into $C$ distinct clusters using k-means clustering based on visual features.
  2. Multi-Class Training: These clusters are treated as separate base classes. This forces the neural network to learn fine-grained visual details to distinguish between different "types" of cyclists (e.g., different poses, lighting, or equipment).
  3. Inference and Mapping: When a novel subclass is introduced, the model leverages these diverse learned features. During testing, any image classified into one of the original cyclist clusters is mapped back to the general "cyclist" label, maintaining the integrity of the base class.

Dataset and Experimental Setup

The model was evaluated using a hybrid approach. Initial feature extraction training was performed on the Mini-ImageNet benchmark. This was followed by testing on a custom UAV Bicycle Race Dataset featuring aerial footage.

Visual examples of the general 'Cyclist' base class (a) and the specific 'Maglia Rosa' novel subclass (b).

The dataset defines "Cyclist" as the base class and "Maglia Rosa" (the race leader in a pink jersey) as the novel subclass. The goal is to identify the leader without losing the ability to recognize other riders.

Key Results and Discussion

Experimental results demonstrate that standard attention-based models suffer a significant drop in base class accuracy as they learn the novel subclass. In some cases, accuracy for the general "cyclist" class plummeted to nearly 40% as the model became biased toward the pink jersey.

In contrast, the CFS approach provides a robust regularization effect. By using 10 clusters, the model achieved a balanced performance.

Accuracy results for the CFS method with 10 clusters, showing stable performance for both base (BasicAcc) and novel (NovelAcc) classes.

As shown in the results, while the accuracy for the "Maglia Rosa" remained high (approx. 82-87%), the accuracy for the general "cyclist" class was vastly improved, staying around 85%. This indicates that forcing the model to distinguish between clusters of the same category develops a more "high-resolution" feature space capable of handling subclass nuances.

Conclusion and Future Work

The Cluster Few-Shot method effectively mitigates subclass confusion in UAV-based sports tracking. By partitioning base classes into clusters, the model maintains a balance between general recognition and specific identification.

Future research could explore automated methods for determining the optimal number of clusters and testing the framework in more visually diverse sports environments. Additionally, integrating this static image recognition into real-time video tracking loops remains a key objective for practical deployment in drone cinematography.

Summary updated

BibTeX
@inproceedings{patsiouras2020few,
  author = {Emmanouil Patsiouras and Anastasios Tefas and Ioannis Pitas},
  title = {{Few-shot Image Recognition for UAV Sports Cinematography}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00127},
}
Figure