Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System

Xiaolong Zhou, Sixian Chan, Chenhao Qiu, Xiaodan Jiang, Tinglong Tang

TL;DR

This paper introduces a novel multi-object tracking (MOT) system that enhances robustness in complex scenarios by integrating a Convolutional Block Attention Module (CBAM) for better feature extraction and an occlusion-sensing module to prevent appearance feature pollution. The proposed method achieves competitive performance on standard datasets, demonstrating improved accuracy and data association capabilities, particularly in challenging occlusion situations.

What Problem Does This Paper Address?

Multi-object tracking (MOT) is crucial for applications like behavior analysis in healthcare, surveillance, and autonomous driving. However, existing MOT methods often struggle with accuracy and efficiency in real-world environments characterized by occlusions, lighting changes, and similar-looking objects. These challenges can lead to increased algorithm complexity, slower tracking speeds, and a higher rate of identity switches. Specifically, when objects are temporarily occluded, their appearance features can become corrupted, leading to incorrect associations in subsequent frames. This paper aims to address these limitations by improving the robustness and efficiency of MOT systems, especially in the presence of occlusions.

Methods

The proposed method builds upon the FairMOT framework, a one-step MOT algorithm that jointly performs detection and re-identification. The key innovations are:

  • Convolutional Block Attention Module (CBAM): This module is integrated into the network to adaptively refine feature maps. CBAM combines spatial and channel attention mechanisms. It first calculates attention weights for both spatial and channel dimensions from the feature map. These weights are then used to fuse the feature maps, allowing the model to focus on more informative regions and channels, thereby extracting more robust object representations.
  • Occlusion-Sensing Module: This module is designed to detect when an object is occluded. When an occlusion is detected, the system temporarily stops updating the appearance features of the occluded object. This prevents the appearance features from being polluted by partial or corrupted views, which is a common issue that degrades tracking performance. The module enhances the traditional IoU-based occlusion detection by also considering the object's center point.

The overall architecture uses a DLAseg backbone for feature extraction, followed by detection and recognition branches. CBAM is applied to the frame size predictor and object recognition predictor. The occlusion-sensing module is integrated into the data association part of the tracker.

Datasets

The paper utilizes the following datasets:

  • CrowdHuman Dataset: Used for training the model. This dataset focuses on pedestrian detection in crowded scenes.
  • MIX Dataset: A hybrid dataset comprising six datasets (Caltech Pedestrian, CityPersons, CUHK-SYSU, PRW, ETHZ, and MOT17). This was used for further training of the MOT task models.
  • MOT Series Dataset (MOT16, MOT17, MOT20): Used for validation and testing. These datasets are standard benchmarks for pedestrian tracking challenges, with MOT20 being the densest.

Key Insights and Ideas

  • Attention for Robust Features: The integration of CBAM highlights the power of attention mechanisms not just for classification but for enhancing the quality of features used in tracking. By adaptively weighting spatial and channel information, the model can better capture object characteristics even under challenging visual conditions.
  • Proactive Occlusion Handling: The occlusion-sensing module's strategy of freezing appearance feature updates for occluded objects is a crucial insight. Instead of trying to infer features from partial views, it prioritizes maintaining the integrity of known features, which directly combats appearance pollution and improves identity association.
  • Synergy of Components: The paper demonstrates that combining advanced feature extraction (CBAM) with intelligent occlusion handling significantly boosts overall tracking performance, suggesting that these are not independent problems but rather interconnected aspects of robust MOT.

Key Results

The proposed method achieved competitive results on standard MOT benchmarks:

  • MOT20 Dataset: The improved algorithm showed an increase of 1.7% in MOTA and 1.54% in IDF1 compared to the baseline FairMOT. This indicates improved tracking accuracy and better identity assignment.
  • MOT16 Dataset: The method achieved a MOTA of 74.8% and an IDF1 of 74.7%, outperforming several state-of-the-art methods. Notably, the number of ID switches (IDs↓) was significantly lower (819 compared to 1074 for FairMOT), indicating more stable identity tracking.
  • MOT17 Dataset: The proposed method achieved a MOTA of 73.2% and an IDF1 of 73.9%, demonstrating strong performance.
  • Ablation Study: An ablation study on occlusion sensing showed that the proposed occlusion-sensing module improved MOTA by 1.49% and IDF1 by 2.14% on the MOT20 dataset, confirming its effectiveness. Visualizations also showed that the OS module prevented ID switches in cases of occlusion.

Limitations and Future Work

  • Computational Cost: While the method aims for efficiency, the addition of attention mechanisms and occlusion sensing can increase computational overhead. The paper mentions that applying CBAM to low-channel feature maps is not effective, suggesting a need for careful placement of such modules.
  • Generalization to Other Object Types: The current work focuses primarily on pedestrian tracking. Future work could explore the applicability of this approach to tracking other types of objects in diverse environments.
  • Real-time Performance: While the paper aims for real-time performance, further optimization might be needed for deployment in highly demanding applications.

The authors suggest that their approach provides a solid foundation for future research in robust and efficient multi-object tracking, particularly in complex, real-world scenarios.

Summary updated

Abstract

Multi-object tracking (MOT) is a topic of great interest in the field of computer vision, which is essential in smart behavior-analysis systems for healthcare, such as human-flow monitoring, crime analysis, and behavior warnings. Most MOT methods achieve stability by combining object-detection and re-identification networks. However, MOT requires high efficiency and accuracy in complex environments with occlusions and interference. This often increases the algorithm’s complexity, affects the speed of tracking calculations, and reduces real-time performance. In this paper, we present an improved MOT method combining an attention mechanism and occlusion sensing as a solution. A convolutional block attention module (CBAM) calculates the weights of space and channel attention from the feature map. The attention weights are used to fuse the feature maps to extract adaptively robust object representations. An occlusion-sensing module detects an object’s occlusion, and the appearance characteristics of an occluded object are not updated. This can enhance the model’s ability to extract object features and improve appearance feature pollution caused by the short-term occlusion of an object. Experiments on public datasets demonstrate the competitive performance of the proposed method compared with the state-of-the-art MOT methods. The experimental results show that our method has powerful data association capability, e.g., 73.2% MOTA and 73.9% IDF1 on the MOT17 dataset.
BibTeX
@inproceedings{zhou2023multi,
  author = {Xiaolong Zhou and Sixian Chan and Chenhao Qiu and Xiaodan Jiang and Tinglong Tang},
  title = {{Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=R9LbqTCYEF},
}
Figure