Object Tracking System for Video Recording based Qt and OpenCV

Zhiyong Tang, Kun Liu, Zhenji Yang, Zhongcai Pei, Zeyang Zhang2016 · DOI

Abstract

In this paper, an object tracking system was introduced for video recording based on Qt and OpenCV. The system can track any moving target in a dynamic background. The tracking algorithm was designed based on the TLD (Tracking-Learning-Detection) tracking algorithm with specific refactoring for performance. The architecture of the system is clear, stable, and features a visualized user interface. The system is suitable for military reconnaissance, monitoring systems, and sports event recording.

Introduction

Object tracking in dynamic environments—where both the target and the camera may be in motion—presents significant challenges for computer vision systems. Traditional trackers often suffer from drift during occlusions, computational slowdown as the target model grows, and latency that prevents effective real-time hardware control. To address these issues, researchers at Beihang University developed a robust tracking system that optimizes the classic TLD framework. By integrating local-area searching and a sample-scoring mechanism within a multi-threaded Qt and OpenCV environment, the system achieves the speed and stability necessary to drive a physical two-degree-of-freedom (2-DOF) camera platform.

System Architecture

To maximize CPU utilization and ensure a responsive user interface, the system is built on a three-thread model. This separation of concerns prevents the heavy computational load of image processing from lagging the UI or the hardware controller.

The multi-threaded system architecture showing the interaction between the PC software threads and the external camera hardware.
  • User Interface (UI) Thread: Manages the graphical display and user interactions.
  • Tracking Algorithm Thread: Executes the core OpenCV-based computer vision logic.
  • Controller Communication Thread: Handles real-time data transmission to the camera platform hardware.

The Refactored TLD Algorithm

The core tracking logic utilizes an improved version of the TLD framework, which decomposes the task into three parallel components: tracking, detection, and learning.

Flowchart of the tracking algorithm, detailing the parallel processing of the optical flow tracker and the multi-stage detector.

1. Optical Flow Tracker

The system employs a "forward-backward" tracking method. It initially tracks 100 points from the previous frame to the current one, then tracks them back to the original frame. Points that do not return to their starting position are discarded as noise, pruning the point cloud to the 25 most reliable points to reduce jitter.

2. Triple Detector

When the tracker fails due to occlusion or rapid movement, a three-stage detector scans the frame to re-acquire the target:

  • Variance Classifier: Quickly filters out patches with low texture.
  • Random Fern Classifier: Uses a forest of trees to classify patches based on simple features.
  • Nearest Neighbor Classifier: Compares the patch against a database of positive and negative samples.

3. Integrator and Learning

The Integrator combines the outputs of the tracker and detector. If the detector finds a high-confidence match, the system updates its model through the Learning process, allowing it to adapt to changes in the target's appearance over time.

Key Optimizations

To ensure long-term stability and real-time performance, two critical improvements were implemented:

  • Local Search Optimization: Instead of a global frame scan, the detector prioritizes the area surrounding the last known target position. This significantly reduces the search space and increases processing speed.
  • Sample Scoring Mechanism: To prevent the sample database from growing indefinitely—which would eventually slow the system down—the authors implemented a scoring system. Low-scoring (less similar) samples are purged, keeping the memory footprint fixed and the execution speed consistent.

Conclusion and Future Work

The system successfully demonstrates the integration of high-level computer vision with physical hardware control. By refactoring the TLD algorithm for efficiency and utilizing a multi-threaded architecture, the researchers achieved a stable tracking loop capable of keeping a target centered on a motorized platform. Future work will focus on porting this framework to unmanned aerial vehicles (UAVs) and integrating augmented reality overlays for real-time reconnaissance data.

Summary updated

Abstract

In this paper, an object tracking system was introduced for video recording based Qt and OpenCV. The system can track any moving target in dynamic background. The tracking algorithm was designed based on TLD tracking algorithm and some refactoring was implemented. The architecture of the system is clear and the system runs stability. The UI of the system was visualized and convenient. The system can be widely used in military reconnaissance, monitoring system and sports event recording.
BibTeX
@misc{tang2016object,
  author = {Zhiyong Tang and Kun Liu and Zhenji Yang and Zhongcai Pei and Zeyang Zhang},
  title = {{Object Tracking System for Video Recording based Qt and OpenCV}},
  year = {2016},
  doi = {10.2991/icmeit-16.2016.38},
}
Figure