Robust Individual-Cell/Object Tracking via PCANet Deep Network in Biomedicine and Computer Vision

Bineng Zhong, Shengnan Pan, Cheng Wang, Tian Wang, Ji‐Xiang Du, Duansheng Chen, Liujuan CaoBioMed 2016 · DOI

TL;DR

This paper introduces a novel deep learning-based tracking method, PCANet, that learns discriminative appearance models for individual cells or objects without requiring extensive pretraining. By combining unsupervised feature learning with a supervised classifier and a robust sample update strategy, the method achieves state-of-the-art performance on challenging biomedical and computer vision tracking benchmarks.

What Problem Does This Paper Address?

Tracking individual cells or objects over time is crucial for applications like understanding drug treatment effects on cancer cells and in general computer vision tasks. A major challenge in this field is the significant appearance variation that objects undergo due to both intrinsic factors (like pose, scale, and deformation) and extrinsic factors (like illumination changes and occlusions). These variations can easily cause trackers to "drift" away from the target. Furthermore, many existing cell tracking methods require substantial manual intervention, making them inefficient. The paper aims to develop an automatic and robust tracker that can handle these appearance changes effectively without relying on large, pre-trained models.

Methods

The proposed method, termed PCANet-based tracking, leverages a deep learning architecture inspired by PCANet. Here's a breakdown of the key components:

  • Unsupervised Feature Learning (PCANet): In the initial frame, an unsupervised approach is used to learn abstract features of the target. This involves using Principal Component Analysis (PCA) filters, inspired by deep learning architectures. These learned PCA eigenvectors act as filters. A novel representation is then built using a PCA-based filter bank layer, a nonlinear layer, and a patch-based pooling layer. This process effectively extracts hierarchical features from image patches.
  • Supervised Discriminative Appearance Model: Based on the features learned by PCANet, a simple neural network with one hidden layer is trained in a supervised manner. This network constructs a discriminative model that can distinguish the target from its background.
  • Particle Filtering Framework: The tracking process is embedded within a particle filtering framework. This involves predicting particle states using a motion model and then updating their weights based on the observation model (the PCANet-based appearance model). The particle with the highest weight indicates the most likely object state.
  • Sample Update Scheme: To combat tracker drifting, a simple yet effective sample update scheme is employed. This scheme ensures that the tracker maintains a set of representative and diverse samples of the target, allowing it to adapt to appearance changes and recover from temporary occlusions.

Datasets

The paper evaluates its method on two standard benchmarks:

  • Mitocheck Cell Dataset: This dataset consists of time-lapse microscopic image sequences with high cell density, significant intensity variations, and illumination changes, making it challenging for cell tracking.
  • Online Tracking Benchmark (OTB): This is a widely used benchmark for general object tracking, comprising 50 video sequences annotated with 11 attributes that represent various tracking challenges (e.g., deformation, fast motion, occlusion).

Key Insights and Ideas

  • Learning without Large-Scale Pretraining: A significant contribution is the ability to learn effective features directly from the target's initial appearance, bypassing the need for massive pre-trained models. This makes the tracker more adaptable to specific objects or cell types.
  • Combining Global and Local Information: The PCANet architecture, with its cascaded PCA, binarization, and block histograms, allows for the extraction of both global structural and local detail information, leading to a more robust representation.
  • Hierarchical Feature Learning: The layered structure of PCANet enables the learning of increasingly abstract and discriminative features, which is crucial for handling complex appearance variations.
  • Effective Sample Management: The simple yet effective sample update scheme is key to preventing tracker drift, a common failure mode in many tracking algorithms.

Key Results

The proposed PCANet-based tracking method achieved state-of-the-art performance on both the Mitocheck cell dataset and the OTB benchmark.

  • OTB Benchmark:
    • In the One-Pass Evaluation (OPE) precision plot, the method achieved a precision score of 0.707, ranking first among the top 10 trackers.
  • In the OPE success plot, it achieved an Area Under Curve (AUC) score of 0.566, also ranking first.
  • The method demonstrated robust performance across various challenging attributes within the OTB dataset.
  • Mitocheck Cell Dataset: Qualitative results showed that the tracker could effectively maintain tracking of individual cells, even with significant appearance changes.
  • Computational Efficiency: The typical training time for the PCANet-based deep network was about 10 seconds (without GPUs), and the tracker processed each video frame in about one second.

Limitations and Future Work

  • Occlusion Handling: The authors acknowledge that the current PCANet-based appearance model update is simple and does not explicitly detect long-term occlusions. This can lead to performance degradation when a target is occluded for extended periods.
  • Future Directions: To address the occlusion limitation, the authors suggest incorporating more sophisticated occlusion detection and forgetting mechanisms into the tracker for more effective model updating. They also note that while increasing PCA layers in PCANet can improve performance, the gains diminish and computational cost increases.

Summary updated

Abstract

Tracking individual-cell/object over time is important in understanding drug treatment effects on cancer cells and video surveillance. A fundamental problem of individual-cell/object tracking is to simultaneously address the cell/object appearance variations caused by intrinsic and extrinsic factors. In this paper, inspired by the architecture of deep learning, we propose a robust feature learning method for constructing discriminative appearance models without large-scale pretraining. Specifically, in the initial frames, an unsupervised method is firstly used to learn the abstract feature of a target by exploiting both classic principal component analysis (PCA) algorithms with recent deep learning representation architectures. We use learned PCA eigenvectors as filters and develop a novel algorithm to represent a target by composing of a PCA-based filter bank layer, a nonlinear layer, and a patch-based pooling layer, respectively. Then, based on the feature representation, a neural network with one hidden layer is trained in a supervised mode to construct a discriminative appearance model. Finally, to alleviate the tracker drifting problem, a sample update scheme is carefully designed to keep track of the most representative and diverse samples during tracking. We test the proposed tracking method on two standard individual cell/object tracking benchmarks to show our tracker's state-of-the-art performance.
BibTeX
@inproceedings{zhong2016robust,
  author = {Bineng Zhong and Shengnan Pan and Cheng Wang and Tian Wang and Ji‐Xiang Du and Duansheng Chen and Liujuan Cao},
  title = {{Robust Individual-Cell/Object Tracking via PCANet Deep Network in Biomedicine and Computer Vision}},
  booktitle = {BioMed},
  year = {2016},
  doi = {10.1155/2016/8182416},
}
Figure