Multichannel-Kernel Canonical Correlation Analysis for Cross-View Person Reidentification

Giuseppe Lisanti, Svebor Karaman, Iacopo Masi

TL;DR

This paper tackles the challenge of identifying the same person across different camera views, even when their appearance changes drastically due to pose, lighting, or viewpoint. They propose a novel "Multi-Channel, Multi-Kernel Canonical Correlation Analysis" (MCK-CCA) method that leverages multiple feature representations and multiple kernel functions to learn robust cross-view correspondences, achieving state-of-the-art results on several benchmarks.

What Problem Does This Paper Address?

Person re-identification (re-ID) is a critical task in video surveillance, aiming to track individuals across a network of cameras. A major hurdle is the significant variation in a person's appearance between different camera views. Factors like changes in pose, illumination, camera angle, and even clothing can make it difficult to match the same person. Existing methods often struggle with these extreme appearance changes, especially when cameras have limited or no overlapping fields of view. This paper specifically addresses this "cross-view appearance variability" challenge.

Methods

The core of the proposed method, MCK-CCA, lies in its multi-faceted approach to learning robust feature representations for re-ID.

  1. Multi-Channel Feature Representation: Instead of relying on a single type of feature, the authors extract a diverse set of features from the person's image. These include color features (HS, RGB, Lab) and texture features (HOG, LBP). Crucially, these features are not just extracted from the whole image but also from spatially segmented regions (upper, middle, lower body parts). This allows the model to capture appearance variations at different body parts.
  2. Multi-Kernel Canonical Correlation Analysis (KCCA): For each feature channel, KCCA is employed to learn projection spaces that maximize the correlation between features from different camera views. The novelty here is the use of multiple kernels (linear, RBF, $\chi^2$, and exponential $\chi^2$) for each feature channel. This allows KCCA to learn different types of correlations and capture a wider range of appearance similarities.
  3. Iterative Logistic Regression for Fusion: After obtaining multiple KCCA projections for each feature channel and kernel combination, a late fusion strategy is used. An iterative logistic regression model is trained to learn weights for each of these projection spaces. This process adaptively selects and emphasizes the most discriminative feature-kernel combinations, effectively "filtering out" less useful ones and giving more importance to those that contribute most to accurate matching.

Datasets

The authors evaluate their method on four standard, publicly available datasets:

  • VIPER: Known for illumination variations and pose changes.
  • PRID: Considered more challenging due to distractors and strong illumination changes.
  • PRID 450s: Similar to PRID but without distractors.
  • CUHK01: Features high-resolution images with frontal and profile views, and low illumination variations.

They also mention using a specific split for VIPER provided by another research group.

Key Insights and Ideas

  • Diversity is Key: The paper strongly advocates for using a rich set of diverse features (color and texture, across different body parts) to capture the complex variations in person appearance.
  • Multiple Perspectives (Kernels): Employing multiple kernel functions within KCCA is crucial. Different kernels capture different types of relationships between features, and using them in combination provides a more comprehensive view of appearance similarity.
  • Adaptive Fusion: The iterative logistic regression for weighting projection spaces is a clever way to automatically learn which feature-kernel combinations are most effective for a given dataset, rather than relying on a fixed fusion strategy. This "late fusion" approach allows for more fine-grained control over how different representations contribute to the final decision.
  • Texture Noise Sensitivity: On datasets with lower image resolution (like VIPER), texture features (HOG, LBP) can be more susceptible to noise and are sometimes filtered out by the iterative logistic regression, highlighting the importance of dataset characteristics.

Key Results

The MCK-CCA method demonstrates strong performance across the evaluated datasets.

  • State-of-the-Art Performance: The paper reports achieving state-of-the-art results on the PRID and CUHK01 datasets.
  • Competitive Performance: It also achieves comparable performance on VIPER and PRID 450s, often outperforming other methods, especially when compared to ensemble methods or those relying on single representations.
  • Improvement over Metric Learning: When compared against metric learning techniques like LMNN and LDML, MCK-CCA (even with just a linear kernel) shows significant improvements, and further gains are observed when using all kernels.
  • Effectiveness of Iterative Filtering: The iterative logistic regression filtering mechanism is shown to be effective in selecting the most relevant kernel-channel combinations, leading to better performance.

Limitations and Future Work

  • Dataset Specificity: While the method performs well, the filtering analysis (Figure 4) suggests that the optimal kernel-channel combinations can vary significantly across datasets. This implies that a fully automated, dataset-agnostic selection might still be a challenge.
  • Texture Feature Sensitivity: The paper notes that texture features can be noisy on low-resolution images, leading to their filtering. This suggests that future work could explore more robust texture descriptors or methods to denoise them.
  • Future Directions: The authors suggest that incorporating metric learning techniques directly into their approach could be an interesting avenue for future research. They also mention that their method's success in learning multiple representations and combining them could be extended to other computer vision tasks.

Summary updated

Abstract

In this article, we introduce a method to overcome one of the main challenges of person reidentification in multicamera networks, namely cross-view appearance changes. The proposed solution addresses the extreme variability of person appearance in different camera views by exploiting multiple feature representations. For each feature, kernel canonical correlation analysis with different kernels is employed to learn several projection spaces in which the appearance correlation between samples of the same person observed from different cameras is maximized. An iterative logistic regression is finally used to select and weight the contributions of each projection and perform the matching between the two views. Experimental evaluation shows that the proposed solution obtains comparable performance on the VIPeR and PRID 450s datasets and improves on the PRID and CUHK01 datasets with respect to the state of the art.
BibTeX
@article{lisanti2017multichannel,
  author = {Giuseppe Lisanti and Svebor Karaman and Iacopo Masi},
  title = {{Multichannel-Kernel Canonical Correlation Analysis for Cross-View Person Reidentification}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2017},
  doi = {10.1145/3038916},
}
Figure