SVDNet for Pedestrian Retrieval

Yifan Sun, Liang Zheng, Weijian Deng, Shengjin Wang

Abstract

This paper proposes the SVDNet for retrieval problems, with focus on the application of person re-identification (reID). We view each weight vector within a fully connected (FC) layer in a convolutional neuron network (CNN) as a projection basis. It is observed that the weight vectors are usually highly correlated. This problem leads to correlations among entries of the FC descriptor, and compromises the retrieval performance based on the Euclidean distance. To address the problem, this paper proposes to optimize the deep representation learning process with Singular Vector Decomposition (SVD). Specifically, with the restraint and relaxation iteration (RRI) training scheme, we are able to iteratively integrate the orthogonality constraint in CNN training, yielding the so-called SVDNet. We conduct experiments on the Market-1501, CUHK03, and DukeMTMC-reID datasets, and show that RRI effectively reduces the correlation among the projection vectors, produces more discriminative FC descriptors, and significantly improves the re-ID accuracy.

Introduction

Person re-identification (re-ID) is a high-stakes retrieval task where a "query" image of a person must be matched against a large "gallery" of images from different camera views. While deep learning has revolutionized this field, standard Convolutional Neural Networks (CNNs) often suffer from a subtle mathematical inefficiency: correlation among learned features.

Training samples of ID1 illustrating the visual similarity and diversity challenges in person re-identification datasets.

In typical re-ID systems, the final layers of a CNN are used as feature descriptors, and retrieval is performed using Euclidean distance. However, the weight vectors in these final layers are often highly correlated because many individuals in the training set share similar visual traits (e.g., red clothing). These redundant, correlated entries dominate the distance calculation, making it harder to distinguish between different individuals who share common characteristics.

Methodology: The Eigenlayer and RRI

To address this, SVDNet transforms a standard fully connected (FC) layer into an Eigenlayer—a layer where the weight vectors are strictly orthogonal to each other. This decorrelation ensures that each entry in the feature descriptor provides unique, non-redundant information.

Geometric representation of weight vectors $w_1, w_2, w_3$ and the objective of achieving orthogonality to reduce feature redundancy.

Restraint and Relaxation Iteration (RRI)

Simply forcing weights to be orthogonal often degrades model accuracy. To solve this, the authors introduced the Restraint and Relaxation Iteration (RRI) training scheme:

  1. Decorrelation (The SVD Step): The weight matrix $W$ is decomposed via SVD ($W = USV^T$) and replaced with $US$. This mathematically ensures orthogonality while perfectly preserving the model's original discriminative power.
  2. Restraint: The Eigenlayer is frozen, and the rest of the network is fine-tuned to adapt to this new orthogonal basis, forcing earlier layers to learn more efficient features.
  3. Relaxation: The Eigenlayer is unfrozen, and the entire network is trained together to reach a better overall local minimum.

Experiments and Results

The authors evaluated SVDNet on three gold-standard benchmarks: Market-1501, CUHK03, and DukeMTMC-reID. The performance gains were substantial across various backbones:

  • Market-1501: Using CaffeNet, Rank-1 accuracy soared from 55.3% to 80.5%. With ResNet-50, it improved from 73.8% to 82.3%.
  • DukeMTMC-reID: SVDNet achieved a Rank-1 accuracy of 76.7%, significantly outperforming the baseline of 65.5%.

The results confirm that as the Eigenlayer becomes more orthogonal, retrieval accuracy consistently improves, validating the hypothesis that reducing redundancy leads to more discriminative descriptors.

Conclusion

SVDNet introduces a novel structural regularization approach by enforcing orthogonality in deep features. By iteratively applying SVD through the RRI scheme, the model overcomes the limitations of correlated weights, leading to massive improvements in pedestrian retrieval. Future work may extend this framework to generic instance retrieval tasks, such as searching for products or landmarks.

Summary updated

Abstract

This paper proposes the SVDNet for retrieval problems, with focus on the application of person re-identification (reID). We view each weight vector within a fully connected (FC) layer in a convolutional neuron network (CNN) as a projection basis. It is observed that the weight vectors are usually highly correlated. This problem leads to correlations among entries of the FC descriptor, and compromises the retrieval performance based on the Euclidean distance. To address the problem, this paper proposes to optimize the deep representation learning process with Singular Vector Decomposition (SVD). Specifically, with the restraint and relaxation iteration (RRI) training scheme, we are able to iteratively integrate the orthogonality constraint in CNN training, yielding the so-called SVDNet. We conduct experiments on the Market-1501, CUHK03, and DukeMTMC-reID datasets, and show that RRI effectively reduces the correlation among the projection vectors, produces more discriminative FC descriptors, and significantly improves the re-ID accuracy. On the Market-1501 dataset, for instance, rank-1 accuracy is improved from 55.3% to 80.5% for CaffeNet, and from 73.8% to 82.3% for ResNet-50.
BibTeX
@inproceedings{sun2017svdnet,
  author = {Yifan Sun and Liang Zheng and Weijian Deng and Shengjin Wang},
  title = {{SVDNet for Pedestrian Retrieval}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2017},
  url = {https://openreview.net/forum?id=HyED_xM_-B},
}
Figure