Virtual CNN Branching: Efficient Feature Ensemble for Person Re-Identification

Albert Gong, Qiang Qiu, Guillermo SapiroarXiv 2018 · arXiv

Abstract

Person re-identification (re-ID) faces significant challenges due to intra-class variations in pose, lighting, and camera viewpoints. While ensemble methods offer robustness, they are often computationally prohibitive. This paper introduces "Virtual Branching," a novel architectural technique that partitions neurons in the final layers of a standard CNN into distinct functional sets. By training these "virtual branches" on specialized tasks such as landmark localization and pose orientation, the model achieves the benefits of an ensemble within a single network. Experimental results on major benchmarks like Market-1501 and DukeMTMC-reID demonstrate state-of-the-art performance with significantly fewer parameters than traditional deep ensembles.

Introduction

Identifying the same individual across non-overlapping camera views is a cornerstone of modern surveillance and computer vision. The primary hurdle in person re-ID is the high degree of intra-class variation; a single person's appearance can change drastically depending on their orientation to the camera or the specific body parts visible.

To combat this, researchers typically rely on model ensembles or complex part-based architectures. However, ensembles increase memory and computational costs linearly, while part-based models often require auxiliary region proposal networks. Virtual Branching addresses these inefficiencies by reorganizing the internal structure of a single CNN to foster functional specialization without adding extra parameters.

Proposed Method: Virtual CNN Branching

The core of the approach lies in the strategic partitioning of neurons in the top layers of a backbone network (e.g., DenseNet). Instead of all neurons contributing to a single global feature vector, they are divided into $b$ sets, or "virtual branches."

Architecture and Parameter Sharing

A hyperparameter $\delta$ determines the balance between shared and unique neurons across branches. This structure allows the lower layers to learn general visual features while forcing the upper layers to develop specialized expertise. During training, activations of neurons not assigned to a specific branch are omitted, effectively creating a deterministic, semantic form of Dropout.

Overview of the Virtual Branching architecture within a DenseNet backbone.

Specialized Training Schemes

To induce functional diversity among the branches, the authors employ two primary strategies:

  1. Human Landmark Localization: Specific branches are tasked with regressing heatmaps for key body regions like the neck, hips, and ankles. A localization-inducing loss is added to the standard triplet loss to ground the network's attention on physical landmarks.
  2. Pose Orientation Specialization: Data is partitioned by viewpoint (front, side, back), and individual branches are trained to specialize in these specific orientations.
Visualization of landmark localization heatmaps for neck, hip, and ankle regions.

Experimental Results

The method was evaluated on the Market-1501, CUHK03, and DukeMTMC-reID datasets.

Performance and Efficiency

On the Market-1501 dataset, the Virtual Branching model achieved a mean Average Precision (mAP) of 71.1%, a notable improvement over the 68.3% baseline. Remarkably, this performance is achieved with only 8.2 million parameters. In contrast, competing models like TriNet require approximately 25.8 million parameters to reach similar accuracy levels.

Computational Overhead

Because the branching occurs only in the final stages of the network, the impact on inference latency is minimal. Testing showed that increasing the number of branches from 1 to 10 adds only a few milliseconds of processing time per 1,000 images, making it highly suitable for real-time applications.

Conclusion

Virtual CNN Branching provides a highly efficient way to gain the benefits of model ensembles. By forcing functional specialization through neuron partitioning and targeted loss functions, the approach achieves state-of-the-art re-ID performance while maintaining a small memory footprint and low latency. Future work may explore automated partitioning strategies to remove the need for manual landmark or pose labeling.

Summary updated

Abstract

In this paper we introduce an ensemble method for convolutional neural network (CNN), called "virtual branching," which can be implemented with nearly no additional parameters and computation on top of standard CNNs. We propose our method in the context of person re-identification (re-ID). Our CNN model consists of shared bottom layers, followed by "virtual" branches, where neurons from a block of regular convolutional and fully-connected layers are partitioned into multiple sets. Each virtual branch is trained with different data to specialize in different aspects, e.g., a specific body region or pose orientation. In this way, robust ensemble representations are obtained against human body misalignment, deformations, or variations in viewing angles, at nearly no any additional cost. The proposed method achieves competitive performance on multiple person re-ID benchmark datasets, including Market-1501, CUHK03, and DukeMTMC-reID.
BibTeX
@misc{gong2018virtual,
  author = {Albert Gong and Qiang Qiu and Guillermo Sapiro},
  title = {{Virtual CNN Branching: Efficient Feature Ensemble for Person Re-Identification}},
  year = {2018},
  eprint = {1803.05872},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.05872},
}
Figure