Progressive Local Filter Pruning for Image Retrieval Acceleration

Xiaodong Wang, Zhedong Zheng, Yang He, Fei Yan, Zhiqiang Zeng, Yi Yang

Abstract

This paper focuses on network pruning for image retrieval acceleration. Prevailing image retrieval works target at the discriminative feature learning, while little attention is paid to how to accelerate the model inference, which should be taken into consideration in real-world practice. The challenge of pruning image retrieval models is that the middle-level feature should be preserved as much as possible. Such different requirements of the retrieval and classification model make the traditional pruning methods not that suitable for our task. To solve the problem, we propose a new Progressive Local Filter Pruning (PLFP) method for image retrieval acceleration. Specifically, layer by layer, we analyze the local geometric properties of each filter and select the one that can be replaced by the neighbors. Then we progressively prune the filter by gradually changing the filter weights. In this way, the representation ability of the model is preserved. To verify this, we evaluate our method on two widely-used image retrieval datasets, i.e., Oxford5k and Paris6K, and one person re-identification dataset, i.e., Market-1501. The proposed method arrives with superior performance to the conventional pruning methods, suggesting the effectiveness of the proposed method for image retrieval.

Introduction

Most neural network pruning research focuses on image classification, where the goal is to map an image to a discrete label. Because the output is categorical, these models can tolerate significant changes to internal feature distributions as long as the final classification remains correct. However, image retrieval and person re-identification tasks are fundamentally different. These tasks rely on learning a continuous feature space where the distance between points represents visual similarity.

Researchers have developed Progressive Local Filter Pruning (PLFP), a new method to speed up image retrieval models by removing redundant neural network filters. Unlike standard methods that prune based on global importance, PLFP preserves the delicate geometric structure of features, allowing for up to an 88.9% reduction in computation with minimal loss in accuracy. Standard pruning techniques often use a "global center-based" approach, removing filters that are close to the geometric center of all filters in a layer. This research demonstrates that such approaches "collapse" the feature distribution, destroying the fine-grained relationships needed to distinguish between visually similar objects.

Methodology: Progressive Local Filter Pruning (PLFP)

To address the limitations of global pruning, the authors propose Progressive Local Filter Pruning (PLFP), which consists of two main innovations:

1. Local Filter Selection

Instead of evaluating a filter's importance relative to the entire layer, PLFP examines its local neighborhood. If a filter is highly similar to its $k$-nearest neighbors (measured by Euclidean distance), it is deemed redundant because its neighbors can effectively "cover" its representational role. By pruning these locally redundant filters, the overall "shape" and "texture" of the filter distribution remain intact, preventing the creation of "holes" in the feature space.

Comparison of global center-based pruning (left) vs. the proposed local pruning method (right) and their impact on filter distribution.

2. Progressive Weight Decreasing

Traditional "hard pruning" immediately sets a filter's weights to zero, which can be disruptive to the model's learned representations. PLFP employs a "soft" approach where weights of candidate filters are gradually scaled down (multiplying by a factor $\gamma < 1$) over several epochs of fine-tuning. This progressive decay allows the model to adapt its remaining weights to compensate for the loss. Furthermore, if a filter was incorrectly selected for pruning, it maintains the potential to recover its importance before final removal.

Results and Discussion

The PLFP method was evaluated on standard benchmarks including Oxford5k, Paris6k, and the Market-1501 person re-identification dataset.

  • Distribution Preservation: As shown in the methodology, local pruning acts as a uniform thinning of the distribution rather than a localized collapse. This preservation is critical for maintaining high representation power, especially at aggressive pruning rates (e.g., >50% reduction).
  • Performance on Market-1501: The method achieved an 88.9% reduction in FLOPs with only a 4.2% increase in top-5 error. PLFP maintained approximately 8% higher mean Average Precision (mAP) compared to traditional $l_1$-norm pruning at the same compression rate.
  • Image Retrieval Efficiency: On the Oxford5k dataset, the model maintained a 76.20% mAP even after a 57.37% reduction in FLOPs.
  • Neighborhood Sensitivity: The optimal number of neighbors ($k$) is sensitive to the pruning rate. A larger $k$ is effective for light pruning, while a smaller $k$ is necessary for aggressive pruning to avoid deleting entire clusters of unique features.

Conclusion

PLFP provides a vital bridge between model compression and high-precision retrieval systems. By focusing on local geometric properties and employing a progressive pruning strategy, the method significantly accelerates inference while preserving the delicate feature relationships required for image retrieval. Future work may explore the application of these principles to Vision Transformers (ViTs) and the automation of hyperparameter selection for $k$ and $\gamma$.

Summary updated

Abstract

This paper focuses on network pruning for image retrieval acceleration. Prevailing image retrieval works target at the discriminative feature learning, while little attention is paid to how to accelerate the model inference, which should be taken into consideration in real-world practice. The challenge of pruning image retrieval models is that the middle-level feature should be preserved as much as possible. Such different requirements of the retrieval and classification model make the traditional pruning methods not that suitable for our task. To solve the problem, we propose a new Progressive Local Filter Pruning (PLFP) method for image retrieval acceleration. Specifically, layer by layer, we analyze the local geometric properties of each filter and select the one that can be replaced by the neighbors. Then we progressively prune the filter by gradually changing the filter weights. In this way, the representation ability of the model is preserved. To verify this, we evaluate our method on two widely-used image retrieval datasets,i.e., Oxford5k and Paris6K, and one person re-identification dataset,i.e., Market-1501. The proposed method arrives with superior performance to the conventional pruning methods, suggesting the effectiveness of the proposed method for image retrieval.
BibTeX
@article{wang2020progressive,
  author = {Xiaodong Wang and Zhedong Zheng and Yang He and Fei Yan and Zhiqiang Zeng and Yi Yang},
  title = {{Progressive Local Filter Pruning for Image Retrieval Acceleration}},
  journal = {IEEE Trans. Multimedia},
  year = {2020},
  doi = {10.1109/tmm.2023.3256092},
}
Figure