Incremental Learning for Animal Pose Estimation using RBF k-DPP

Gaurav Kumar Nayak, Het Shah, Anirban Chakraborty

Abstract

Incremental learning is crucial for deploying computer vision models that can adapt to new data over time. In this paper, we address the challenge of incremental learning for animal pose estimation, where a model must learn to locate key joints in new animal species without forgetting previously learned ones. We propose a novel sampling method, RBF k-DPP, which utilizes a Radial Basis Function kernel to efficiently select a diverse set of exemplars for memory retention. Additionally, we introduce an anatomy-aware image-warping augmentation technique to generate diverse poses from limited memory samples. Experimental results on the Animal-Pose dataset demonstrate that our approach significantly outperforms existing incremental learning baselines, maintaining high accuracy across sequential learning steps.

Introduction

Pose estimation for animals is significantly more complex than for humans due to the vast morphological differences between species. While deep learning models have achieved high performance on static datasets, they struggle in dynamic environments where new animal categories must be added sequentially. Retraining a model from scratch every time a new class is introduced is computationally prohibitive, while simple fine-tuning leads to catastrophic forgetting—the rapid loss of performance on previously learned classes.

To address this, we frame animal pose estimation as an incremental learning problem. The goal is to learn a sequence of animal classes while maintaining a small, fixed-size "exemplar memory" of past data.

Methodology

Our framework focuses on two key components to mitigate forgetting: diverse exemplar selection and data augmentation.

1. RBF k-DPP Sampling

To maximize the utility of the limited memory budget, we must select the most representative and diverse images from each class. Determinantal Point Processes (DPPs) are well-suited for this as they naturally favor diversity. However, standard k-DPP implementations often use linear kernels, which suffer from rank limitations that restrict the number of samples that can be drawn. We propose using an RBF kernel (RBF k-DPP), which overcomes these rank constraints, eliminates the need for intermediate clustering steps, and ensures the selection of a highly diverse set of poses.

2. Anatomy-Aware Image Warping

With a limited memory of old classes and abundant data for new classes, models often overfit to the new data. We introduce a warping augmentation technique using Thin-Plate Splines (TPS). By rotating limbs around skeletal joints (elbows and knees) and using in-painting to fill gaps, we can synthesize new, anatomically valid poses from existing memory samples, effectively balancing the training distribution.

Overview of the proposed incremental learning framework, illustrating the RBF k-DPP sampling and image warping augmentation processes.

Experimental Results

We evaluated our approach on the Animal-Pose dataset (Cat, Cow, Dog, Horse, Sheep) using AlphaPose and PoseResnet architectures. We compared our method against several adapted state-of-the-art incremental learning baselines, including iCaRL, EEIL, and DER++.

As shown in the results table, our RBF k-DPP method consistently outperforms standard sampling strategies like Herding and Random selection. At a memory size of 1,000 samples, our full pipeline achieved a performance boost of over 14% compared to strong baselines like EEIL. The RBF kernel specifically provided a significant advantage over linear DPP variants by better capturing the high-dimensional diversity of animal poses.

Performance comparison ([email protected]) of various approaches across incremental steps and memory sizes.

Conclusion

This work establishes a robust baseline for incremental animal pose estimation. By combining mathematically grounded diversity sampling via RBF k-DPP with anatomy-aware data augmentation, we can effectively combat catastrophic forgetting. Our results highlight that maintaining a diverse and balanced representation of past knowledge is essential for building scalable and adaptable pose estimation systems.

Summary updated

BibTeX
@inproceedings{nayak2021incremental,
  author = {Gaurav Kumar Nayak and Het Shah and Anirban Chakraborty},
  title = {{Incremental Learning for Animal Pose Estimation using RBF k-DPP}},
  booktitle = {BMVC},
  year = {2021},
  url = {https://openreview.net/forum?id=5T9_edbbp1b},
}
Figure