Automatic Recognition of Mammal Genera on Camera-Trap Images using Multi-Layer Robust Principal Component Analysis and Mixture Neural Networks

Jhony-Heriberto Giraldo-Zuluaga, Augusto Salazar, Alexander Gómez, Angélica Diaz-PulidoarXiv 2017 · OpenReview

TL;DR

This paper presents a novel, fully automatic method for identifying mammal genera in camera-trap images by jointly tackling animal segmentation and classification. Their approach combines Multi-Layer Robust Principal Component Analysis (RPCA) for segmentation with deep Convolutional Neural Networks (CNNs) for feature extraction and classification, achieving high accuracy even with challenging image conditions.

What Problem Does This Paper Address?

Camera-trap images are a crucial tool for wildlife monitoring, but analyzing them is a labor-intensive process. The core challenges lie in two areas: first, accurately segmenting animals from complex backgrounds and dealing with a high percentage of images that contain no animals (false positives). Second, classifying the identified animals into species or genera, which is complicated by varying image quality, lighting, and animal behavior. Previous work often tackled these problems separately or relied on manual segmentation, which is not scalable. This paper aims to bridge this gap by proposing an end-to-end automatic system that handles both segmentation and classification simultaneously.

Methods

The proposed method integrates several key techniques:

  • Multi-Layer Robust Principal Component Analysis (RPCA): This is used for the initial segmentation of animals from camera-trap images. RPCA is adept at separating low-rank (background) and sparse (foreground/animal) components in data. The "multi-layer" aspect suggests an extension of standard RPCA to handle more complex image structures.
  • Convolutional Neural Networks (CNNs): Deep CNN architectures (GoogLeNet, ResNet50, ResNet101, ResNet152) are employed for feature extraction from the segmented regions. The authors explore using these CNNs in a "black-box" feature extractor mode and also fine-tuning their last layers. They also propose a "MixtureNet" by concatenating features from multiple CNNs.
  • Least Absolute Shrinkage and Selection Operator (LASSO): LASSO is used for feature selection. This technique is valuable for reducing the dimensionality of the extracted features and making the classification process more robust by selecting the most informative features and potentially discarding noisy ones.
  • Classifiers (ANNs and SVMs): Artificial Neural Networks (ANNs) and Support Vector Machines (SVMs) are used to classify the selected features into mammal genera.

The novelty lies in the end-to-end integration of these components, particularly the joint handling of segmentation and classification, and the use of RPCA for automatic segmentation.

Datasets

The primary dataset used in this study comes from the Alexander von Humboldt Biological Resources Research Institute in Colombia. This dataset comprises 176 camera traps deployed across 10 regions, capturing images in both daytime color and nighttime infrared formats. The mammal genera in these images were manually labeled by biologists. The paper also mentions using the Snapshot Serengeti database as a reference for the prevalence of false positives in camera-trap data.

Key Insights and Ideas

  • Joint Segmentation and Classification is Crucial: The paper strongly advocates for tackling both segmentation and classification simultaneously, arguing that this integrated approach leads to better overall performance and a more robust system compared to methods that rely on pre-segmented or manually segmented images.
  • RPCA as an Effective Automatic Segmenter: Multi-Layer RPCA proves to be a capable tool for automatically segmenting animals, even in challenging conditions, providing regions of interest (ROIs) for subsequent classification.
  • LASSO for Robustness and Efficiency: LASSO feature selection not only reduces the computational burden by decreasing the number of features but also enhances the robustness of the classifiers by filtering out noisy or less informative features. This is particularly important when dealing with the inherent variability in camera-trap imagery.
  • CNN Ensembles (MixtureNet) Offer Benefits: Combining features from multiple CNN architectures (MixtureNet) leads to improved classification accuracy, suggesting that different CNNs capture complementary information.

Key Results

  • High Accuracy with Automatic Segmentation: The method achieved a remarkable 92.65% accuracy for classifying 9 mammal genera (including a "False Positive" class) using automatically segmented images with an Intersection over Union (IoU) greater than 0.5.
  • Strong Performance with Expert Segmentation: When using ground-truth (expert-segmented) images, the accuracy was 90.32% for 10 mammal genera.
  • LASSO's Impact: LASSO selection consistently improved or maintained accuracy across various CNN architectures and experiments, demonstrating its effectiveness in feature selection and robustness. It also significantly reduced the number of features, leading to faster classifier training.
  • Sparsity Achieved: LASSO selection resulted in significant sparsity, zeroing out over 96% of features in the MixtureNet, highlighting the redundancy in raw CNN features.

Limitations and Future Work

  • IoU Thresholding: The method's performance was evaluated on automatically segmented regions with an IoU > 0.5. Regions with 0 < IoU ≤ 0.5 were considered limitations, as they might confuse the learning algorithm or represent false detections.
  • Handling All RPCA Outputs: The authors suggest exploring the method's performance when feeding all outputs from the Multi-Layer RPCA (including those with IoU > 0) to the CNNs, rather than just those above a certain threshold.
  • Unbalanced Problem: The issue of unbalanced datasets (where some genera have far fewer images than others) remains a challenge. Future work could address this using techniques like one-shot learning or modifications to CNN training algorithms.
  • Sparse CNNs: Investigating the use of Sparse Convolutional Neural Networks (SCNNs) is proposed as a future direction.

Summary updated

Abstract

The segmentation and classification of animals from camera-trap images is due to the conditions under which the images are taken, a difficult task. This work presents a method for classifying and segmenting mammal genera from camera-trap images. Our method uses Multi-Layer Robust Principal Component Analysis (RPCA) for segmenting, Convolutional Neural Networks (CNNs) for extracting features, Least Absolute Shrinkage and Selection Operator (LASSO) for selecting features, and Artificial Neural Networks (ANNs) or Support Vector Machines (SVM) for classifying mammal genera present in the Colombian forest. We evaluated our method with the camera-trap images from the Alexander von Humboldt Biological Resources Research Institute. We obtained an accuracy of 92.65% classifying 8 mammal genera and a False Positive (FP) class, using automatic-segmented images. On the other hand, we reached 90.32% of accuracy classifying 10 mammal genera, using ground-truth images only. Unlike almost all previous works, we confront the animal segmentation and genera classification in the camera-trap recognition. This method shows a new approach toward a fully-automatic detection of animals from camera-trap images.
BibTeX
@misc{giraldozuluaga2017automatic,
  author = {Jhony-Heriberto Giraldo-Zuluaga and Augusto Salazar and Alexander Gómez and Angélica Diaz-Pulido},
  title = {{Automatic Recognition of Mammal Genera on Camera-Trap Images using Multi-Layer Robust Principal Component Analysis and Mixture Neural Networks}},
  year = {2017},
  eprint = {1705.02727},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1705.02727},
}
Figure