Asymptotic Cross-Entropy Weighting and Guided-Loss in Supervised Hierarchical Setting using Deep Attention Network

Charles A. Kantor, Brice Rauby, Léonard Boussioux, Emmanuel Jehanno, André-Philippe Drapeau Picard, Maxim Larrivée, Hugues TalbotAAAI 2020 · OpenReview

Introduction

Fine-grained image classification represents a significant challenge in computer vision, requiring models to distinguish between highly similar categories, such as different species of butterflies. Unlike standard classification tasks where categories are distinct (e.g., truck vs. giraffe), fine-grained tasks often involve subtle morphological differences obscured by complex backgrounds. This paper addresses two primary hurdles: background bias, where models focus on environmental noise rather than the subject, and hierarchical neglect, where the inherent taxonomic relationships between labels are ignored.

Methodology

The authors propose a multi-stage pipeline leveraging a ResNet backbone enhanced with two novel techniques designed to improve feature extraction and label consistency.

Guided Attention Mechanism

To combat background bias, the researchers integrate a Convolutional Block Attention Module (CBAM) and introduce a Guided Attention Loss. Using a pre-trained Mask R-CNN to generate segmentation masks, the model is penalized if its internal attention maps fall outside the object's mask. This "blinders" effect forces the network to extract features exclusively from the insect's body rather than the surrounding foliage.

Hierarchical Weighted Cross-Entropy (WCE)

Biological labels are naturally nested (Family → Genus → Species). The proposed WCE loss combines predictions across these levels, ensuring the model respects taxonomic structures.

Asymptotic Weighting and Designed Loss

A common issue in hierarchical learning is "gradient starvation," where parent nodes (like Family) reach high confidence quickly, causing the model to stop refining the more difficult leaf nodes (Species). The authors designed an asymptotic weighting scheme to maintain significant gradient magnitude even as confidence increases, ensuring the model continues to learn the minute details necessary for species-level identification.

Visualization of the designed loss function (L) landscape relative to hierarchical probabilities (Pg, Pf).

Experimental Setup

The study utilizes the eButterfly dataset, a citizen science repository of North American butterfly observations. For these experiments, a balanced subset of 100 species was used to isolate the impact of the proposed loss functions. A small portion of the data (10%) was manually annotated to fine-tune the Mask R-CNN for the segmentation-guided attention task.

Key Results

The integration of the attention module and the designed hierarchical loss yielded measurable improvements over the baseline ResNet model.

Performance comparison between baseline ResNet and the proposed enhancements.

While the Top-1 accuracy improvement was incremental (reaching 81.01%), the addition of CBAM significantly boosted Top-3 accuracy to 93.35%. This suggests that the guided attention mechanism is particularly effective at narrowing down the correct species among a shortlist of visually similar candidates.

Insights and Future Work

The research highlights that background noise is a major distractor in fine-grained tasks. By using segmentation as a supervisory signal rather than just a preprocessing step, the model learns more robust features. Furthermore, treating hierarchy as a regularizer prevents "nonsensical" errors, such as confusing species from entirely different families.

Future work will focus on addressing overfitting through techniques like Stochastic Depth and scaling the approach to handle the "long-tailed" distribution typical of real-world citizen science data. The authors also aim to optimize the pipeline for real-time deployment by using segmentation primarily during the training phase to reduce inference overhead.

Summary updated

BibTeX
@inproceedings{kantor2020asymptotic,
  author = {Charles A. Kantor and Brice Rauby and Léonard Boussioux and Emmanuel Jehanno and André-Philippe Drapeau Picard and Maxim Larrivée and Hugues Talbot},
  title = {{Asymptotic Cross-Entropy Weighting and Guided-Loss in Supervised Hierarchical Setting using Deep Attention Network}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=IS3WpCJNfG},
}
Figure