Graph-Based High-Order Relation Discovery for Fine-Grained Recognition

Yifan Zhao, Ke Yan, Feiyue Huang, Jia LiCVPR 2021 · OpenReview

Abstract

Fine-grained object recognition aims to learn effective features that can identify the subtle differences between visually similar objects. Most existing works tend to amplify discriminative part regions with attention mechanisms. Besides unstable performance under complex backgrounds, the intrinsic interrelationship between different semantic features is less explored. Toward this end, we propose an effective graph-based relation discovery approach to build a contextual understanding of high-order relationships. In our approach, a high-dimensional feature bank is first formed and jointly regularized with semantic- and positional-aware high-order constraints, endowing rich attributes to feature representations. Second, to overcome the high-dimension curse, we propose a graph-based semantic grouping strategy to embed this high-order tensor bank into a low-dimensional space. Meanwhile, a group-wise learning strategy is proposed to regularize the features focusing on the cluster embedding center. Experimental evidence demonstrates our approach achieves new state-of-the-art on 4 widely-used fine-grained object recognition benchmarks.

Introduction

Fine-grained object recognition (FGOR) represents an "expert-level" challenge in computer vision, requiring models to distinguish between visually similar sub-categories, such as specific bird species or car models. The primary difficulties lie in identifying subtle discriminative features (e.g., beak shape) and managing the "curse of dimensionality" when modeling complex inter-part relationships. Traditional attention-based methods often struggle with complex backgrounds and fail to fully exploit the intrinsic relationships between different semantic features.

Proposed Methodology: GaRD

We introduce the Graph-based High-Order Relation Discovery (GaRD) framework, which utilizes an "expansion and compression" strategy to capture rich contextual details.

Relation Discovery Module (Expansion)

Standard global pooling often discards vital spatial structures. GaRD addresses this via a position-aware gating operation. By analyzing how semantic channels (e.g., "beak" vs. "wing" features) interact across spatial locations, the module constructs a high-dimensional "tensor bank" of relationships. This ensures that high-order interactions are spatially grounded, relating parts in an anatomically consistent manner.

Graph-based Semantic Grouping (Compression)

To mitigate the computational burden of high-dimensional feature banks, we employ Graph Convolutional Networks (GCNs). Relationship features are treated as nodes in a graph, and GCNs are used to cluster nodes sharing mutual information. This semantic grouping maps high-dimensional data into a compact space while preserving critical "expert-level" details.

The GaRD architecture, featuring the Relation Discovery Module for feature expansion and the Graph Grouping Module for semantic compression.

Group-wise Learning Strategy

To improve stability, we propose training on the center of a class cluster within a mini-batch rather than individual instances. This strategy acts as a regularizer, encouraging the model to learn stable, representative features for a category and reducing sensitivity to noisy outliers or "hard samples."

Experimental Results

The GaRD approach was evaluated on four benchmark datasets: CUB-200-2011, Stanford-Cars, FGVC-Aircraft, and NA-birds. Using a standard ResNet-50 backbone, GaRD achieved state-of-the-art (SOTA) performance across all benchmarks, notably reaching 89.6% accuracy on CUB-200-2011 and 95.1% on Stanford-Cars.

Qualitative analysis via Grad-CAM visualizations reveals that GaRD develops a more holistic understanding of objects. While baseline models often focus on a single region, GaRD simultaneously attends to multiple discriminative parts, such as the head, torso, and tail of a bird.

Grad-CAM visualizations comparing the baseline, trilinear attention, and the proposed GaRD model. GaRD demonstrates a more comprehensive activation across multiple discriminative object parts.

Conclusion

The GaRD framework effectively captures high-order semantic relationships through a novel graph-based expansion and compression motif. By integrating spatial context and group-wise learning, the model achieves superior performance in fine-grained recognition tasks. Future work will explore dynamic graph structures to further optimize the trade-off between model complexity and representational power.

Summary updated

Abstract

Fine-grained object recognition aims to learn effective features that can identify the subtle differences between visually similar objects. Most of the existing works tend to amplify discriminative part regions with attention mechanisms. Besides its unstable performance under complex backgrounds, the intrinsic interrelationship between different semantic features is less explored. Toward this end, we propose an effective graph-based relation discovery approach to build a contextual understanding of high-order relationships. In our approach, a high-dimensional feature bank is first formed and jointly regularized with semantic- and positional-aware high-order constraints, endowing rich attributes to feature representations. Second, to overcome the high-dimension curse, we propose a graph-based semantic grouping strategy to embed this high-order tensor bank into a low-dimensional space. Meanwhile, a group-wise learning strategy is proposed to regularize the features focusing on the cluster embedding center. With the collaborative learning of three modules, our module is able to grasp the stronger contextual details of fine-grained objects. Experimental evidence demonstrates our approach achieves new state-of-the-art on 4 widely-used fine-grained object recognition benchmarks.
BibTeX
@inproceedings{zhao2021graph,
  author = {Yifan Zhao and Ke Yan and Feiyue Huang and Jia Li},
  title = {{Graph-Based High-Order Relation Discovery for Fine-Grained Recognition}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=VYwcPxxzaer},
}

Loading related content...

Figure