Thoughts on a Recursive Classifier Graph: a Multiclass Network for Deep Object Recognition

Marius Leordeanu, Rahul SukthankararXiv 2014 · arXiv

TL;DR

This paper proposes the Classifier Graph, a unified framework for visual recognition where parts, objects, and scenes are treated as "equal citizens" in a recursive, directed graph. By allowing any learned classifier to become a feature for future nodes, the model builds a deep, multi-level hierarchy that naturally integrates top-down and bottom-up context for robust object recognition.

What Problem Does This Paper Address?

Traditional computer vision often separates low-level feature extraction from high-level reasoning, or uses rigid hierarchical structures (like simple cascades or trees). However, human perception is fluid: seeing a "wheel" helps you see a "car," but knowing you are in a "garage" also helps you identify a "wheel."

The authors argue that current models struggle to capture these rich, multi-directional interactions between different levels of abstraction. They aim to bridge the gap between engineered features (like HoG or SIFT) and deep learned architectures by creating a system that can grow its own complex graph of classifiers, where every node can potentially influence every other node.

Methods

The core of the proposal is the Classifier Graph, composed of nodes that are simple logistic linear units. The innovation lies in how this graph is built and navigated:

  • Recursive Feature Pool: The system starts with a pool of initial features (e.g., pixels, gradients, HoG). Once a classifier is learned (a "concept-node"), it is copied many times at different scales and locations and added back into the feature pool. These "feature-nodes" then become potential inputs for the next generation of classifiers.
  • Clusterboost: To learn new nodes, the authors propose an algorithm that combines Adaboost (for supervised weight updates) with unsupervised clustering. This ensures the system selects diverse features that specialize in different "views" or sub-classes of a concept, preventing the ensemble from becoming redundant.
  • Deep Detection Inference: Recognition is performed via a recursive algorithm. A child node calls its parents, which recursively call their own parents until the base features are reached. It incorporates a "search area" for each node, effectively performing a hierarchical version of max-pooling to handle spatial uncertainty.
Figure 2. Schematic description of a recursive, hierarchical face classifier. The initial node classifiers use the initial features types (e.g., HoG descriptors) and linear logistic regression. Once new classifiers (nodes) are learned, many random copies of them at different locations, scales and with different search areas (region of presence) are added to the pool of potential features. In this manner, input features could achieve recursively any level of abstraction. With each learning iteration a set of potentially more powerful and specific features are picked automatically using boosting and a one-layer classifier is learned. The process is repeated in this manner leading to a recursive structure, which could potentially handle any number of classes with links both bottom-up and top-down between different levels of interpretation and abstraction. The explicit labels used in this example (eye, nose, mouth, face) are given for clarity of presentation. In reality, intermediate classes or sub-classes (e.g., facelets) could be discovered and learned automatically. In this example: an eye classifier is first learned. Copies of it are added to the pool, at different locations, scales and with different search areas (regions of presence). When a face classifier is learned, the eye classifier with a given location, scale and search area, could be picked automatically and added to the graph, together with its own parent nodes (generic eye, iris and eyebrow) and their relative location and search area relative to their child, the whole eye classifier.

Datasets

As this is a Concept Paper, the authors do not present a final evaluation on a specific benchmark. However, they discuss the framework's compatibility with:

  • PASCAL VOC: Mentioned as a standard format for training data with bounding box annotations.
  • YouTube/Large-scale Video: Suggested as a source for learning spatiotemporal concepts through weak labels and temporal coherence.

Key Insights and Ideas

  • "Equal Citizens": In this framework, there is no conceptual difference between a "part" (an eye), an "object" (a face), and a "scene" (a room). They are all just classifiers operating at different scales and regions of interest.
  • Simultaneous Realities: A single pixel belongs to many categories at once. As shown in Figure 1, a pixel on a pupil is simultaneously part of an eye, a face, and a person. The Classifier Graph attempts to model these overlapping interpretations.
  • Top-Down and Bottom-Up Fusion: Unlike standard feed-forward nets, the graph allows directed edges in any direction. A scene classifier can provide context to a part detector (top-down), just as parts build up to objects (bottom-up).
  • Robustness through Redundancy: By having multiple classifiers for the same concept (e.g., a HoG-based person detector vs. a context-heavy person detector), the system can remain confident even if some features are missing or occluded.
Figure 1. Drawing of a woman’s face. Each pixel of the drawing belongs to many categories at the same time. For example, a pixel on the pupil, sits also on an eye, a face, and a person. Each category is seen by different classifiers, some specialized for the same class. Underneath the holistic visual experience of seeing a woman’s face, many classifiers at many levels of abstraction are simultaneously combined together to form a contextual environment for each others recognition.

Key Results

The paper focuses on the theoretical formulation rather than quantitative benchmarks. The primary "result" is the definition of a scalable, recursive architecture that:

  1. Generalizes existing hierarchical models (cascades, DPMs, DAGs).
  2. Scales to potentially billions of feature-nodes (estimated at hundreds of gigabytes to terabytes of storage).
  3. Extends to Video: The authors outline how the spatial "search area" can be mapped to a "temporal window," allowing the graph to recognize spatiotemporal "stories" or events.

Limitations and Future Work

  • Computational Complexity: The authors estimate a pool of $5 \times 10^9$ feature-nodes. While they argue this is manageable with pointers and weights, the memory and processing requirements for real-time inference are significant.
  • Learning Dynamics: While "Clusterboost" is proposed, the exact dynamics of how such a massive, interconnected graph converges during training across many epochs remains an open research question.
  • Empirical Validation: The paper lacks experimental results on standard computer vision benchmarks (like ImageNet or COCO), which would be necessary to prove the model's superiority over modern Convolutional Neural Networks.

Summary updated

Abstract

We propose a general multi-class visual recognition model, termed the Classifier Graph, which aims to generalize and integrate ideas from many of today's successful hierarchical recognition approaches. Our graph-based model has the advantage of enabling rich interactions between classes from different levels of interpretation and abstraction. The proposed multi-class system is efficiently learned using step by step updates. The structure consists of simple logistic linear layers with inputs from features that are automatically selected from a large pool. Each newly learned classifier becomes a potential new feature. Thus, our feature pool can consist both of initial manually designed features as well as learned classifiers from previous steps (graph nodes), each copied many times at different scales and locations. In this manner we can learn and grow both a deep, complex graph of classifiers and a rich pool of features at different levels of abstraction and interpretation. Our proposed graph of classifiers becomes a multi-class system with a recursive structure, suitable for deep detection and recognition of several classes simultaneously.
BibTeX
@misc{leordeanu2014thoughts,
  author = {Marius Leordeanu and Rahul Sukthankar},
  title = {{Thoughts on a Recursive Classifier Graph: a Multiclass Network for Deep Object Recognition}},
  year = {2014},
  eprint = {1404.2903},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1404.2903},
}
Figure