Generalized Convolutional Forest Networks for Domain Generalization and Visual Recognition

Jongbin Ryu, Gitaek Kwon, Ming-Hsuan Yang, Jongwoo Lim

Overview

Generalized Convolutional Forest Networks (GCFN) bridge the gap between the deep feature learning capabilities of Convolutional Neural Networks (CNNs) and the robust ensemble logic of Random Forests. By employing a novel "probabilistic triplet sampling" strategy based on how data points are split within decision trees, GCFN successfully maximizes individual tree accuracy while minimizing redundant correlations. This approach leads to state-of-the-art results in both domain generalization and image recognition tasks.

The Problem: Strength vs. Correlation

The performance of a Random Forest is governed by a delicate "Strength vs. Correlation" trade-off. To achieve a low error rate, individual trees must be "strong" (highly accurate) but also "uncorrelated" (making different types of mistakes).

The challenge is that these two goals usually conflict. If you optimize trees to be as accurate as possible, they often end up learning identical patterns, which spikes their correlation and hurts the overall ensemble's ability to generalize to new data. While CNNs are masters of feature learning, they do not inherently account for this forest-level trade-off. The core question this research addresses is: Can we train a CNN to learn a feature space specifically designed to make a Random Forest both stronger and more diverse?

Methods: The GCFN Framework

The authors introduce GCFN, an iterative learning framework that alternates between building a Random Forest and updating a CNN backbone.

GCFN Architecture

The "secret sauce" of this framework is its Probabilistic Triplet Sampling method:

  1. Feature Extraction: A CNN (such as ResNet or DenseNet) extracts features from input images.
  2. Forest Construction: A Random Forest is built on top of these extracted features.
  3. The Triplet Strategy: Instead of standard triplet loss—which simply pulls same-class items together—GCFN uses the forest's own internal structure to pick samples:
  4. Anchor ($a$): A random data point.
  5. Positive ($p$): A point with the same label as the anchor but located in a different leaf node. This encourages the network to find diverse features that lead to the same correct classification.
  6. Negative ($n$): A point that frequently ends up in the same leaf node as the anchor, regardless of its label. By pushing these apart, the model reduces correlation and prevents trees from becoming too similar.
  7. Backpropagation: The CNN is updated using a triplet loss function based on these forest-informed samples.

Key Insights

  • The "Aha" Moment on Correlation: Simply pulling same-labeled data together (Strengthened Learning) makes trees very accurate but too similar. By specifically targeting data points that "clump together" in the same leaf nodes for negative sampling, GCFN forces the network to "diffuse" the data, creating a more heterogeneous and robust ensemble.
  • Escaping Local Minima: Standard CNN training can get stuck in local optima. The proposed negative sampling acts as a form of regularization, pushing the optimization process out of these ruts by discouraging redundant feature representations.
  • Universal Applicability: Unlike many specialized models, GCFN isn't tuned for one specific task. It performs well across scenes, textures, and fine-grained objects, suggesting it learns a truly generalized feature representation.

Key Results

GCFN was evaluated across several challenging domains, including Visual Recognition (MIT-Indoor, Scene-15, 4D-Light, DTD, Stanford-Dog) and Domain Generalization (OfficeHome, VLCS). It consistently outperformed both baseline Random Forests and specialized state-of-the-art methods.

Performance Comparison across Datasets
  • Domain Generalization: On the OfficeHome dataset, GCFN achieved 64.7% average accuracy, significantly higher than previous methods like D-SAMs (60.8%) and MMD-AAE.
  • Visual Recognition: On the Stanford-Dog dataset, GCFN reached 86.8% accuracy using a DenseNet-161 backbone, outperforming other advanced methods.
  • Ensemble Growth: The research demonstrates that while standard "strengthened" models might start strong with just one tree, GCFN scales much better as the number of trees increases because it actively keeps correlation in check.

Limitations and Future Work

  • Computational Overhead: The iterative process requires rebuilding the forest and sampling triplets every few epochs. While this adds only about 25 minutes to the total training time for a ResNet-50 on MIT-Indoor, it is still more complex than a standard end-to-end CNN pipeline.
  • Hyperparameter Sensitivity: The method relies on several parameters (number of trees, depth, triplet margin). While the paper provides defaults, optimal performance might require tuning for new, unseen datasets.
  • Future Directions: The authors suggest that this logic could be extended to other ensemble types or more complex split functions (like node-wise SVMs) to further push the boundaries of visual recognition.

Summary updated

Abstract

When constructing random forests, it is of prime importance to ensure high accuracy and low correlation of individual tree classifiers for good performance. Nevertheless, it is typically difficult for existing random forest methods to strike a good balance between these conflicting factors. In this work, we propose a generalized convolutional forest networks to learn a feature space to maximize the strength of individual tree classifiers while minimizing the respective correlation. The feature space is iteratively constructed by a probabilistic triplet sampling method based on the distribution obtained from the splits of the random forest. The sampling process is designed to pull the data of the same label together for higher strength and push away the data frequently falling to the same leaf nodes. We perform extensive experiments on five image classification and two domain generalization datasets with ResNet-50 and DenseNet-161 backbone networks. Experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.
BibTeX
@inproceedings{ryu2020generalized,
  author = {Jongbin Ryu and Gitaek Kwon and Ming-Hsuan Yang and Jongwoo Lim},
  title = {{Generalized Convolutional Forest Networks for Domain Generalization and Visual Recognition}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
}
Figure