CNN2GNN: How to Bridge CNN With GNN

Ziheng Jiao, Hongyuan Zhang, Xuelong Li

TL;DR

CNN2GNN is a novel framework that bridges the gap between Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs) through heterogeneous knowledge distillation. By using a large CNN as a "teacher" and a tiny, 2-layer GNN as a "student," the authors demonstrate that a GNN can learn to extract both individual image features and complex relationships between images, often outperforming much deeper CNNs (like ResNet-152) on benchmarks like Mini-ImageNet.

What Problem Does This Paper Address?

Deep learning currently relies heavily on two different architectures:

  1. CNNs: Excellent at extracting "intra-sample" features (like the texture of a cat's fur or the shape of a car) but require stacking dozens or hundreds of layers, making them computationally expensive.
  2. GNNs: Excellent at capturing "inter-sample" relationships (how different data points relate to each other) using very few layers. However, GNNs traditionally require a pre-defined graph structure (which images don't have) and suffer from high latency during inference on large datasets.

The authors seek to answer a fundamental question: How can we bridge these two heterogeneous networks to enjoy the intra-sample strengths of CNNs and the topological strengths of GNNs simultaneously?

Methods

The CNN2GNN framework consists of two primary components:

1. Differentiable Sparse Graph Head

Since images don't come with a graph, the authors designed a "graph head" that learns one dynamically.

  • Sparsity Control: It uses a differentiable module to learn an $s$-sparse graph (where each node only connects to its $s$ most relevant neighbors). This prevents the graph from becoming a "noisy" mess of irrelevant connections.
  • Inductive Learning: To handle new, unseen images during testing, the authors propose two mechanisms. Mechanism 1 cascades a test image with a training batch to find its place in the graph. Mechanism 2 (more efficient) selects the most similar training samples to approximate the graph structure for a batch of test images.

2. Heterogeneous Distillation

The core idea is to transfer knowledge from a heavy-duty CNN teacher to a lightweight GNN student.

  • The Teacher: A pre-trained, deep CNN (e.g., ResNet-152 or VGG-19).
  • The Student: A simple 2-layer GNN.
  • The Loss Function: The GNN is trained using a combination of standard cross-entropy (for classification) and a Kullback-Leibler (KL) divergence loss. This "response-based distillation" forces the GNN to mimic the output probability distributions of the CNN teacher, effectively "inheriting" the CNN's feature-extraction capabilities.

Datasets

The authors evaluated their framework on four widely-used computer vision benchmarks:

  • STL-10: Used primarily for visualizing the learned topological relationships.
  • CIFAR-10 & CIFAR-100: Standard image classification datasets.
  • Mini-ImageNet: A more complex subset of ImageNet used to test the model's ability to handle diverse categories.

Key Insights and Ideas

  • GNNs as Bilinear Models: The authors argue that GNNs are inherently more efficient than CNNs because they are "bilinear." While a CNN layer is a linear projection, a GNN layer uses two factors: a projection matrix (for features) and a graph-related matrix (for relationships). This allows a 2-layer GNN to represent more complex information than a 2-layer CNN.
  • Beyond Compression: Unlike standard knowledge distillation (which usually goes CNN-to-CNN), this "heterogeneous" approach doesn't just compress the model; it changes the underlying logic of how the model "sees" the data—combining local features with global context.
  • The "Boosted" Student: Surprisingly, the distilled GNN student often outperforms its CNN teacher. This suggests that the GNN's ability to look at the "neighborhood" of similar images provides a performance boost that raw depth in a CNN cannot match.

Key Results

  • Superior Accuracy: On CIFAR-100, the CNN2GNN student consistently outperformed other state-of-the-art distillation methods (like KD, FitNets, and RKD). For example, a GNN student distilled from a ResNet-34 teacher achieved 65.13% accuracy, compared to the teacher's 62.19%.
  • Beating the Giants: On Mini-ImageNet, a 2-layer GNN student distilled from a ResNet-152 teacher achieved 60.64% accuracy, significantly higher than the teacher's 50.35%.
  • Efficiency: The GNN student achieves these results with significantly fewer FLOPs (Floating Point Operations) than the teacher models, representing a much better performance-to-efficiency trade-off.

Limitations and Future Work

  • Inference Latency: While Mechanism 2 improves efficiency, GNNs still face challenges with inference speed on extremely large-scale scenarios compared to pure linear models.
  • Graph Construction Cost: Although the graph head is differentiable, calculating the similarity matrix for very large batches can still be computationally intensive.
  • Future Directions: The authors suggest that this framework could be extended to other non-graph domains beyond images, potentially bridging GNNs with Transformers or other architectures.

Summary updated

Abstract

Thanks to extracting the intra-sample representation, the convolution neural network (CNN) has achieved excellent performance in vision tasks. However, its numerous convolutional layers take a higher training expense. Recently, graph neural networks (GNN), a bilinear model, have succeeded in exploring the underlying topological relationship among the graph data with a few graph neural layers. Unfortunately, due to the lack of graph structure and high-cost inference on large-scale scenarios, it cannot be directly utilized on non-graph data. Inspired by these complementary strengths and weaknesses, we discuss a natural question, how to bridge these two heterogeneous networks? In this paper, we propose a novel CNN2GNN framework to unify CNN and GNN together via distillation. First, to break the limitations of GNN, we design a differentiable sparse graph learning module as the head of the networks. It can dynamically learn the graph for inductive learning. Then, a response-based distillation is introduced to transfer the knowledge and bridge these two heterogeneous networks. Notably, due to extracting the intra-sample representation of a single instance and the topological relationship among the datasets simultaneously, the performance of the distilled “boosted” two-layer GNN on Mini-ImageNet is much higher than CNN containing dozens of layers, such as ResNet152.
BibTeX
@article{jiao2025cnn,
  author = {Ziheng Jiao and Hongyuan Zhang and Xuelong Li},
  title = {{CNN2GNN: How to Bridge CNN With GNN}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2025},
  doi = {10.1109/tpami.2025.3583357},
}
Figure