Improved Image Classification with Manifold Neural Networks

Caio F. Deberaldini Netto, Zhiyang Wang, Luana Ruiz

TL;DR

This paper proposes a novel framework for image classification by leveraging the manifold hypothesis. They construct an "image manifold" using Variational Autoencoders (VAEs) and then represent images as nodes in a graph, with their VAE embeddings as node features. By training Graph Neural Networks (GNNs) on these graphs, they achieve improved generalization compared to traditional methods, demonstrating that GNNs can effectively learn from data embedded in a low-dimensional manifold.

What Problem Does This Paper Address?

The paper tackles the challenge of applying Graph Neural Networks (GNNs) to image classification, a domain where explicit graph structures are not immediately apparent. While GNNs excel at learning from data with inherent graph structures (like social networks or molecular graphs), their application to image data is less explored. The authors are motivated by the "manifold hypothesis," which suggests that high-dimensional data, such as images, often resides on or near a lower-dimensional manifold within the ambient space. They aim to bridge the gap between this hypothesis and practical image classification by constructing a graph representation of this underlying image manifold and applying GNNs to it. This approach seeks to exploit the geometric structure of the data that might be missed by standard image processing techniques.

Methods

The core of the proposed method involves two main steps:

  1. Image Manifold Construction:
    • Variational Autoencoders (VAEs): The authors first use VAEs to learn a low-dimensional latent representation (embedding) for each image. Unlike standard autoencoders, VAEs learn a probabilistic mapping, which results in a smoother and more structured embedding space. This is crucial for capturing the underlying manifold.
  2. Graph Construction: Once images are embedded into this latent space, a graph is constructed. Each embedded image becomes a node in the graph. The edge weights between nodes are determined by the Gaussian kernel distance between their respective embeddings. This means images with similar latent representations are more strongly connected.
  3. GNN for Classification:
    • Graph Signals: The VAE embeddings of the images are treated as "graph signals" or node features.
  4. GNN Training: A Graph Neural Network (GNN) is then trained to predict the class labels of the images based on these graph signals and the constructed graph structure. The GNN's message-passing mechanism allows it to aggregate information from neighboring nodes (similar images), effectively learning from the manifold structure.

The paper also provides theoretical analysis on the generalization capabilities of GNNs trained on such manifold-sampled graphs, showing that their generalization gap (the difference between training and test error) is bounded and decreases with the number of sampled nodes.

Datasets

The proposed framework was evaluated on two standard image classification datasets:

  • MNIST: A dataset of handwritten digits in grayscale.
  • CIFAR10: A dataset of RGB images of 10 different objects.

The authors note that for MNIST, they used a latent space of size m=128, and for CIFAR10, m=4096.

Key Insights and Ideas

  • Images as points on a manifold: The paper successfully operationalizes the manifold hypothesis for images by using VAEs to create a smooth latent space that approximates this manifold.
  • GNNs as manifold learners: By treating VAE embeddings as graph signals and constructing graphs based on their proximity, GNNs are shown to be effective at learning from the implicit geometric structure of the image manifold.
  • Improved Generalization: The theoretical and empirical results strongly suggest that this manifold-based GNN approach leads to better generalization performance compared to methods that treat images as independent data points. This is attributed to the GNN's ability to leverage the learned manifold structure.
  • Theoretical grounding for GNN generalization: The paper connects GNNs trained on manifold-sampled graphs to Manifold Neural Networks (MNNs), providing theoretical bounds on the generalization gap that depend on the manifold dimension and graph size.

Key Results

  • Competitive Accuracy: The proposed GNN-based method achieved competitive accuracy on both MNIST and CIFAR10 datasets.
  • Reduced Generalization Gap: Figures 1 and 2 clearly illustrate that the proposed GNN approach exhibits a smaller generalization gap (difference between train and test accuracy) compared to a vanilla MLP trained on the same VAE embeddings, especially as the number of sampled nodes increases. This empirically validates the theoretical findings on generalization.
  • Superiority over Pixel-based GNNs: Table I shows that their method outperforms a GNN approach that builds graphs by interpreting image pixels as graph nodes, suggesting that learning a manifold representation first is more effective.

Limitations and Future Work

  • Hyperparameter Tuning: The authors mention that they did not fine-tune hyperparameters extensively, implying that further optimization could potentially yield even better results.
  • VAE Latent Space Dimension: The choice of latent space dimension (m=128 for MNIST, m=4096 for CIFAR10) was based on empirical performance, and exploring optimal dimensionality could be a future direction.
  • Generalization to Unseen Graphs: While the paper demonstrates generalization to unseen images within the same manifold, future work could explore generalization to entirely new manifolds or datasets.
  • Explicit Geometry: The paper focuses on implicit manifold learning. Future research could investigate how to incorporate explicit geometric information into this framework.

Summary updated

Abstract

Graph Neural Networks (GNNs) have gained popularity in various learning tasks, with successful applications in fields like molecular biology, transportation systems, and electrical grids. These fields naturally use graph data, benefiting from GNNs' message-passing framework. However, the potential of GNNs in more general data representations, especially in the image domain, remains underexplored. Leveraging the manifold hypothesis, which posits that high-dimensional data lies in a low-dimensional manifold, we explore GNNs' potential in this context. We construct an image manifold using variational autoencoders, then sample the manifold to generate graphs where each node is an image. This approach reduces data dimensionality while preserving geometric information. We then train a GNN to predict node labels corresponding to the image labels in the classification task, and leverage convergence of GNNs to manifold neural networks to analyze GNN generalization. Experiments on MNIST and CIFAR10 datasets demonstrate that GNNs generalize effectively to unseen graphs, achieving competitive accuracy in classification tasks.
BibTeX
@inproceedings{netto2025improved,
  author = {Caio F. Deberaldini Netto and Zhiyang Wang and Luana Ruiz},
  title = {{Improved Image Classification with Manifold Neural Networks}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10889415},
}
Figure