ZeoBench: A Benchmark for Self-Supervised Learning on 3D Zeolite Representations

Aaron Sun, Yachan Liu, Ping Yang, Peng Bai, Subhransu MajiarXiv 2026 · arXiv

TL;DR

ZeoBench is a new benchmark for self-supervised learning (SSL) on 3D volumetric representations of zeolites, porous crystalline materials used in industrial catalysis and gas storage. By pretraining on over 331,000 unlabeled structures, the researchers demonstrate that SSL significantly improves property prediction in low-data regimes and even generalizes to out-of-domain materials like Metal-Organic Frameworks (MOFs).

What Problem Does This Paper Address?

Zeolites are essential "molecular sieves" in the chemical industry, but finding the perfect zeolite for a specific application is a needle-in-a-haystack problem. While we have databases of hundreds of thousands of theoretical zeolite structures, determining their properties (like how well they adsorb a specific hydrocarbon) requires computationally expensive Monte Carlo simulations that can take hours per structure.

This creates a "low-data regime" where we have a massive corpus of unlabeled 3D structures but very few expensive labels. Previous work used supervised 3D Convolutional Neural Networks (CNNs) to predict properties, but these models require thousands of labels to be effective. ZeoBench addresses this gap by investigating how self-supervised learning can leverage the unlabeled 300k+ structures to learn generalizable geometric features, reducing the need for labeled data.

Methods

The authors represent zeolites as 3D Signed Distance Grids, where each voxel encodes the distance to the nearest framework surface. They evaluate four popular SSL objectives:

  • Contrastive: SimCLR and MoCo v2+ (learning by distinguishing between different views of the same structure).
  • Distillation: BYOL and DINO (learning by matching a student network's output to a teacher network).

A key technical contribution is the exploration of 3D data augmentations. While standard 2D image augmentations (like color jittering) don't apply to distance grids, the authors found that scale augmentation—independently varying the grid resolution along each axis—is critical for learning robust features.

They also provide a rigorous comparison against "ImageNet-to-3D" adaptations, where 2D models pretrained on natural images are adapted to 3D volumes via three strategies:

  1. 2D Slices: Stacking random 2D slices as RGB channels.
  2. Channel Inflation: Duplicating 2D weights to match 3D volume depth.
  3. Depth Tokenization: Treating the 3D volume as a sequence of image patches for a Vision Transformer (ViT).
Figure 2: Adapting 2D networks for 3D inputs. We explore three methods to get ImageNet features from volumetric inputs: 2D slices, channel inflation, and depth tokenization. 2D slices (left) involve stacking three different slices to form an image. Channel inflation (center) repeats the weights of the first convolutional layer. Depth tokenization (right) treats the volume as a stack of D/3 images.

Datasets

  • Pretraining Corpus: 331,172 theoretical and 402 experimentally synthesized all-silica zeolite structures.
  • Downstream Tasks: Six molecular adsorption tasks (predicting Henry’s constant for various hydrocarbons like C18, C24, and branched isomers) across varying label budgets (100 to 10,000 samples).
  • Transfer Learning: The hMOF dataset, consisting of 137,000 Metal-Organic Frameworks, used to test if zeolite-pretrained features generalize to different chemical domains.

Key Insights and Ideas

  • CNNs > Transformers (for now): In this 3D geometric setting, 3D ResNets consistently outperformed Vision Transformers (ViT) and Swin Transformers. The authors suggest that the translational equivariance priors in CNNs are better suited for grid-based molecular data.
  • Contrastive SSL Wins on Frozen Features: For "linear probing" (where the backbone is frozen), contrastive methods like SimCLR and MoCo produced the strongest representations. This is likely because their training objective encourages a more uniform distribution of embeddings, which is beneficial for continuous regression tasks like adsorption prediction.
  • Scale is the Secret Sauce: Without scale augmentation during pretraining, SSL performance drops significantly. Scaling forces the network to learn global topological features rather than just memorizing local surface curvatures.
  • ImageNet is a Surprisingly Strong Baseline: When adapted via "depth tokenization," ImageNet-pretrained features outperformed hand-crafted descriptors and even some medical-domain 3D models, though they still lagged behind in-domain zeolite SSL.

Key Results

The study establishes that in-domain SSL provides a massive boost in performance, particularly when labels are scarce.

  • Low-Label Performance: With only 100 labels, SSL-pretrained models achieved significantly lower Mean Squared Error (MSE) compared to training from scratch or using hand-crafted descriptors.
  • State-of-the-Art: On the full C18 adsorption task, SSL pretraining improved performance by 16% over the previous best supervised baseline (ZeoNet).
  • Cross-Domain Generalization: A SimCLR model pretrained on zeolites outperformed existing MOF-specific architectures (like MOFormer) on the hMOF dataset, proving that the model learns fundamental principles of nanoporous geometry.
Figure 3: Method performance on ZeoBench averaged across all six downstream tasks for varying subset sizes (lower is better). For linear probe (left), SimCLR and MoCo outperform other methods in all contexts. For fine-tuning (right), SSL methods (solid) outperform training from scratch (dotted), especially when labeled data is limited. ImageNet results shown are the best- performing adaptation method, depth tokenization.

Limitations and Future Work

While ZeoBench provides a comprehensive foundation, the authors note several areas for growth:

  • Architecture Diversity: The study focused primarily on 3D ResNet-18. Future work could explore larger backbones or group-equivariant networks that natively handle 3D rotations.
  • Chemical Complexity: The current benchmark focuses on all-silica zeolites. Expanding to zeolites with different framework atoms (like Aluminum) or extra-framework ions would increase the benchmark's industrial relevance.
  • Task Expansion: Beyond adsorption, future versions could include tasks like catalytic activity or synthetic feasibility.

The code and dataset splits are available at https://github.com/cvl-umass/ZeoBench.

Summary updated

Abstract

Zeolites are an important class of porous crystalline materials with broad applications in gas storage, separations, and catalysis. Although hundreds of thousands of experimentally realized and hypothetical zeolite structures are known, identifying materials optimized for a target application remains challenging because property labels are expensive to obtain and available only for limited structure-molecule pairs. While prior work has demonstrated that 3D neural representations can predict zeolite properties effectively, it remains unclear which representation-learning strategies are most effective in the low-data regime relevant to materials discovery. We present ZeoBench, a benchmark for self-supervised learning on dense 3D representations of all-silica zeolites, and evaluate the resulting features on six existing downstream adsorption-property prediction tasks, across a range of training set sizes. Our study compares self-supervised 3D pretraining on volumetric zeolite data, adaptations of image-pretrained 2D ConvNets and vision transformers to 3D via multiview and channel-inflation strategies, and 3D ConvNets pretrained on out-of-domain volumetric datasets, alongside hand-crafted descriptors and supervised baselines. Using our pretraining and evaluation framework, we find self-supervised 3D representations outperform existing methods and achieve state-of-the-art performance across both in-domain label regimes and for the unseen domain of metal organic frameworks. Learned representations consistently outperform hand-crafted descriptors, and convolutional architectures are generally more effective than vision transformers in this setting. These results establish ZeoBench as a practical benchmark for 3D representation learning in zeolites and provide guidance for data-limited adsorption property prediction.
BibTeX
@misc{sun2026zeobench,
  author = {Aaron Sun and Yachan Liu and Ping Yang and Peng Bai and Subhransu Maji},
  title = {{ZeoBench: A Benchmark for Self-Supervised Learning on 3D Zeolite Representations}},
  year = {2026},
  eprint = {2611.09880},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09880},
}
Figure