Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets (Ultra Large Dataset)
Lists
Only you see this
Abstract
Recently, pre-trained foundation models have shown significant advancements in multiple fields. However, the lack of datasets with labeled features and codebases has hindered the development of a supervised foundation model for molecular tasks. Here, we have carefully curated seven datasets specifically tailored for node- and graph-level prediction tasks to facilitate supervised learning on molecules. Moreover, to support the development of multi-task learning on our proposed datasets, we created the Graphium graph machine learning library. Our dataset collection encompasses two distinct categories. Firstly, the TOYMIX category modifies three small existing datasets with additional data for multi-task learning. Secondly, the LARGEMIX category includes four large-scale datasets with 344M graph-level data points and 409M node-level data points from ∼5M unique molecules. Finally, the ultra-large dataset contains 2,210M graph-level data points and 2,031M node-level data points coming from 86M molecules. Hence our datasets represent an order of magnitude increase in data volume compared to other 2D-GNN datasets. In addition, recognizing that molecule-related tasks often span multiple levels, we have designed our library to explicitly support multi-tasking, offering a diverse range of multi-level representations, i.e., representations at the graph, node, edge, and node-pair level. We equipped the library with an extensive collection of models and features to cover different levels of molecule analysis. By combining our curated datasets with this versatile library, we aim to accelerate the development of molecule foundation models. Datasets and code are available at https://github.com/datamol-io/graphium.
BibTeX
@inproceedings{beaini2023towards,
author = {Dominique Beaini and Shenyang Huang and Joao Alex Cunha and Zhiyi Li and Gabriela Moisescu-Pareja and Oleksandr Dymov and Samuel Maddrell-Mander and Callum McLean and Ali Parviz and Luis T. Díaz Müller and Jama Hussein Mohamud and Frederik Wenkel and Michael Craig and Michał Koziarski and Jiarui Lu and Zhaocheng Zhu and Cristian Gabellini and Guillaume Rabusseau and Reihaneh Rabbany and Jian Tang and Christopher G. Morris and Mirco Ravanelli and Guy Wolf and Prudencio Tossou and Hadrien Mary and Błażej Banaszewski and Chad Martin and Dominic Masters},
title = {{Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets (Ultra Large Dataset)}},
booktitle = {Zenodo},
year = {2023},
doi = {10.5281/zenodo.8370548},
}Loading related content...