Foundry-ML - Software and Services to Simplify Access to Machine Learning Datasets in Materials Science

K. J. Schmidt, Aristana Scourtas, Logan T. Ward, Steve Wangen, Marcus Schwarting, Isaac Darling, Ethan Truelove, Aadit Ambadkar, Ribhav Bose, Zoa Katok, Jingrui Wei, Xiangguo Li, Ryan Jacobs, Lane Schultz, Doyeon Kim, Michael C. Ferris, Paul M. Voyles, Dane Morgan, Ian T. Foster, Ben BlaiszikJ. Open Source Softw 2024 · DOI · DBLP

Introduction

In the rapidly evolving field of AI-driven materials science, data is the most valuable currency. However, scientific data is often "FAIR-ly" difficult to use. While the FAIR principles (Findable, Accessible, Interoperable, and Reusable) are the gold standard, the reality is that most datasets are decentralized, lack shared standards, and are scattered across various repositories.

For a machine learning researcher, this often necessitates significant "data janitorial" work—manually downloading files, deciphering inconsistent metadata, and writing custom scripts just to load data into a model. Foundry-ML addresses this bottleneck by providing a centralized infrastructure that standardizes how scientific datasets are published, discovered, and accessed.

Architecture and Methods

Foundry-ML is an open-source Python library and web platform designed to make materials science and chemistry datasets truly "ML-ready." By providing a unified interface for discovery, standardized metadata, and simplified data loading, it allows researchers to move from a dataset DOI to a training-ready dataframe in just a few lines of code. Its architecture consists of three main pillars:

  1. The Python Client: A library that allows users to instantiate a "Foundry" object and load datasets directly into common data structures like Pandas or Dask dataframes using a DOI.
  2. Standardized Metadata: Every dataset in Foundry-ML requires specific metadata (stored via Globus Search). This includes details on data types, units, and—crucially for ML—pre-defined splits for training, testing, and validation.
  3. Backend Integration: It leverages the Materials Data Facility (MDF) for robust data storage and Globus for secure authentication and high-speed data transfer. This allows the system to handle everything from small CSV files to multi-terabyte databases.
The Foundry-ML workflow: (a) loading a dataset via DOI, (b) inspecting standardized metadata, and (c) loading the data into training-ready dataframes for analysis and visualization.

Dataset Ecosystem

As of early 2023, Foundry-ML hosted a curated collection of 30 datasets. These cover a wide range of representations, including tabular data (CSV/Excel), key-value pairs (JSON), image sets, and hierarchical data (HDF5). Specific benchmarks and datasets hosted include:

  • Zeolite Design: A dataset for organic structure-directing agents (OSDAs) from the Gomez-Bombarelli team at MIT.
  • Semiconductors: Impurity charge-state transition levels.
  • Metallic Glasses: Properties and compositions.
  • Solvation Energy: G4MP2 estimates for various solvents.
  • Steel Properties: Mechanical properties of different steel grades.
  • Band Gaps: Graph network-based deep learning of computational band gaps.

Key Features and Impact

Foundry-ML treats metadata as a first-class citizen. By enforcing requirements such as units and input/target labels, it ensures that models are interpretable and results are reproducible. While currently focused on materials science and chemistry, the underlying software is designed to be domain-agnostic. The "Foundry" pattern can be applied to any scientific field struggling with data fragmentation.

The platform significantly lowers the barrier to entry by providing a "self-service" publishing model, encouraging researchers to share their ML-ready datasets without needing to build their own web infrastructure. The paper demonstrates that a researcher can load a complex dataset and its associated metadata with essentially three lines of Python code. This scalability and ease of use have led to adoption in educational curricula and research teams at institutions like Argonne National Laboratory, UChicago, MIT, and the University of Toronto.

Conclusion and Future Directions

While the framework is powerful, it is currently most mature within the materials science and chemistry communities. Future directions include generalizing metadata schemas to support other scientific fields and the "Garden" project—a planned expansion to create a combined ecosystem where datasets are directly connected to pre-trained ML models, allowing for a seamless "data-to-inference" pipeline.

Summary updated

BibTeX
@article{schmidt2024foundry,
  author = {K. J. Schmidt and Aristana Scourtas and Logan T. Ward and Steve Wangen and Marcus Schwarting and Isaac Darling and Ethan Truelove and Aadit Ambadkar and Ribhav Bose and Zoa Katok and Jingrui Wei and Xiangguo Li and Ryan Jacobs and Lane Schultz and Doyeon Kim and Michael C. Ferris and Paul M. Voyles and Dane Morgan and Ian T. Foster and Ben Blaiszik and Logan Ward and Steven R. Wangen and R. Bose and Lane E. Schultz and Ian T. Foster},
  title = {{Foundry-ML - Software and Services to Simplify Access to Machine Learning Datasets in Materials Science}},
  journal = {J. Open Source Softw.},
  year = {2024},
  doi = {10.21105/joss.05467},
}
Figure