<tt>metatensor</tt> and <tt>metatomic</tt> : Foundational libraries for interoperable atomistic machine learning

Filippo Bigi, Joseph W. Abbott, Philip R. Loche, Arslan Mazitov, Davide Tisi, Marcel F. Langer, Alexander Goscinski, Paolo Pegolo, Sanggyu Chong, Rohit Goswami, Pol Febrer, Sofiia Chorna, Matthias Kellner, Michele Ceriotti, Guillaume FrauxThe Journal of Chemical Physics 2026 · DOI

TL;DR

This paper introduces metatensor and metatomic, two foundational libraries designed to bridge the gap between traditional atomistic simulation software and modern machine learning frameworks. They provide standardized ways to store, share, and use data and ML models, enabling greater interoperability and accelerating research in atomistic machine learning.

What Problem Does This Paper Address?

Atomistic simulations, crucial for understanding materials and chemistry, are increasingly leveraging machine learning (ML) to improve accuracy and reduce computational costs. However, this integration faces significant challenges:

  • Software Fragmentation: ML tools are often written in Python, while traditional simulation engines use Fortran, C, or C++. These ecosystems have different data structures and programming paradigms.
  • Interoperability Barriers: Sharing ML models and data between these disparate tools requires custom, time-consuming interfaces, limiting flexibility and reproducibility.
  • Rapid ML Evolution: The fast pace of ML development makes it difficult for researchers to keep up and integrate new tools into existing workflows.

This fragmentation hinders the widespread adoption and effective use of ML in atomistic simulations, making it challenging to build, deploy, and extend ML-powered scientific software.

Methods

The paper introduces two core libraries:

  • metatensor: This library provides a multi-platform, multi-language data format for arrays used in atomistic ML. Its key features include:
  • Metadata-Rich Arrays: Arrays are augmented with metadata that describes their nature, including geometric information and gradients with respect to atomic positions. This makes arrays "self-describing."
    • Block-Sparse Storage: Efficiently handles sparse data patterns common in atomistic ML, reducing memory usage.
  • Gradient Handling: Seamlessly stores and manipulates gradients alongside the primary data, essential for many physical quantities.
  • Serialization: Uses the npz format for language-agnostic, long-term storage and exchange of data.
  • metatomic: This library provides a standardized interface for packaging and distributing atomistic ML models. It allows models to be stored portably with their code, weights, and metadata, facilitating their use across different simulation packages.

The libraries are implemented with a C API, providing bindings for Python, C++, Rust, and TorchScript, enabling integration with various programming languages and ML frameworks. An ecosystem of companion libraries (metatensor-operations, metatensor-learn) and integrations with existing tools (LAMMPS, ASE, PLUMED, etc.) is also presented.

Datasets

The paper does not explicitly detail specific datasets used for training or evaluation of the libraries themselves. However, it mentions that the libraries are designed to handle data relevant to atomistic ML, including examples from:

  • Molecular Dynamics Simulations: Liquid water simulations are used to demonstrate performance.
  • Atomistic Properties: Examples include energies, forces, dipole moments, electron density, and chemical shieldings.
  • Existing Benchmarks: The libraries are designed to work with data from established atomistic ML workflows and datasets used in the field.

Key Insights and Ideas

  • Standardization is Key: The core idea is that a standardized data format (metatensor) and model interface (metatomic) are crucial for overcoming the fragmentation in atomistic ML software.
  • Metadata is Power: Attaching rich metadata to arrays makes them self-describing, significantly improving interoperability and reducing the need for custom parsing.
  • Bridging Worlds: The libraries act as a crucial bridge, allowing Python-centric ML tools to seamlessly interact with Fortran/C/C++ simulation engines.
  • Ecosystem Approach: The success of these libraries relies on building a robust ecosystem of tools and integrations, fostering community adoption and development.
  • Gradient-First Design: Explicitly handling gradients from the ground up is essential for scientific ML applications where gradients represent physical quantities like forces.

Key Results

  • Negligible Performance Overhead: Integration with LAMMPS using metatomic showed negligible performance overhead (around 2 µs/atom) compared to native implementations, demonstrating its efficiency.
  • Broad Ecosystem Adoption: The libraries are already being adopted by numerous research projects and integrated into a growing number of simulation and analysis tools.
  • Demonstrated Use Cases: The paper showcases successful integration for various tasks, including:
  • Interatomic Potentials: PET-MAD, a universal interatomic potential, is integrated and used for simulations.
  • Chemical Shielding Prediction: ShiftML models are integrated for NMR spectroscopy applications.
  • Molecular Dynamics Trajectories: FlashMD uses the libraries to predict trajectories.
  • Exploration of Potential Energy Surfaces: eOn and PLUMED leverage the libraries for advanced sampling and analysis.
  • Performance Benchmarks: Comparisons with other descriptor libraries (e.g., featomic vs. DScribe, librascal, QUIP) show competitive or superior performance in terms of speed and memory efficiency, especially for gradient calculations.

Limitations and Future Work

  • TorchScript Dependency: Currently, metatomic relies on TorchScript for model serialization, which might limit compatibility with models not easily exportable to TorchScript. Future work aims to decouple this and support more model types (pure Python, Julia, native C++).
  • Language/Platform Support: While aiming for multi-platform and multi-language support, ongoing efforts are planned to extend integration with JAX, Julia, and Fortran.
  • Ecosystem Expansion: The authors plan to continue expanding the ecosystem by integrating with more tools and developing new functionalities within metatensor and metatomic.
  • Custom Model Definition: Future work will focus on making it easier for users to define custom models within the metatensor framework.

Summary updated

Abstract

Incorporation of machine learning (ML) techniques into atomic-scale modeling has proven to be an extremely effective strategy to improve the accuracy and reduce the computational cost of simulations. It also entails conceptual and practical challenges, as it involves combining very different mathematical foundations as well as software ecosystems that are very well developed in their own right but do not share many commonalities. To address these issues and facilitate the adoption of ML in atomistic simulations, we introduce two dedicated software libraries. The first one, metatensor, provides multi-platform and multi-language storage and manipulation of arrays with many potentially sparse indices, designed from the ground up for atomistic ML applications. By combining the actual values with metadata that describes their nature and that facilitates the handling of geometric information and gradients with respect to the atomic positions, metatensor provides a common framework to enable data sharing between ML software-typically written in Python-and established atomistic modeling tools-typically written in Fortran, C, or C++. The second library, metatomic, provides an interface to store an atomistic ML model and metadata about this model in a portable way, facilitating the implementation, training, and distribution of models, and their use across different simulation packages. We showcase a growing ecosystem of tools, including low-level libraries, training utilities, and interfaces with existing software packages, that demonstrate the effectiveness of metatensor and metatomic in bridging the gap between traditional simulation software and modern ML frameworks.
BibTeX
@article{bigi2026tt,
  author = {Filippo Bigi and Joseph W. Abbott and Philip R. Loche and Arslan Mazitov and Davide Tisi and Marcel F. Langer and Alexander Goscinski and Paolo Pegolo and Sanggyu Chong and Rohit Goswami and Pol Febrer and Sofiia Chorna and Matthias Kellner and Michele Ceriotti and Guillaume Fraux},
  title = {{<tt>metatensor</tt> and <tt>metatomic</tt> : Foundational libraries for interoperable atomistic machine learning}},
  journal = {The Journal of Chemical Physics},
  year = {2026},
  doi = {10.1063/5.0304911},
}
Figure