CyRSoXS: A GPU-accelerated virtual instrument for Polarized Resonant Soft X-ray Scattering (P-RSoXS)

Kumar Saurabh, Peter J. Dudenas, Eliot Gann, Veronica G. Reynolds, Subhrangsu Mukherjee, Daniel Sunday, Tyler B. Martin, Peter A. Beaucage, Michael L. Chabinyc, Dean M. DeLongchamp, Adarsh Krishnamurthy, Baskar GanapathysubramanianarXiv 2022 · arXiv

TL;DR

CyRSoXS is a new open-source, GPU-accelerated simulation tool that speeds up Polarized Resonant Soft X-ray Scattering (P-RSoXS) modeling by over 1,000x. By enabling near real-time simulations and providing a seamless Python interface, it transforms P-RSoXS from a slow, niche analysis into a high-throughput engine ready for machine learning and inverse material design.

What Problem Does This Paper Address?

Understanding the relationship between a material's structure and its performance is the "holy grail" of materials science. For soft materials like polymers and biomaterials, this is incredibly difficult because they are semi-disordered and their properties depend on molecular orientation at the nanoscale.

Polarized Resonant Soft X-ray Scattering (P-RSoXS) is a powerful technique that combines X-ray scattering with spectroscopy to "see" these molecular orientations and chemical heterogeneities. However, analyzing P-RSoXS data is a computational nightmare. Unlike standard X-ray scattering, which treats materials as simple density maps, P-RSoXS requires representing the sample as a complex 3D tensor at every point in space. The previous state-of-the-art simulation software was slow, commercially licensed, and difficult to integrate into modern data science workflows, making it nearly impossible to use for iterative tasks like fitting experimental data or training AI models.

Methods

The authors developed CyRSoXS, a virtual instrument designed from the ground up for high-performance computing. Key technical innovations include:

  • GPU Acceleration: The framework offloads heavy mathematical operations—like induced polarization calculations and Fast Fourier Transforms (FFTs)—to NVIDIA GPUs using CUDA.
  • GPU-Friendly Data Structures: They utilized "Real4" data types (vectorized floating-point numbers) and 1D memory arrays to maximize memory bandwidth and minimize the time spent moving data between the CPU and GPU.
  • Dual-Algorithm Approach:
    1. Algorithm 1 (Communication Minimization): Loads the entire material morphology onto the GPU once. It is incredibly fast but limited by the GPU's onboard memory (VRAM).
  • Algorithm 2 (Memory Minimization): Streams data per material component. This allows for the simulation of massive systems that exceed VRAM capacity at the cost of some speed.
  • Python Integration: Using Pybind11, the authors exposed the C++ core to Python. This allows researchers to pass data directly through memory, eliminating slow file input/output (I/O) operations and enabling easy use of libraries like NumPy, Matplotlib, and PyTorch.

The physical principles of the simulation are illustrated in Figure 1, showing how a polarized beam interacts with a voxel-based morphology to produce a scattering pattern.

Schematic of P-RSoXS physical principles and the simulation pipeline.

Datasets

To validate CyRSoXS, the authors used a series of rigorous benchmarks rather than traditional training datasets:

  • Analytical Solutions: They compared simulations against known mathematical results for form factor scattering (spheres) and periodic structures (hexagonal lattices and line gratings).
  • Cross-Software Validation: They tested CyRSoXS against the previous state-of-the-art Igor Pro-based simulator using a complex model of polymer-grafted nanoparticles (PGNs), which feature radial molecular stretching.

Key Insights and Ideas

  • The "Virtual Instrument" Paradigm: The authors treat the software not just as a calculator, but as a "digital twin" of a synchrotron beamline. This allows for in silico experimentation before ever stepping foot in a multi-billion dollar facility.
  • Overcoming the I/O Bottleneck: By providing a Python interface that stays in-memory, they removed the primary hurdle for "inverse design"—where an algorithm automatically tweaks a model until the simulation matches experimental data.
  • Orientation is Non-Negotiable: The paper emphasizes that ignoring molecular orientation in RSoXS can lead to fundamentally incorrect results. CyRSoXS makes it computationally feasible to include these orientation effects by default.

Key Results

  • 1000x Speedup: CyRSoXS achieves a speedup of over three orders of magnitude compared to previous software.
  • Massive Scale: It can handle near real-time simulations of up to 268 million voxels on a single NVIDIA V100 GPU.
  • Linear Scaling: The software shows ideal scaling across multiple GPUs, meaning doubling the hardware effectively halves the simulation time.
  • Perfect Fidelity: Validation tests showed that CyRSoXS perfectly reproduces the results of the reference simulator and matches analytical Bragg peak locations with high precision.

Limitations and Future Work

While powerful, the current version of CyRSoXS has a few constraints:

  • Uniaxial Assumption: The model currently assumes molecules have one dominant orientation axis (uniaxial). While sufficient for many polymers, more complex "biaxial" molecules are not yet supported.
  • Sub-voxel Homogeneity: The software cannot currently represent complex orientation distributions within a single voxel; it assumes one orientation per material per voxel.
  • Future Directions: The authors plan to expand the framework to support biaxial representations and further integrate it into MLOps (Machine Learning Operations) workflows for automated, high-throughput material discovery.

The code is open-source and available at https://github.com/usnistgov/cyrsoxs.

Summary updated

Abstract

Polarized Resonant Soft X-ray scattering (P-RSoXS) has emerged as a powerful synchrotron-based tool that combines principles of X-ray scattering and X-ray spectroscopy. P-RSoXS provides unique sensitivity to molecular orientation and chemical heterogeneity in soft materials such as polymers and biomaterials. Quantitative extraction of orientation information from P-RSoXS pattern data is challenging because the scattering processes originate from sample properties that must be represented as energy-dependent three-dimensional tensors with heterogeneities at nanometer to sub-nanometer length scales. We overcome this challenge by developing an open-source virtual instrument that uses GPUs to simulate P-RSoXS patterns from real-space material representations with nanoscale resolution. Our computational framework CyRSoXS (https://github.com/usnistgov/cyrsoxs) is designed to maximize GPU performance. We demonstrate the accuracy and robustness of our approach by validating against an extensive set of test cases, which include both analytical solutions and numerical comparisons, demonstrating a speedup of over three orders relative to the current state-of-the-art simulation software. Such fast simulations open up a variety of applications that were previously computationally infeasible, including (a) pattern fitting, (b) co-simulation with the physical instrument for operando analytics, data exploration, and decision support, (c) data creation and integration into machine learning workflows, and (d) utilization in multi-modal data assimilation approaches. Finally, we abstract away the complexity of the computational framework from the end-user by exposing CyRSoXS to Python using Pybind. This eliminates I/O requirements for large-scale parameter exploration and inverse design, and democratizes usage by enabling seamless integration with a Python ecosystem (https://github.com/usnistgov/nrss).
BibTeX
@misc{saurabh2022cyrsoxs,
  author = {Kumar Saurabh and Peter J. Dudenas and Eliot Gann and Veronica G. Reynolds and Subhrangsu Mukherjee and Daniel Sunday and Tyler B. Martin and Peter A. Beaucage and Michael L. Chabinyc and Dean M. DeLongchamp and Adarsh Krishnamurthy and Baskar Ganapathysubramanian},
  title = {{CyRSoXS: A GPU-accelerated virtual instrument for Polarized Resonant Soft X-ray Scattering (P-RSoXS)}},
  year = {2022},
  eprint = {2209.13121},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.13121},
}
Figure