Towards Run Time Estimation of the Gaussian Chemistry Code for SEAGrid Science Gateway

Angel Beltre, Shehtab Zaman, Kenneth Chiu, Sudhakar Pamidighantam, Xingye Qiao, Madhusudhan Govindaraju

In High-Performance Computing (HPC), efficient resource allocation is critical. Researchers using science gateways like SEAGrid for complex molecular simulations often struggle with the "Goldilocks" dilemma: requesting too little time leads to job termination, while requesting too much results in excessive queue wait times and idle resources. This study investigates the use of machine learning to predict runtimes for the Gaussian computational chemistry code, a widely used tool in the field.

The Challenge of Runtime Prediction

Predicting scientific code execution time is difficult due to the non-linear and high-dimensional relationship between inputs and runtimes. In Gaussian, quantum mechanical algorithms can scale from $O(N \log N)$ to $O(N^7)$, where $N$ represents the number of basis functions. Currently, users rely on heuristics, leading to significant inefficiencies. The goal of this research is to build a regression-based model to automate accurate estimates from input files.

Methodology and Feature Engineering

The researchers treated runtime estimation as a canonical regression problem, evaluating several algorithms:

  • Linear Models: Standard Linear Regression, Ridge, LASSO, and Elastic-Net.
  • Non-Linear Models: K-Nearest Neighbors (k-NN), Regression Trees (CART), and Deep Neural Networks (DNN).
  • Ensemble Models: Random Forest (RF).

To process the data, categorical "Route Keywords" were transformed into binary vectors via one-hot encoding. "Derived features," such as total atom and electron counts, were also included to capture molecular complexity.

Table of features used in the machine learning models, categorized by system features, route keywords, and derived features.

Dataset Characteristics

The study utilized a dataset of 37,034 successful Gaussian runs from the SEAGrid gateway recorded between 2005 and 2015. The data included 106 basis sets and various methods, with "B3LYP" being the most prevalent (43% of jobs). Data cleaning was essential to remove outliers—jobs lasting months due to system hangs or convergence loops.

A key observation was the "Long Tail" distribution of runtimes. While most jobs completed within 200 minutes, a small fraction of extremely long-running jobs dominated the dataset's variance, complicating the prediction task.

Density plot of Gaussian run times showing a heavily skewed distribution with a long tail, fitted against Log-Normal and Weibull distributions.

Key Results and Insights

The analysis yielded several critical findings:

  • Random Forest Performance: The Random Forest model achieved the best results with a Root Mean Square Error (RMSE) of 478.24 minutes.
  • Non-Linearity: DNNs (482.85 min RMSE) significantly outperformed linear baselines (533.06 min RMSE), confirming the non-linear nature of the relationship between chemical inputs and time.
  • Feature Sensitivity: Electronic complexity (electron count) was found to be a more significant driver of runtime than physical size (atom count).
  • Subsetting Success: Models were highly accurate for shorter jobs (under 15 minutes), where the error was approximately 4 minutes. Accuracy degraded for high-complexity jobs where high-order scaling effects dominate.

Conclusion and Future Directions

While non-linear models show promise, categorical keywords alone lack sufficient predictive power for production environments. Two identical sets of keywords can result in vastly different runtimes depending on a molecule's 3D geometry and symmetry. Future work will focus on incorporating molecular adjacency matrices to describe connectivity, hardware-specific metrics (e.g., memory bandwidth), and quantitative primitives like the actual number of basis functions.

Summary updated

Abstract

Accurate estimation of the run time of computational codes has a number of significant advantages for scientific computing. It is required information for optimal resource allocation, improving turnaround times and utilization of science gateways. Furthermore, it allows users to better plan and schedule their research, streamlining workflows and improving the overall productivity of cyberinfrastructure. Predicting run time is challenging, however. The inputs to scientific codes can be complex and high dimensional. Their relationship to the run time may be highly non-linear, and, in the most general case is completely arbitrary and thus unpredictable (i.e., simply a random mapping from inputs to run time). Most codes are not so arbitrary, however, and there has been significant prior research on predicting the run time of applications and workloads. Such predictions are generally application-specific, however. In this paper, we focus on the Gaussian computational chemistry code. We characterize a data set of runs from the SEAGrid science gateway with a number of different studies. We also explore a number of different potential regression methods and present promising future directions.
BibTeX
@inproceedings{beltre2019towards,
  author = {Angel Beltre and Shehtab Zaman and Kenneth Chiu and Sudhakar Pamidighantam and Xingye Qiao and Madhusudhan Govindaraju},
  title = {{Towards Run Time Estimation of the Gaussian Chemistry Code for SEAGrid Science Gateway}},
  booktitle = {PEARC},
  year = {2019},
  doi = {10.1145/3332186.3338101},
}
Figure