In High-Performance Computing (HPC), efficient resource allocation is critical. Researchers using science gateways like SEAGrid for complex molecular simulations often struggle with the "Goldilocks" dilemma: requesting too little time leads to job termination, while requesting too much results in excessive queue wait times and idle resources. This study investigates the use of machine learning to predict runtimes for the Gaussian computational chemistry code, a widely used tool in the field.
The Challenge of Runtime Prediction
Predicting scientific code execution time is difficult due to the non-linear and high-dimensional relationship between inputs and runtimes. In Gaussian, quantum mechanical algorithms can scale from $O(N \log N)$ to $O(N^7)$, where $N$ represents the number of basis functions. Currently, users rely on heuristics, leading to significant inefficiencies. The goal of this research is to build a regression-based model to automate accurate estimates from input files.
Methodology and Feature Engineering
The researchers treated runtime estimation as a canonical regression problem, evaluating several algorithms:
- Linear Models: Standard Linear Regression, Ridge, LASSO, and Elastic-Net.
- Non-Linear Models: K-Nearest Neighbors (k-NN), Regression Trees (CART), and Deep Neural Networks (DNN).
- Ensemble Models: Random Forest (RF).
To process the data, categorical "Route Keywords" were transformed into binary vectors via one-hot encoding. "Derived features," such as total atom and electron counts, were also included to capture molecular complexity.
Dataset Characteristics
The study utilized a dataset of 37,034 successful Gaussian runs from the SEAGrid gateway recorded between 2005 and 2015. The data included 106 basis sets and various methods, with "B3LYP" being the most prevalent (43% of jobs). Data cleaning was essential to remove outliers—jobs lasting months due to system hangs or convergence loops.
A key observation was the "Long Tail" distribution of runtimes. While most jobs completed within 200 minutes, a small fraction of extremely long-running jobs dominated the dataset's variance, complicating the prediction task.
Key Results and Insights
The analysis yielded several critical findings:
- Random Forest Performance: The Random Forest model achieved the best results with a Root Mean Square Error (RMSE) of 478.24 minutes.
- Non-Linearity: DNNs (482.85 min RMSE) significantly outperformed linear baselines (533.06 min RMSE), confirming the non-linear nature of the relationship between chemical inputs and time.
- Feature Sensitivity: Electronic complexity (electron count) was found to be a more significant driver of runtime than physical size (atom count).
- Subsetting Success: Models were highly accurate for shorter jobs (under 15 minutes), where the error was approximately 4 minutes. Accuracy degraded for high-complexity jobs where high-order scaling effects dominate.
Conclusion and Future Directions
While non-linear models show promise, categorical keywords alone lack sufficient predictive power for production environments. Two identical sets of keywords can result in vastly different runtimes depending on a molecule's 3D geometry and symmetry. Future work will focus on incorporating molecular adjacency matrices to describe connectivity, hardware-specific metrics (e.g., memory bandwidth), and quantitative primitives like the actual number of basis functions.
Summary updated