Introduction
Computational Fluid Dynamics (CFD) is the cornerstone of modern automotive aerodynamic design, providing high-fidelity simulations of airflow around vehicle geometries. However, the computational cost of these simulations—often requiring days on high-performance clusters—limits the speed of iterative design. AI-based surrogate models offer a promising alternative, capable of providing near-instantaneous feedback. Despite this potential, the field has historically lacked a standardized benchmarking framework. Current research often relies on disparate datasets and standard machine learning metrics like Mean Squared Error (MSE), which can fail to capture critical engineering quantities such as drag and lift coefficients. To address this, researchers have introduced a standardized benchmarking suite within the open-source PhysicsNeMo-CFD library.
The PhysicsNeMo-CFD Framework
The proposed framework, integrated into NVIDIA's PhysicsNeMo ecosystem, shifts the focus from generic ML metrics to physically-grounded engineering evaluations. It provides tools to compute aerodynamic forces by integrating predicted surface pressure and shear stress fields. Furthermore, it evaluates design trends using Spearman rank correlation coefficients and analyzes 1D centerline pressure distributions, which are vital for understanding local flow features.
A key technical challenge in automotive AI is the mesh-to-point-cloud transition. While traditional CFD relies on complex meshes, many AI models operate on point clouds. The framework includes utilities to validate models directly on uniformly sampled point clouds derived from STL files, ensuring that the evaluation remains relevant to the original geometric design.
Benchmarking Methodology
The primary benchmark utilizes the DrivAerML dataset, comprising 500 parametrically varied vehicle geometries based on the DrivAer notchback model. These geometries were simulated using high-fidelity hybrid RANS-LES methods.
To rigorously test model generalizability, the authors proposed a specific 90/10 train-validation split. Rather than a random split, the data is sorted by drag force. As shown in the distribution analysis, the validation set is constructed to include "out-of-distribution" (OOD) samples from the extreme high and low ends of the drag spectrum. This approach ensures that models are evaluated on their ability to predict performance for designs that differ significantly from the average training sample.
Comparative Results
The framework was used to evaluate three state-of-the-art architectures: DoMINO (a neural operator), X-MeshGraphNet (a scalable GNN), and FIGConvNet (a factorized convolution network).
The results highlight a significant gap between average ML error and engineering utility. While all models achieved reasonable MSE, their performance in force prediction varied widely. DoMINO emerged as the top performer, achieving an $R^2$ of 0.98 for drag and 0.97 for lift. Its success is attributed to its ability to train directly on surface mesh data, avoiding interpolation errors common in point-cloud-based models.
Furthermore, models incorporating integrated loss terms (DoMINO and FIGConvNet) significantly outperformed those without, demonstrating that physics-informed constraints are essential for capturing global aerodynamic quantities. While all models captured general pressure trends, DoMINO was uniquely capable of resolving sharp pressure gradients at geometric discontinuities, a critical requirement for accurate drag estimation.
Conclusion and Future Directions
The PhysicsNeMo-CFD benchmarking framework provides the necessary "level playing field" for evaluating AI in aerodynamics. By prioritizing engineering-critical metrics and OOD generalizability, it bridges the gap between machine learning research and industrial application. Future work will expand the framework to include 3D volumetric flow evaluation, uncertainty quantification for safety-critical decisions, and the use of AI to accelerate traditional CFD solvers through intelligent initialization.
Summary updated