A Benchmarking Framework for AI models in Automotive Aerodynamics

Kaustubh Tangsali, Rishikesh Ranade, Mohammad Amin Nabian, Alexey Kamenev, Peter Sharpe, Neil Ashton, Ram Cherukuri, Sanjay ChoudhryarXiv 2025 · DOI · arXiv

Introduction

Computational Fluid Dynamics (CFD) is the cornerstone of modern automotive aerodynamic design, providing high-fidelity simulations of airflow around vehicle geometries. However, the computational cost of these simulations—often requiring days on high-performance clusters—limits the speed of iterative design. AI-based surrogate models offer a promising alternative, capable of providing near-instantaneous feedback. Despite this potential, the field has historically lacked a standardized benchmarking framework. Current research often relies on disparate datasets and standard machine learning metrics like Mean Squared Error (MSE), which can fail to capture critical engineering quantities such as drag and lift coefficients. To address this, researchers have introduced a standardized benchmarking suite within the open-source PhysicsNeMo-CFD library.

The PhysicsNeMo-CFD Framework

The proposed framework, integrated into NVIDIA's PhysicsNeMo ecosystem, shifts the focus from generic ML metrics to physically-grounded engineering evaluations. It provides tools to compute aerodynamic forces by integrating predicted surface pressure and shear stress fields. Furthermore, it evaluates design trends using Spearman rank correlation coefficients and analyzes 1D centerline pressure distributions, which are vital for understanding local flow features.

A key technical challenge in automotive AI is the mesh-to-point-cloud transition. While traditional CFD relies on complex meshes, many AI models operate on point clouds. The framework includes utilities to validate models directly on uniformly sampled point clouds derived from STL files, ensuring that the evaluation remains relevant to the original geometric design.

Benchmarking Methodology

The primary benchmark utilizes the DrivAerML dataset, comprising 500 parametrically varied vehicle geometries based on the DrivAer notchback model. These geometries were simulated using high-fidelity hybrid RANS-LES methods.

To rigorously test model generalizability, the authors proposed a specific 90/10 train-validation split. Rather than a random split, the data is sorted by drag force. As shown in the distribution analysis, the validation set is constructed to include "out-of-distribution" (OOD) samples from the extreme high and low ends of the drag spectrum. This approach ensures that models are evaluated on their ability to predict performance for designs that differ significantly from the average training sample.

Figure 1: Distribution of aerodynamic forces across the dataset, highlighting the selection of extreme values for the validation set to test model generalizability.

Comparative Results

The framework was used to evaluate three state-of-the-art architectures: DoMINO (a neural operator), X-MeshGraphNet (a scalable GNN), and FIGConvNet (a factorized convolution network).

The results highlight a significant gap between average ML error and engineering utility. While all models achieved reasonable MSE, their performance in force prediction varied widely. DoMINO emerged as the top performer, achieving an $R^2$ of 0.98 for drag and 0.97 for lift. Its success is attributed to its ability to train directly on surface mesh data, avoiding interpolation errors common in point-cloud-based models.

Figure 2: Comprehensive evaluation of the DoMINO model, showing predicted surface pressure and velocity fields alongside error maps and 1D centerline pressure distributions.

Furthermore, models incorporating integrated loss terms (DoMINO and FIGConvNet) significantly outperformed those without, demonstrating that physics-informed constraints are essential for capturing global aerodynamic quantities. While all models captured general pressure trends, DoMINO was uniquely capable of resolving sharp pressure gradients at geometric discontinuities, a critical requirement for accurate drag estimation.

Conclusion and Future Directions

The PhysicsNeMo-CFD benchmarking framework provides the necessary "level playing field" for evaluating AI in aerodynamics. By prioritizing engineering-critical metrics and OOD generalizability, it bridges the gap between machine learning research and industrial application. Future work will expand the framework to include 3D volumetric flow evaluation, uncertainty quantification for safety-critical decisions, and the use of AI to accelerate traditional CFD solvers through intelligent initialization.

Summary updated

BibTeX
@misc{tangsali2025benchmarking,
  author = {Kaustubh Tangsali and Rishikesh Ranade and Mohammad Amin Nabian and Alexey Kamenev and Peter Sharpe and Neil Ashton and Ram Cherukuri and Sanjay Choudhry},
  title = {{A Benchmarking Framework for AI models in Automotive Aerodynamics}},
  year = {2025},
  eprint = {2507.10747},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.10747},
}

Loading related content...

Figure