Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning

Jinsun Yoo, ChonLam Lao, Lianjie Cao, Bob Lantz, Minlan Yu, Tushar Krishna, Puneet SharmaarXiv 2025 · DOI · arXiv

Introduction

As machine learning models scale to tens of thousands of GPUs, the network infrastructure—the switches, NICs, and cables connecting them—has become a massive bottleneck. Testing this infrastructure is notoriously difficult due to the "GPU Tax" and the "Simulation Gap." Traditional simulators often fail to model proprietary hardware accurately and cannot predict real-world "gremlins" like a degrading NIC or a misconfigured buffer.

The gap between simulated bandwidth and real-world ground truth during a hardware degradation event.

The GENIE Framework

GENIE (GPUs Eliminated for Network Infrastructure Examination) is a testing framework that allows researchers to evaluate large-scale ML network performance on real hardware without needing a single expensive GPU. The core innovation is the decoupling of computation from communication.

The framework consists of two main parts:

  • The Workload Feeder (Modified ASTRA-sim): Instead of running heavy math on a GPU, GENIE uses the ASTRA-sim simulator to manage the logic of the ML job by reading a Chakra graph. For compute tasks, it simply "sleeps" for the duration a real GPU would take.
  • The Traffic Generator: When it hits a communication task, it triggers the host CPU to inject actual RDMA traffic into the network. This traffic traverses real NICs and switches, mimicking the size and timing of real ML traffic.
The GENIE architecture, illustrating the flow from workload graphs to real network calls.

Key Insights and Results

GENIE provides realism over pure simulation by capturing hardware anomalies that simulators miss. Because it is dependency-aware, if one part of the network slows down, the "simulated" training job slows down in response, capturing the ripple effect of network delays on total training time.

Initial results show that GENIE can:

  • Capture the impact of hardware events like packet drops and backpressure.
  • Replicate the behavior of distributed training ranks across multiple nodes.
  • Be ported to various high-performance network fabrics (Infiniband, RoCE, Slingshot) without rewriting ML model code.

Limitations and Future Work

Future work focuses on validating how well a CPU can mimic the specific burstiness and PCIe-interaction patterns of a GPU. Currently, GENIE focuses on inter-node network traffic and does not yet model intra-node interconnects like NVLink. The team plans to build out the full implementation and run it against large-scale GPU "ground truth" data to prove its accuracy across different failure scenarios.

Summary updated

Abstract

This paper lays the foundation for Genie, a testing framework that captures the impact of real hardware network behavior on ML workload performance, without requiring expensive GPUs. Genie uses CPU-initiated traffic over a hardware testbed to emulate GPU to GPU communication, and adapts the ASTRA-sim simulator to model interaction between the network and the ML workload.
BibTeX
@misc{yoo2025towards,
  author = {Jinsun Yoo and ChonLam Lao and Lianjie Cao and Bob Lantz and Minlan Yu and Tushar Krishna and Puneet Sharma},
  title = {{Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning}},
  year = {2025},
  eprint = {2504.20854},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.20854},
}
Figure