ACiS: Complex Processing in the Switch Fabric

Pouya Haghi, Anqi Guo, Tong Geng, Anthony Skjellum, Martin C. HerbordtarXiv 2025 · DOI · arXiv

Abstract

High-Performance Computing (HPC) and distributed AI systems are increasingly limited by communication latency. The ACiS (Advanced Computing in the Switch) framework addresses this by moving complex, application-level computations directly into the network switch fabric. By extending the Protocol Independent Switch Architecture (PISA) with reconfigurable hardware plugins, ACiS supports user-defined, stateful, and fused collective operations. Experimental results demonstrate up to a 10.1x speedup for Graph Convolutional Networks (GCNs) and significant latency reductions for fused MPI collectives.

Introduction

As distributed clusters scale, the network often becomes the primary bottleneck. Historically, the industry has attempted to mitigate this via SmartNICs at the network edge or basic processing in the switch core (e.g., Mellanox SHArP). However, these solutions are often rigid, supporting only simple scalar operations. When applications utilize complex data structures, loops, or chained operations, data must return to host CPUs, incurring significant latency penalties. ACiS addresses this gap by providing a flexible, programmable architecture for high-complexity custom computations within the switch.

Architectural Framework

The ACiS framework introduces a taxonomy of in-switch computing complexity, ranging from basic stream transformations (Type 0) to fused collectives (Type 4) that chain multiple communication and computation steps.

ACiS architectural integration within the PISA pipeline

To implement this, ACiS modifies the standard PISA pipeline by adding a dedicated payload processing path. This path utilizes a Coarse-Grained Reconfigurable Array (CGRA) or FPGA-based accelerator equipped with SIMD processing units and dedicated memory interfaces. This hardware is supported by an LLVM-based source-to-source compiler that translates standard MPI application code into switch-executable binaries, ensuring the acceleration is largely transparent to the programmer.

Performance Evaluation

The system was evaluated using Graph Convolutional Network (GCN) datasets (PPI, Citeseer, Pubmed, ogbn-mag, ogbn-products) and HPC proxy applications like the NAS Parallel Benchmarks. ACiS consistently outperformed baseline Intel Skylake (SKX) CPU clusters.

GCN performance comparison: SKX vs. ACiS

For GCN workloads, ACiS achieved an average 3.4x speedup across all datasets, with the largest dataset (ogbn-products) showing a 10.1x improvement. Furthermore, for Type 4 "fused collectives" common in finite element analysis (e.g., Allgather_op_Allgatherv), ACiS reduced average latency by approximately 1.98x compared to traditional Python-based MPI (MPI4py).

Conclusion

ACiS demonstrates that moving computation to the center of the network—where paths cross—can drastically reduce data movement and latency. By utilizing reconfigurable hardware, switches can adapt to specific application needs at runtime. Future work includes expanding the library of offloadable MPI patterns and refining the ExaMPI implementation for seamless, transparent integration into existing HPC workflows.

Summary updated

BibTeX
@misc{haghi2025acis,
  author = {Pouya Haghi and Anqi Guo and Tong Geng and Anthony Skjellum and Martin C. Herbordt},
  title = {{ACiS: Complex Processing in the Switch Fabric}},
  year = {2025},
  eprint = {2501.18749},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.18749},
}
Figure