Unleashing In-network Computing on Scientific Workloads

Daehyeok Kim, Ankush Jain, Zaoxing Liu, George Amvrosiadis, Damian Hazen, Bradley W. Settlemyer, Vyas SekararXiv 2020 · OpenReview

TL;DR

This paper introduces NSinC, a novel closed-loop architecture designed to leverage in-network computing for High-Performance Computing (HPC) scientific workloads. By providing runtime feedback from the network to acceleration services, NSinC aims to overcome the limitations of existing open-loop approaches that struggle with the dynamic and data-intensive nature of scientific simulations.

What Problem Does This Paper Address?

Modern High-Performance Computing (HPC) relies on complex scientific simulations to model phenomena in fields like materials science, genomics, and astrophysics. These simulations generate massive amounts of data and involve intricate communication patterns between thousands of processing cores. While "in-network computing" – performing computations directly within network devices like switches – has shown promise for accelerating traditional datacenter applications (e.g., key-value stores, machine learning), it has not been effectively applied to HPC scientific workloads. The core challenge is that scientific workloads are highly dynamic, with data distributions and computational needs changing rapidly. Existing in-network computing approaches are largely "open-loop," meaning they lack the real-time feedback necessary to adapt to these rapid changes, rendering them ineffective for scientific applications.

Methods

The paper proposes NSinC (Network-accelerated ScIeNtific Computing), a closed-loop architecture. The key innovation is the integration of runtime feedback into in-network acceleration services. NSinC operates in three main components:

  1. Sketch-based Multidimensional Telemetry (Data Plane): To efficiently collect information from high-dimensional scientific data without overwhelming network devices, NSinC uses sketching algorithms. These algorithms provide approximate but resource-efficient ways to estimate metrics of interest (like data distribution, entropy) from complex data streams. The design includes a "merged sketch structure" and "merged compute" to reduce memory and computational overhead.
  2. Hierarchical Telemetry Controller (Control Plane): This component aggregates and analyzes the telemetry data. It employs a hierarchical design with local controllers in switches for fast, albeit potentially local, feedback, and a central controller for a more comprehensive, network-wide view. This allows for timely, less-accurate feedback when needed, complementing slower but more accurate global updates.
  3. Northbound APIs: These APIs expose the telemetry-based feedback to acceleration services, allowing them to dynamically reconfigure their behavior based on network conditions and workload characteristics.

Datasets

Not discussed in the paper. The paper focuses on analyzing canonical HPC applications and proposing an architecture, rather than evaluating it on specific datasets.

Key Insights and Ideas

  • Scientific workloads are fundamentally different: The dynamic nature, high data dimensionality, and tightly-coupled computing patterns of HPC scientific workloads make them ill-suited for existing, static, open-loop in-network computing solutions.
  • Closed-loop control is essential: To effectively accelerate scientific workloads, in-network services need real-time feedback to adapt to the rapidly changing state of the simulation.
  • Approximate telemetry is better than no telemetry: For time-sensitive HPC applications, timely but approximate information from sketching techniques is more valuable than slow, perfectly accurate data.
  • Hierarchical control balances speed and scope: A tiered approach to telemetry collection and processing (local vs. central) allows for rapid responses while still enabling network-wide insights.

Key Results

The paper presents a conceptual design and outlines potential benefits rather than quantitative results from a deployed system. The key "results" are the identification of opportunities and the proposed architecture:

  • Opportunities Identified:
    • Accelerating Load-balancing and Mesh Adaptivity: In-network telemetry can provide global data to enable more efficient re-balancing of computational loads and adaptive mesh refinement in simulations.
  • Accelerating Multidimensional Queries: Network-based samples can help construct efficient data partitioning functions and detect dynamic structures in high-dimensional data.
  • Accelerating Data Matching and Scientific Analytics: Domain-aware in-network caching and telemetry can speed up tasks like genome sequencing by providing custom matching rules and tracking successful alignments.
  • NSinC Architecture: The proposed NSinC architecture is designed to address the challenges of high-dimensional data and tight timing constraints in scientific workloads.

Limitations and Future Work

  • Limitations:
    • Preliminary Design: The paper acknowledges that the NSinC design is preliminary.
  • Network-wide Visibility on High-Dimensional Data: Obtaining system-wide visibility on high-dimensional data efficiently remains a challenge, as full statistics can be infeasible to collect.
  • Tight Timing Constraints: Coping with the extremely tight timing requirements for feedback loops in scientific workloads is a significant challenge.
  • Heterogeneity of Network Devices: Future solutions must account for a mix of programmable and legacy network devices.
  • Fault Tolerance: Ensuring the resilience of closed-loop control and telemetry to device failures is crucial.
  • Future Work:
    • Native Multidimensional Sketching: Developing sketching algorithms that are inherently sublinear in the number of dimensions for even greater efficiency.
  • Fault-tolerant Design: Creating mechanisms to make closed-loop control and telemetry resilient to failures.
  • Practical Implementation and Evaluation: Building and evaluating NSinC with concrete scientific applications to demonstrate its performance benefits.

Summary updated

Abstract

Many recent efforts have shown that in-network computing can benefit various datacenter applications. In this paper, we explore a relatively less-explored domain which we argue can benefit from in-network computing: scientific workloads in high-performance computing. By analyzing canonical examples of HPC applications, we observe unique opportunities and challenges for exploiting in-network computing to accelerate scientific workloads. In particular, we find that the dynamic and demanding nature of scientific workloads is the major obstacle to the adoption of in-network approaches which are mostly open-loop and lack runtime feedback. In this paper, we present NSinC (Network-accelerated ScIeNtific Computing), an architecture for fully unleashing the potential benefits of in-network computing for scientific workloads by providing closed-loop runtime feedback to in-network acceleration services. We outline key challenges in realizing this vision and a preliminary design to enable acceleration for scientific applications.
BibTeX
@misc{kim2020unleashing,
  author = {Daehyeok Kim and Ankush Jain and Zaoxing Liu and George Amvrosiadis and Damian Hazen and Bradley W. Settlemyer and Vyas Sekar},
  title = {{Unleashing In-network Computing on Scientific Workloads}},
  year = {2020},
  eprint = {2009.02457},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.02457},
}

Loading related content...

Figure