Topology Aware Resource Management for Non Uniform Memory Access

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2149 papers218 conceptsfrozen 2026-10-04

In the early days of multi-core computing, developers could treat a processor's memory as a single, uniform pool. Whether a thread ran on Core 0 or Core 15, the time it took to fetch data from RAM was essentially the same. This is Symmetric Multi-Processing (SMP).

However, as core counts exploded, this uniform model hit a physical wall. Modern servers now distribute memory controllers across multiple CPU sockets or "chiplets." This creates a Non-Uniform Memory Access (NUMA) architecture: a core can access its local memory bank very quickly, but accessing memory attached to a different socket requires traversing an interconnect (like Intel’s UPI or AMD’s Infinity Fabric), which significantly increases latency and consumes precious interconnect bandwidth.

The research direction across these papers focuses on topology-aware orchestration—the art of aligning software execution with the physical "map" of the hardware to minimize these hidden costs.

The Cost of Crossing the Interconnect

The fundamental problem is that standard operating system schedulers often prioritize "fairness" or "load balancing" over physical locality. As Hong Xu and Zhixiong Niu demonstrated in Benchmarking NFV Software Dataplanes, when high-speed network functions are scattered across sockets, memory contention and NUMA effects can cause performance to "plummet."

When a thread on Socket A requests data residing in Socket B's memory, the system incurs a "remote access" penalty. In disaggregated systems using technologies like Compute Express Link (CXL), Chen Chen notes in Next-Gen Interconnection Systems with Compute Express Link that these cross-boundary latencies can double the access time compared to local connections.

Strategy 1: Thread Pinning and Data Partitioning

The most direct solution is to "pin" threads to specific cores and ensure their data is co-located in the same NUMA node. This is the core philosophy behind the knor library. In NUMA-optimized In-memory and Semi-external-memory Parameterized Clustering, Disa Mhembere shows that by binding threads to nodes and partitioning data locally, systems can achieve a six-fold performance increase over architecture-oblivious implementations.

Similarly, Shuhao Zhang and Bingsheng He developed TStream, which uses a shared-nothing task pool. By hashing operations to specific cores, Scaling Consistent Stateful Stream Processing on Shared-Memory Multicore Architectures ensures that the processing logic and the data it modifies never leave the same local memory domain.

Strategy 2: Sub-NUMA and Chiplet Awareness

Modern hardware has become even more complex with the advent of chiplets. A single NUMA node might now contain multiple "clusters" of cores, each with its own Last-Level Cache (LLC) slice. Traditional NUMA-aware scheduling is too coarse for this; it treats the entire node as a uniform block.

The ARCAS: Adaptive Runtime System for Chiplet-Aware Scheduling framework addresses this by recognizing a three-tier hierarchy: intra-chiplet, inter-chiplet (within one node), and cross-NUMA. By prioritizing "intra-chiplet" placement, ARCAS minimizes the latency of sharing data through the cache, which is even faster than local DRAM. This granularity is essential for data-intensive tasks where, as Wayne Luk argues in Hardware-Aware Execution, data must be replicated across domains to prevent memory bus contention.

Strategy 3: Managing Heterogeneous Interconnects

This topology challenge extends beyond CPUs to GPUs and disaggregated memory. In multi-GPU systems, Jieyang Chen explains in MSREP: A Fast yet Light Sparse Matrix Framework for Multi-GPU Systems that ignoring the physical mapping between CPU sockets and specific accelerators leads to interconnect congestion.

For Large Language Models (LLMs), where memory bandwidth is the primary bottleneck, researchers are moving toward manual control. In V-Seek: Accelerating LLM Reasoning on Open-hardware Server-class RISC-V Platforms, Luca Benini demonstrates that disabling the OS's automatic NUMA balancing in favor of manual memory interleaving prevents the overhead of "helpful" but costly thread migrations, leading to more stable inference throughput.

Why It Works: The Locality Principle

The success of these diverse systems—from VMAgent for VM scheduling to Beluga for CXL-based KV caches—boils down to a single intuition: moving computation is cheaper than moving data.

By making the software "aware" of the physical distance between a core and a byte of data, researchers are transforming the "ordering curse" of complex hardware into a performance blessing. The reader should understand that in the many-core era, the fastest algorithm is often the one that travels the shortest physical distance across the silicon.

Papers in this version

Showing 60 of 149 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 140 papers
  2. Version 1 · 2026-10-04 · 149 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_memory_locality_optimization_747ee221,
  title = {memory locality optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/memory-locality-optimization/rdr_747ee221d1e640a0887dd2656539537e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_topology_aware_resource_management_for_n_747ee221_v1,
  title = {Topology Aware Resource Management for Non Uniform Memory Access},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/topology-aware-resource-management-for-non-uniform-memory-access/rdr_747ee221d1e640a0887dd2656539537e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}