Overcoming On-Chip Memory Constraints in Programmable Switches

151 concepts142 papers1 proposal

Imagine a highway toll booth tasked with inspecting every vehicle, logging its license plate, and cross-referencing a database of millions of registered cars—all while traffic rushes past at 100 miles per hour without slowing down. If the booth takes even a microsecond too long to process a single car, traffic backs up for miles.

This is the fundamental challenge of modern programmable network switches. To process packets at line rate (often 100 Gbps or more), switches rely on specialized hardware pipelines. However, this extreme speed comes at a steep physical cost: the switch's high-speed on-chip memory (SRAM and TCAM) is strictly limited, often to just a few megabytes per chip, as noted by Vinh The Lam et al. and Xiaoxin Su et al..

As networks scale, maintaining fine-grained state for millions of concurrent connections within these rigid boundaries becomes a major bottleneck.

The Hardware Bottleneck

Why can't we simply deploy standard software algorithms to the data plane? The answer lies in the hardware architecture of high-speed switching ASICs. As explained by Lu Tang et al., programmable switches feature highly restricted instruction sets and lack the computational flexibility of general-purpose CPUs. They cannot execute arbitrary loops, perform complex divisions, or maintain dynamic data structures like heaps or traditional hash tables.

Furthermore, Damu Ding et al. emphasize that hardware pipelines allow only a limited number of memory accesses per packet pass. If a network application attempts to track every individual flow naively, it quickly exhausts the available register space or flow table capacity. This resource exhaustion can lead to severe performance degradation or an inability to enforce critical security and routing policies, as shown by Moreno Ambrosin et al..

To overcome these constraints, researchers have developed three primary strategies: compact probabilistic data structures, intelligent rule management, and hybrid workload partitioning.

Compact State via Probabilistic Sketching

Instead of recording an exact ledger of every network event, streaming algorithms can trade a small, bounded amount of accuracy for massive memory savings. Probabilistic data structures, or sketches, compress traffic metadata into sublinear memory footprints by using independent hash functions to map flow identifiers to small arrays of counters.

Because sketches do not store the original flow keys (such as IP addresses), they fit entirely within high-speed SRAM. For example, the BACON sketch developed by Damu Ding et al. tracks traffic cardinality at line rate to identify volumetric DDoS victims without overwhelming switch memory. To optimize memory even further, Samson Zhou et al. introduced Lean Algorithms, which exploit a mathematical property called "flow-additivity" to isolate heavy-hitting or problematic flows using as little as 40KB of memory.

[Incoming Packet] ---> Hash H1(Flow ID) ---> [ Counter Array Row 1 ]
                  ---> Hash H2(Flow ID) ---> [ Counter Array Row 2 ]
                  ---> Hash H3(Flow ID) ---> [ Counter Array Row 3 ]
                  (No flow keys stored; sublinear memory footprint)

Other designs adapt dynamically to changing network conditions to maximize the utility of every bit. Yuhan Wu designed ChameleMon, which leverages a FermatSketch to scale its memory requirements based on the number of flows experiencing performance degradation rather than the total volume of traffic. Similarly, Yiyan Qi et al. developed QSketch, which quantizes continuous streaming variables into compact 8-bit integers, reducing the memory footprint by up to 8x while accelerating update speeds. These compact representations allow complex multi-metric tracking to operate entirely within the constraints of real-time packet processing pipelines, as demonstrated by Eran Assaf et al..

Intelligent Rule Caching and Aggregation

When exact matching is non-negotiable—such as enforcing access control lists or fine-grained routing policies—switches must rely on Ternary Content-Addressable Memory (TCAM). TCAM allows for parallel, wildcard matching across rules, but it is exceptionally expensive and power-hungry, meaning physical switches can only host a limited number of rules before running out of space, as noted by Baochun Li et al..

To prevent TCAM exhaustion, researchers treat the switch memory as a cache and the centralized controller as the backing store. However, traditional packet-driven caching policies (like Least Recently Used) suffer from a total performance collapse when hit with high-volume, unpredictable traffic. To address this, He Li et al. designed Flow-Driven Rule Caching (FDRC), which leverages traffic predictability and a global network view to optimize rule placement, maintaining a hit ratio of up to 99.4% even under strict hardware limits.

Complementing caching is rule aggregation. Cosmin Caba demonstrated that by tagging flows at the server level and aggregating forwarding rules based on traffic classes and destination subnets, networks can slash the switch memory footprint by over 90%. This allows software-defined controllers to manage massive data transfers without exceeding hardware rule capacities.

Hybrid Architecture and Workload Partitioning

A third approach acknowledges that the data plane cannot act alone. Because programmable switches cannot execute complex logic like deep learning models or heavy database joins, modern architectures partition workloads between the fast data plane and a richer, slower control plane or host infrastructure. The switch acts as an agile, line-rate filter that "prunes" or aggregates data, ensuring that only a highly relevant subset of traffic ever reaches the host processors.

+-------------------------------------------------------------+
|                      PROGRAMMABLE SWITCH                    |
|  [Terabit-Scale Traffic] ---> [Line-Rate Pruning/Filtering] |
+------------------------------------------------------+------+
                                                       |
                                        (Filtered Data Only)
                                                       v
                               +-----------------------+------+
                               |          HOST / CPU          |
                               |   [Complex Stateful Logic]   |
                               +------------------------------+

For instance, the database acceleration framework Cheetah, designed by Muhammad Tirmazi et al., offloads data pruning queries directly to the switch. By filtering out irrelevant tuples at line rate before they reach the master server, the authors reduce query completion times significantly while respecting the switch's tiny memory capacity.

In security applications, Poseidon (Menghao Zhang et al.) utilizes programmable switches to handle the brunt of terabit-scale volumetric DDoS attacks at line rate, shielding slower software-based defense layers from saturation. Similarly, for web traffic load balancing, Xiaoyi Shi et al. developed Miresga, which offloads massive, long-lived "elephant flows" to the switch hardware while routing complex, short-lived connections to software servers. This hybrid division of labor ensures that the switch's static random-access memory is reserved exclusively for the traffic that benefits most from hardware acceleration.

Even transient data can be optimized: Ivan Beschastnikh et al. demonstrated with PayloadPark that switches can temporarily buffer packet payloads in memory while forwarding only the headers to network function servers, reducing PCIe bus load without exhausting the switch's transient storage capacity.

Formalizing the Constraints

To guide the systematic design of these memory-restricted algorithms, researchers have formalized these physical boundaries into theoretical frameworks. Ran Ben-Basat et al. introduced the $\mu$-CONGEST model, a distributed computing framework that explicitly incorporates the strict memory limitations of real-world programmable switches. By establishing tight mathematical trade-offs between local memory capacity and the number of communication rounds required to solve fundamental network problems, this model provides a foundational framework for proving what can—and cannot—be computed directly within the network fabric.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 127 papers
  2. Version 1 · 2026-10-04 · 117 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_switch_memory_limitations_c2afcd38,
  title = {switch memory limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/switch-memory-limitations/rdr_c2afcd38f67f4ec280f027b0d072c19f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_switch_memory_limitations_c2afcd38_v2,
  title = {switch memory limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/switch-memory-limitations/rdr_c2afcd38f67f4ec280f027b0d072c19f/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake