User-guided Page Merging for Memory Deduplication in Serverless Systems

Wei Qiu, Marcin Copik, Yun Wang, Alexandru Calotoiu, Torsten Hoefler

Introduction

In Function-as-a-Service (FaaS) environments, scalability often comes at the cost of memory efficiency. Scaling a function to thousands of instances involves duplicating the entire software stack—runtimes (e.g., Python), libraries (e.g., PyTorch), and large machine learning models—across every container. This redundancy leads to significant "memory bloat," wasting massive amounts of RAM.

While the Linux kernel provides Kernel Same-page Merging (KSM) to address this, its background scanning approach is too slow for the ephemeral nature of serverless tasks. Often, a function completes its execution before KSM can identify and merge duplicate pages. This inefficiency forces providers to prematurely terminate idle containers to reclaim RAM, resulting in frequent "cold starts" and lower overall performance.

System Design: User-Guided Page Merging (UPM)

To overcome the latency of background scanning, the authors propose User-Guided Page Merging (UPM). Instead of the kernel searching for duplicates, UPM allows applications or runtimes to proactively signal shareable memory regions.

Comparison of memory management without UPM (left) and with UPM (right).

UPM is implemented as a Linux kernel module with several key technical features:

  • The madvise Interface: UPM leverages the existing madvise system call. Runtimes can "advise" the kernel on specific memory regions that are candidates for merging.
  • Synchronous Deduplication: Upon receiving a hint, UPM immediately hashes the target pages. If a matching hash exists in the UPM table, a byte-by-byte comparison is performed. If identical, the pages are merged into a single physical page using copy-on-write (CoW) semantics.
  • Technology Agnostic: As a kernel-level solution, UPM works across various container technologies, including Docker and microVMs.

Evaluation and Workload Analysis

The researchers evaluated UPM using the Serverless Benchmark Suite (SeBS), focusing on four diverse workloads: dynamic-html, thumbnailer, image-recognition (ML inference), and dna-visualization.

Analysis of memory sharing potential across different serverless workloads.

A key insight from the evaluation is that while standard filesystems (like OverlayFS) handle some sharing, they cannot deduplicate "anonymous" memory. This is particularly critical for machine learning workloads, where model weights—often hundreds of megabytes—reside in anonymous memory. As shown in the workload analysis, image-recognition has a high percentage of shareable anonymous memory that UPM can uniquely target, whereas smaller functions often have their shareable memory already handled by the filesystem.

Key Results

UPM significantly enhances the density of serverless platforms by lowering the memory floor for concurrent functions:

  • Memory Footprint Reduction: For AlexNet inference, UPM reduced the Proportional Set Size (PSS) by 55% across 16 concurrent containers. ResNet saw a 21% reduction.
  • Increased Container Density: Within a fixed memory budget, UPM enabled the execution of 21 additional AlexNet containers, more than doubling the original capacity.
  • Low Overhead: The deduplication process is highly efficient. For a 100MB model, the madvise call takes only 0.1 to 0.3 seconds, a negligible fraction of the total initialization time for heavy runtimes like PyTorch.

Conclusion and Future Work

UPM demonstrates that user-guided hints can effectively bridge the gap between memory efficiency and serverless performance. While it currently requires manual integration of madvise calls, future work aims to automate hint generation through profiling or compiler analysis. Additionally, while UPM's opt-in nature provides a security advantage over KSM, further research is needed to mitigate potential side-channel attacks in multi-tenant environments.

Summary updated

BibTeX
@inproceedings{qiu2023user,
  author = {Wei Qiu and Marcin Copik and Yun Wang and Alexandru Calotoiu and Torsten Hoefler},
  title = {{User-guided Page Merging for Memory Deduplication in Serverless Systems}},
  booktitle = {IEEE Big Data},
  year = {2023},
  url = {https://openreview.net/forum?id=r2YTU2QJgv},
}

Loading related content...

Figure