User-guided Page Merging for Memory Deduplication in Serverless Systems

Wei Qiu, Marcin Copik, Yun Wang, Alexandru Calotoiu, Torsten Hoefler

Introduction

In Function-as-a-Service (FaaS) environments, scalability often comes at the cost of memory efficiency. Scaling a function to thousands of instances involves duplicating the entire software stack—runtimes (e.g., Python), libraries (e.g., PyTorch), and large machine learning models—across every container. This redundancy leads to significant "memory bloat," wasting massive amounts of RAM.

While the Linux kernel provides Kernel Same-page Merging (KSM) to address this, its background scanning approach is too slow for the ephemeral nature of serverless tasks. Often, a function completes its execution before KSM can identify and merge duplicate pages. This inefficiency forces providers to prematurely terminate idle containers to reclaim RAM, resulting in frequent "cold starts" and lower overall performance.

System Design: User-Guided Page Merging (UPM)

To overcome the latency of background scanning, the authors propose User-Guided Page Merging (UPM). Instead of the kernel searching for duplicates, UPM allows applications or runtimes to proactively signal shareable memory regions.

Comparison of memory management without UPM (left) and with UPM (right).

UPM is implemented as a Linux kernel module with several key technical features:

  • The madvise Interface: UPM leverages the existing madvise system call. Runtimes can "advise" the kernel on specific memory regions that are candidates for merging.
  • Synchronous Deduplication: Upon receiving a hint, UPM immediately hashes the target pages. If a matching hash exists in the UPM table, a byte-by-byte comparison is performed. If identical, the pages are merged into a single physical page using copy-on-write (CoW) semantics.
  • Technology Agnostic: As a kernel-level solution, UPM works across various container technologies, including Docker and microVMs.

Evaluation and Workload Analysis

The researchers evaluated UPM using the Serverless Benchmark Suite (SeBS), focusing on four diverse workloads: dynamic-html, thumbnailer, image-recognition (ML inference), and dna-visualization.

Analysis of memory sharing potential across different serverless workloads.

A key insight from the evaluation is that while standard filesystems (like OverlayFS) handle some sharing, they cannot deduplicate "anonymous" memory. This is particularly critical for machine learning workloads, where model weights—often hundreds of megabytes—reside in anonymous memory. As shown in the workload analysis, image-recognition has a high percentage of shareable anonymous memory that UPM can uniquely target, whereas smaller functions often have their shareable memory already handled by the filesystem.

Key Results

UPM significantly enhances the density of serverless platforms by lowering the memory floor for concurrent functions:

  • Memory Footprint Reduction: For AlexNet inference, UPM reduced the Proportional Set Size (PSS) by 55% across 16 concurrent containers. ResNet saw a 21% reduction.
  • Increased Container Density: Within a fixed memory budget, UPM enabled the execution of 21 additional AlexNet containers, more than doubling the original capacity.
  • Low Overhead: The deduplication process is highly efficient. For a 100MB model, the madvise call takes only 0.1 to 0.3 seconds, a negligible fraction of the total initialization time for heavy runtimes like PyTorch.

Conclusion and Future Work

UPM demonstrates that user-guided hints can effectively bridge the gap between memory efficiency and serverless performance. While it currently requires manual integration of madvise calls, future work aims to automate hint generation through profiling or compiler analysis. Additionally, while UPM's opt-in nature provides a security advantage over KSM, further research is needed to mitigate potential side-channel attacks in multi-tenant environments.

Summary updated

Abstract

Serverless computing is an emerging cloud paradigm that offers an elastic and scalable allocation of computing resources with pay-as-you-go billing. In the Function-as-a-Service (FaaS) programming model, applications comprise short-lived and stateless serverless functions executed in isolated containers or microVMs, which can quickly scale to thousands of instances and process terabytes of data. This flexibility comes at the cost of duplicated runtimes, libraries, and user data spread across many function instances, and cloud providers do not utilize this redundancy. The memory footprint of serverless forces removing idle containers to make space for new ones, which decreases performance through more cold starts and fewer data caching opportunities.We address this issue by proposing deduplicating memory pages of serverless workers with identical content, based on the content-based page-sharing concept of Linux Kernel Same-page Merging (KSM). We replace the background memory scanning process of KSM, as it is too slow to locate sharing candidates in short-lived functions. Instead, we design User-Guided Page Merging (UPM), a built-in Linux kernel module that leverages the madvise system call: we enable users to advise the kernel of memory areas that can be shared with others. We show that UPM reduces memory consumption by up to 55% on 16 concurrent containers executing a typical image recognition function, more than doubling the density for containers of the same function that can run on a system.
BibTeX
@inproceedings{qiu2023user,
  author = {Wei Qiu and Marcin Copik and Yun Wang and Alexandru Calotoiu and Torsten Hoefler},
  title = {{User-guided Page Merging for Memory Deduplication in Serverless Systems}},
  booktitle = {IEEE Big Data},
  year = {2023},
  url = {https://openreview.net/forum?id=r2YTU2QJgv},
}

Loading related content...

Figure