InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cache

Ao Wang, Jingyuan Zhang, Xiaolong Ma, Ali Anwar, Lukas Rupprecht, Dimitrios Skourtis, Vasily Tarasov, Feng Yan, Yue Cheng

Introduction

Modern web applications face a significant challenge in managing large-object data. Traditional In-Memory Object Caches (IMOCs) like Redis or Memcached are often cost-prohibitive for large files (MBs to GBs), such as Docker images or high-resolution media, because cloud providers charge for provisioned capacity regardless of actual usage. Furthermore, mixing large and small objects in the same cache leads to "cache churn," where large files frequently evict smaller, high-value ones.

InfiniCache addresses this gap by leveraging ephemeral serverless functions (e.g., AWS Lambda) to create a truly elastic, pay-per-use memory tier. By orchestrating thousands of these "stateless" functions, InfiniCache achieves up to 96x cost savings compared to traditional provisioned caches like AWS ElastiCache, while maintaining high performance for large-object workloads.

System Architecture

InfiniCache treats the temporary memory of serverless functions as a distributed cache pool. The architecture consists of three primary components:

InfiniCache architecture overview showing the interaction between the client library, proxy, and Lambda-based cache pools.
  1. Client Library: This component manages Erasure Coding (EC) using Reed-Solomon coding. It partitions large objects into chunks distributed across multiple Lambda functions, enabling fault tolerance and massive parallel I/O.
  2. Proxy: Serving as a "rendezvous" point, the proxy manages persistent connections to Lambda functions and coordinates data streaming.
  3. Lambda Runtime: A custom execution environment within each function manages cached chunks in memory. It employs Anticipatory Billed Duration Control, a timeout mechanism that keeps functions "warm" just long enough to serve requests within a billing cycle without incurring unnecessary costs.

To mitigate the ephemeral nature of serverless environments, InfiniCache utilizes a periodic delta-sync backup protocol, mirroring data to clones to ensure high availability.

Dataset and Evaluation

The system was evaluated using production traces from an IBM Docker registry (London and Dallas datacenters) collected over several months.

Analysis of the IBM Docker registry traces, showing object size distribution, byte footprint, access counts, and reuse intervals.

The trace analysis revealed a skewed distribution: while only 20% of objects are larger than 10MB, they account for 95% of the total storage footprint. This characteristic makes them ideal candidates for InfiniCache's elastic model. Synthetic microbenchmarks were also used to stress-test the system across various Lambda memory configurations and erasure coding settings.

Key Insights and Results

InfiniCache demonstrates significant advantages for large-object caching:

  • Cost Efficiency: It is 31x to 96x cheaper than AWS ElastiCache for production workloads by exploiting the pay-per-use FaaS billing model.
  • Parallelism over Provisioning: By aggregating the network bandwidth of hundreds of small Lambdas, InfiniCache can outperform traditional single-node caches for objects larger than 100MB.
  • Reliability: It maintains 95.4% data availability per hour despite the underlying ephemeral infrastructure, using redundancy to mitigate both data loss and "straggler" functions.
  • Scalability: Throughput scales linearly with the number of clients and Lambda nodes.

Limitations and Future Work

While powerful, InfiniCache is not a universal replacement for all caching needs. For high-frequency, small-object requests (e.g., >312K requests/hour), the per-invocation cost of serverless functions can exceed that of provisioned caches. Additionally, the system's efficiency is tied to current FaaS provider reclamation policies and billing models. Future work includes expanding portability to other cloud providers like Azure and Google Cloud, which have different networking and billing constraints.

Summary updated

Abstract

Internet-scale web applications are becoming increasingly storage-intensive and rely heavily on in-memory object caching to attain required I/O performance. We argue that the emerging serverless computing paradigm provides a well-suited, cost-effective platform for object caching. We present InfiniCache, a first-of-its-kind in-memory object caching system that is completely built and deployed atop ephemeral serverless functions. InfiniCache exploits and orchestrates serverless functions' memory resources to enable elastic pay-per-use caching. InfiniCache's design combines erasure coding, intelligent billed duration control, and an efficient data backup mechanism to maximize data availability and cost-effectiveness while balancing the risk of losing cached state and performance. We implement InfiniCache on AWS Lambda and show that it: (1) achieves 31 -- 96X tenant-side cost savings compared to AWS ElastiCache for a large-object-only production workload, (2) can effectively provide 95.4% data availability for each one hour window, and (3) enables comparative performance seen in a typical in-memory cache.
BibTeX
@inproceedings{wang2020infinicache,
  author = {Ao Wang and Jingyuan Zhang and Xiaolong Ma and Ali Anwar and Lukas Rupprecht and Dimitrios Skourtis and Vasily Tarasov and Feng Yan and Yue Cheng},
  title = {{InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cache}},
  booktitle = {FAST},
  year = {2020},
  url = {https://openreview.net/forum?id=nD_lF_40Or},
}
Figure