Introduction
Modern web applications face a significant challenge in managing large-object data. Traditional In-Memory Object Caches (IMOCs) like Redis or Memcached are often cost-prohibitive for large files (MBs to GBs), such as Docker images or high-resolution media, because cloud providers charge for provisioned capacity regardless of actual usage. Furthermore, mixing large and small objects in the same cache leads to "cache churn," where large files frequently evict smaller, high-value ones.
InfiniCache addresses this gap by leveraging ephemeral serverless functions (e.g., AWS Lambda) to create a truly elastic, pay-per-use memory tier. By orchestrating thousands of these "stateless" functions, InfiniCache achieves up to 96x cost savings compared to traditional provisioned caches like AWS ElastiCache, while maintaining high performance for large-object workloads.
System Architecture
InfiniCache treats the temporary memory of serverless functions as a distributed cache pool. The architecture consists of three primary components:
- Client Library: This component manages Erasure Coding (EC) using Reed-Solomon coding. It partitions large objects into chunks distributed across multiple Lambda functions, enabling fault tolerance and massive parallel I/O.
- Proxy: Serving as a "rendezvous" point, the proxy manages persistent connections to Lambda functions and coordinates data streaming.
- Lambda Runtime: A custom execution environment within each function manages cached chunks in memory. It employs Anticipatory Billed Duration Control, a timeout mechanism that keeps functions "warm" just long enough to serve requests within a billing cycle without incurring unnecessary costs.
To mitigate the ephemeral nature of serverless environments, InfiniCache utilizes a periodic delta-sync backup protocol, mirroring data to clones to ensure high availability.
Dataset and Evaluation
The system was evaluated using production traces from an IBM Docker registry (London and Dallas datacenters) collected over several months.
The trace analysis revealed a skewed distribution: while only 20% of objects are larger than 10MB, they account for 95% of the total storage footprint. This characteristic makes them ideal candidates for InfiniCache's elastic model. Synthetic microbenchmarks were also used to stress-test the system across various Lambda memory configurations and erasure coding settings.
Key Insights and Results
InfiniCache demonstrates significant advantages for large-object caching:
- Cost Efficiency: It is 31x to 96x cheaper than AWS ElastiCache for production workloads by exploiting the pay-per-use FaaS billing model.
- Parallelism over Provisioning: By aggregating the network bandwidth of hundreds of small Lambdas, InfiniCache can outperform traditional single-node caches for objects larger than 100MB.
- Reliability: It maintains 95.4% data availability per hour despite the underlying ephemeral infrastructure, using redundancy to mitigate both data loss and "straggler" functions.
- Scalability: Throughput scales linearly with the number of clients and Lambda nodes.
Limitations and Future Work
While powerful, InfiniCache is not a universal replacement for all caching needs. For high-frequency, small-object requests (e.g., >312K requests/hour), the per-invocation cost of serverless functions can exceed that of provisioned caches. Additionally, the system's efficiency is tied to current FaaS provider reclamation policies and billing models. Future work includes expanding portability to other cloud providers like Azure and Google Cloud, which have different networking and billing constraints.
Summary updated