Abstract
Serverless computing is increasingly popular because of its lower cost and easier deployment. Several cloud service providers (CSPs) offer serverless computing on their public clouds, but it may bring the vendor lock-in risk. To avoid this limitation, many open-source serverless platforms come out to allow developers to freely deploy and manage functions on self-hosted clouds. However, building effective functions requires much expertise and thorough comprehension of platform frameworks and features that affect performance. It is a challenge for a service developer to differentiate and select the appropriate serverless platform for different demands and scenarios. Thus, we elaborate the frameworks and event processing models of four popular open-source serverless platforms and identify their salient idiosyncrasies. We analyze the root causes of performance differences between different service exporting and auto-scaling modes on those platforms. Further, we provide several insights for future work, such as auto-scaling and metric collection.
Introduction
Serverless computing, or Function-as-a-Service (FaaS), has gained traction due to its "pay-as-you-go" model and simplified deployment. However, proprietary solutions often lead to vendor lock-in. Open-source alternatives like Nuclio, OpenFaaS, Knative, and Kubeless allow deployment on self-hosted Kubernetes clusters, but they differ significantly in architecture and performance. This paper provides a systematic comparison of these platforms to help developers navigate their unique characteristics.
Methodology
The researchers evaluated the platforms using the CloudLab testbed with a Kubernetes v1.20.0 stack. The study combined architectural decomposition with performance benchmarking using wrk for HTTP workloads and tcpdump for latency analysis. They specifically tested workload-based scaling (Requests Per Second) and resource-based scaling (CPU/Memory) to understand how each platform handles pressure.
Architectural Idiosyncrasies
Internal design choices drastically affect performance:
- The "Sidecar" Tax: Knative uses a "queue-proxy" sidecar container. While feature-rich, this introduces higher communication overhead compared to Nuclio’s model, which uses a single event listener to manage worker processes.
- Process Management: Kubeless often uses a "fork-per-request" model, leading to initiation delays of nearly 5ms, whereas Nuclio’s pre-warmed workers respond in approximately 0.63ms.
- Scaling Latency: OpenFaaS employs a complex alerting chain (Gateway → Prometheus → AlertManager → Gateway → Faas-netes), resulting in a slow 26-second response to reach 10 pods, compared to Knative's 12 seconds.
- Queuing and Reliability: Nuclio and Knative allow internal queuing. Kubeless lacks this, causing it to drop connections under high load rather than buffering them.
Key Results
Nuclio emerged as the overall performance leader, providing the highest throughput and lowest 99th-percentile latency. Its ability to manage multiple workers within a single pod makes it exceptionally efficient for multi-core environments.
As illustrated in the performance data, Nuclio maintains superior throughput even as concurrency scales to 1000 requests. In contrast, Kubeless performance drops significantly under high concurrency due to its lack of internal request queuing.
Regarding resource efficiency, Knative is the only platform supporting "scale-to-zero" in its standard version, making it ideal for infrequent traffic despite the cold-start penalty. For auto-scaling, Nuclio triggered Kubernetes HPA thresholds faster than competitors because its architecture is more "CPU hungry," leading to more rapid scaling responses in resource-bound scenarios.
Conclusion and Future Work
Choosing the right serverless platform depends on specific needs: Nuclio is optimal for high-performance requirements, while Knative is best for resource-constrained, sporadic workloads. Future research should focus on reducing metric collection overhead in large-scale deployments and developing automated scaling algorithms that adapt to specific function runtime characteristics.
Summary updated