Abstract
Web applications traffic demand fluctuates widely and usually unpredictably. The common practice of provisioning a fixed capacity would either result in unsatisfied customers or waste valuable capital investment. By leveraging an infrastructure cloud's on-demand, pay-per-use capabilities, we can match system capacity with demand in real time. Although static client-side load balancing is common, it fails in dynamic clusters where nodes and policies change at runtime. This paper proposes a dynamic client-side load balancing mechanism integrating distributed cluster-view maintenance and control flow piggybacking. Implemented in the PKUAS application server, this mechanism supports dynamic node membership and adaptive policy adjustment with high scalability.
Introduction
In traditional web architectures, load balancing is often a static affair. Upon initial connection, a client receives a fixed "cluster view" and a balancing strategy. However, the elasticity of modern cloud environments—where servers are provisioned or decommissioned based on real-time demand—renders these static approaches obsolete. A static client-side load balancer is "blind" to backend changes, leading to missed opportunities with new nodes or errors when trying to reach decommissioned "ghost" servers. This paper introduces a mechanism to bridge this gap, ensuring the client-side proxy remains as dynamic as the cloud backend.
Proposed Architecture
The core of the proposed system is a decentralized architecture that splits responsibilities between the server-side and a sophisticated client-side proxy.
As shown in Figure 2, the architecture distinguishes between Service Flow (actual business requests) and Control Flow (metadata regarding cluster state). Key components include:
- Distributed Cluster-View Maintenance: Servers use a group communication infrastructure (such as JGroups) to maintain a consistent view of active nodes. When a node joins or leaves, the group is notified via callbacks.
- Control Flow Piggybacking: To minimize overhead, cluster updates and policy changes are "piggybacked" onto the headers of standard request/response messages. This avoids the need for constant polling or separate heartbeat connections.
- Dynamic Policy Switching: Administrators can push new load-balancing algorithms (e.g., switching from Random to Static Weighted) to clients at runtime via the control flow.
The group communication infrastructure (Figure 6) ensures that whenever a server joins or quits the cluster, the event is captured and propagated through the system, eventually reaching the client-side view manager.
Implementation and Evaluation
The mechanism was implemented within PKUAS, an open-source JEE application server. Evaluation was conducted using a simulated e-commerce billing system on a cluster of four servers connected via a 1000 Mbps LAN.
Key Results
- Dynamic Join: Adding a fourth node to a three-node cluster resulted in immediate detection by the dynamic client, increasing total throughput by approximately 30%.
- Dynamic Exit: When a node was removed, the dynamic system successfully rerouted traffic with minimal failures (2 out of 100 clients), whereas static clients continued to fail until the end of the test.
- Policy Adjustment: The system demonstrated seamless runtime transitions between different balancing policies, allowing for manual rebalancing toward more powerful nodes without service interruption.
Conclusion
By decentralizing load-balancing intelligence and utilizing piggybacked control flows, this mechanism provides a scalable solution for dynamic cloud environments. While the overhead of metadata in high-frequency environments and the complexity of autonomous policies remain areas for future research, the current implementation in PKUAS proves the effectiveness of dynamic client-side management in maintaining high availability and performance.
Summary updated