Load-Balancing as a Network Primitive
With the emergence of large web applications in the late 1990s, there was much interest in load-balancing to spread incoming requests across a set of identical web servers. Usually, they exploit some trick in the network (e.g. DNS, anycast, etc) to make it work without altering the network logic. Many commercial load-balancing products have been built that sit on the path of incoming requests and spread them over a set of servers. Load-balancing is used increasingly for other tasks beyond balancing web requests too. For example, it is used in CDNs for serving content from multiple servers. It has become a commonly used element of all scale-out network services.
Current load-balancing methods make a number of assumptions about the services:
- Requests enter through a single point in the network; the load-balancing device is placed at a choke point through which all traffic must pass. For networks where this condition does not hold true, operators end up using many of these expensive devices. Yet, they create congestion in the network.
- The network structure is regular. In practice, it may be (e.g. a datacenter); but enterprise networks are not.
- The congestion is at the servers, not within the network. Again, this may be true in datacenters hosting only one service. In a cloud datacenter with many services, the network may be congested differently in different places. Often datacenters don’t have full bisection bandwidth and can be congested. In an enterprise network, there can be many choke points, like egress connections to the WAN, campus backbones, etc.
- The servers are static. Operators of virtualized datacenters move VMs around to make efficient use of their servers. As VMs move, the load-balancers need to track their location so as to direct new requests to the right place.
- The network load is static. Most load-balancers spread traffic obliviously through the network, using static schemes like ECMP. This is suboptimal when some parts of the network are congested.
- All services need the same load-balancing algorithm. This means the service provider has to use the same scheme for load-balancing, say, HTTP requests and video requests; or he has to install two load-balancers. In virtualized data centers this will be much harder as more services will be deployed by different users, and they will be moving around.
Our work is premised on the following observation: load-balancing is essentially congestion-aware routing (based on network and server congestion). It, therefore, leads us to the belief that load-balancing should be an integral property of the network. If we think of the network datapath as implementing a basic small set of plumbing primitives‚ (e.g. forward a packet to one or more ports) then load-balancing fits nicely into this model. It just means that the datapath has to intelligently (and dynamically, and quickly) decide which outgoing port to send a request to, and make sure all the packets associated with the request follow the same path to the same server. We therefore seek a solution with the following characteristics:
- Distributed: If every switch in the network can load-balance incoming requests, we don’t need to re-design the network to accommodate the load-balancers it simply becomes a property of the network. We can do away with choke points and constrained routing. It is then naturally scalable too - just add more switches.
- Dynamic: It must react to network and server congestion, and pick routes and servers accordingly.
- Auto-configured: It must automatically adapt and scale when servers and network capacity are added or deleted. It must continue to operate as servers move around (e.g. VM migration).
- Flexible: It must balance load in a way that is optimized for each service or application; the service creator must be able to decide how load is balanced. This means new algorithms must easily be created, tested and deployed.
To this end, we have built Aster*x, a prototype distributed load-balancer. Aster*x is premised on the belief that every switch and router should easily be able to do load-balancing, and that it is cheap to do so. Our approach builds on the growing trend towards ‚ software-defined networking such as OpenFlow/NOX. In these approaches, the network switches are treated as a dumb, minimal flow-based datapath, under the control of a remote, software control plane. OpenFlow is the common, narrow, vendor-agnostic interface to the flow switches; NOX is the control plane upon which services like Aster*x are built.
Aster*x treats each individual request - or a bundle of aggregated requests - as a flow, and decides how to route the flow. The flow could be, for example, a single HTTP request, or it could be all the requests for a particular service. Aster*x can decide whether to route each individual request, or use ECMP-like oblivious load-balancing in any combination. For example, it could choose to send all HTTP requests to one pool of servers, and all video requests to another pool; and then do ECMP-like load-balancing over each pool. Or it could choose to do oblivious load-balancing over different regions of a data center, then do careful, per-request load- balancing within one region. The key here is that Aster*x can be used flexibly to define how the load-balancing is done, under the control of the service or application. It is not pre-defined by a fixed-function load-balancer. Aster*x has the following characteristics:
- Distributed throughout the network: If the switches are low-cost OpenFlow-enabled switches, then they can do load-balancing. It is therefore very scalable.
- Logically centralized: Load-balancing is defined in one place for the entire network ‚ in the control plane app. It is easy to scale by replicating the control plane.
- Flexible: Each service can have its own load-balancing algorithm. Each type of request can be load-balanced by a combination of oblivious and intelligent (flow-by-flow) schemes depending on the needs of the service, and the capabilities of the datapath and controller. The service creator is free to decide.
- Build now, change later: Because the datapath supports general load-balancing, we don’t need to decide how load-balancing will work when the datacenter is built; we can evolve it over time to suit the services deployed.
Comparing the performance of random and smart load-balancing with Aster*x
Demo: Aster*x load-balancing HTTP traffic in a CDN-like WAN environment