Load Balancing

aday121

I'm trying to load balance to application servers and am a little confused with the different techniques.

Everybody seems to agree that a hardware load balancer is the best bet but should I configure it to balance based on the session cookie or by the users IP address?

My natural tendancy would be to use the session cookie. Has anybody any experience of this?

mneylon

If you use the IP address how are you going to deal with NAT?

steve-hosting36

Hardware load balancing is always best, we use F5 Big-IP units with great results.

If you are balancing web/app servers, it's best to balance based on the session (session stickiness) which ensures that if an app node goes down, users already mid-session on another node don't get impacted.

There are many ways to divide up the roles, and it depends on the app as much as the architecture / infrastructure.

phil

aday121 wrote: »

I'm trying to load balance to application servers and am a little confused with the different techniques.

Everybody seems to agree that a hardware load balancer is the best bet but should I configure it to balance based on the session cookie or by the users IP address?

My natural tendancy would be to use the session cookie. Has anybody any experience of this?

The effective techniques for load balancing depends very much on what you are trying to load balance.

For example, if you've got a flickr like site, and you're load balancing a bunch of servers serving the static images, you don't need anything at all like session stickyness. Images aren't part of any user defined content (when I connect and download an image to view in my browser, it's the same as when you do it).

Where you need to be careful is where your application keeps state, either via cookies or some other session mechnism.

If you've got your application spread out over 3 servers, A, B & C. The simplest technique is to simply spread load to A, then B, then C. Again, this works extremely well if no persistent state is required.

If all servers have access to all user data (i.e. a shared master database or shared filesystem), then this simplifies the job of session state keeping.

There are plenty of layer-7 load balancers which will intercept the browser request and manage the cookie sessions for you, pointing client A to server A for the lifetime of his session etc.

In general, load balancers with L7 capabilities are much more expensive and obviously CPU intensive as they have to actually inspect the contents of the packets (as opposed to IP based load balancing which means that only the headers need to be inspected).

This isn't something you've specifically asked about, but it's worth noting nonetheless:

1. The term "load balancing" does not always imply redundancy. In fact, where previously you may have had redundancy, sticking one machine managing all the connections to your webservers introduces a Single Point of Failure. If your load balancer fails, the fact that all of your webservers are working means nothing. Hence, if you are looking at load balancing for a High Availability solution, you need to look at multiple load balancers. At the very least, you should always be aware of the shortcomings of any solution you implement.

2. Load balancing does not always imply failover. What I mean by this is pure load balancers will not necessarily detect that a particular webserver is not available. If it's balancing between servers A, B & C and server B dies, the load balancer needs to detect that server B is down and remove it from the pool of available servers to redirect load to.

Both of these combined factors can mean that introducing load balancing into your system can increase the chances of an outage (even if it's only a partial one).

In general, the lowest common denominator would be to use layer-7 load balancing. You should have the least problems using this method. However in most cases the performance of your load balancer will be greatly reduced.

We can probably help more if you suggest:

a. what load balancer
b. what the application is

If the application isn't a well known web application, you need to do some fact finding to see whether the application can handle being load balanced. If it's homegrown, you need to talk to your developers. If it's 3rd party, it's time to hit up the vendor.

blacknight wrote:

If you use the IP address how are you going to deal with NAT?

Can you explain further what you see the problems to be with NAT?

steve-hosting36 wrote:

If you are balancing web/app servers, it's best to balance based on the session (session stickiness) which ensures that if an app node goes down, users already mid-session on another node don't get impacted.

There are many ways to divide up the roles, and it depends on the app as much as the architecture / infrastructure.

I don't necessarily agree with your assertion that it's best to balance based on the session (i.e. at layer 7). Most people use L7 simply because they've been constrianed to do so by their application. In some cases your session is restricted to a machine or group of machines (shard). If that shard becomes unavailable, other machines/shards may not be aware of your session. In most web applications it would certainly seem like a waste of resources to make all session information (which is usually trivial) available to all servers. In some cases it is unintentionally (e.g. session info stored a shared database) shared.

Basically, it's extremely application specific exactly like you said.

cgarvey

steve-hosting36 wrote: »

If you are balancing web/app servers, it's best to balance based on the session (session stickiness) which ensures that if an app node goes down, users already mid-session on another node don't get impacted.

Do you not mean the opposite? If you use sticky sessions and a node goes down, then subsequent requests (in that session) will fail, and you have less control over weighted loading (i.e. slower operations might theoretically be all on one machine, rather than spread).

steve-hosting36

It's very application specific, session stickiness depends on how the app handles sessions in the first place.

phil

cgarvey wrote: »

Do you not mean the opposite? If you use sticky sessions and a node goes down, then subsequent requests (in that session) will fail, and you have less control over weighted loading (i.e. slower operations might theoretically be all on one machine, rather than spread).

Aye exactly. The general theory is pretty much the opposite way around to what Steve said.

The other thing to be aware of I guess, is that a lot of "reverse proxying" is basically load balancing with added benefits (like having a girlfriend with a 911). You'll see it being done with numerous web servers which basically proxy requests to one or more servers. A lot of the benefit is session management and caching done at the frontend using webservers with a lower memory footprint to serve static content.

This is basically load balancing, but reverse proxy is probably a more accurate keyphrase to Google for.

Phil.

steve-hosting36

Most of the hardware LB's we deploy are deployed with session stickiness. As I said, LB is hugely dependant on the app and it's architecture.

brendanuk

blacknight wrote: »

If you use the IP address how are you going to deal with NAT?

Not sure if its what blacknight meant, but there is technique where some networks load balance outgoing requests across proxies, so 2 sequential requests from same client show up as different IPs.

So IP stickyness is just mostly sticky for most people