Been toying around with horizontal scaling in Amazon EC2 for distributed searches. In short, I was looking for an effective software load balancer and came across HAProxy and Pound. Both a very nice software based load balancers.
HAProxy Software Load Balancer
HAProxy is a bit more bare metal as it targets a very specific set of scenarios focused on TCPIP more than HTTP. You can use cookie based injection with HAProxy to do round robin and stick users to a specific server. However, you can not do this if your site is running SSL traffic. HAProxy can not decrypt the SSL traffic. This is more of the authors dead-fast belief that SSL should not be terminated because of CPU load on the load balancer preventing scaling as you would need to scale the load balancers at some point (we’re talking millions of requests, facebook style).
Pound Software Load Balancer
Pound is a bit more specific to HTTP/Web scenarios. It functions as a 100% Layer-7 load balancer as it does full HTTP(S) integration and has full access to the HTTP stack. What this means is that you can do some fancy routing based on cookies, url regex, and do this with SSL termination. You can combine the two and have Pound decrypt the traffic and forward to HAProxy but your setting up a scaling issue if you do.
Overview
In the above context diagram we can see that we should have a firewall (software and/or hardware based) to only allow port 80 and 443. We use pound proxy (see below reference) to decrypt the SSL traffic and then pass the request on to HAProxy. HAProxy then inspects/injects the server cookie ID used for sticking somebody to a particular server and then proceeds to do round robin.
Note that the above assumes all servers/appdomain are up. HAProxy will detect if a server is down and redirect any new clients / old clients from the current domain to a different domain (sometimes causing a session loss event, but not much you can do about that). If you are performing an upgrade and would like to “bleed” users off the box that you want to recycle see the below article as HAProxy can support this as well.
Basic HAProxy Configuration
Below is an example configuration that shows a sharepoint farm (using NTLM) a aspnet_farm (using asp.net forms/basic) and a tomcat farm.
# /etc/haproxy.cfg
global
user haproxy
group haproxy
defaults
mode http
option forwardfor
option redispatch
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
backend sharepoint_farm
balance roundrobin
option redispatch
cookie SERVERID insert nocache indirect
server myServer1 192.168.1.1:80 cookie sharepoint01 check
server myServer2 192.168.1.2:80 cookie sharepoint02 check
server myServer3 192.168.1.3:80 cookie sharepoint03 check
backend aspnet_farm
balance roundrobin
option redispatch
cookie ASP.NET_SessionId prefix
option httpclose
option forwardfor
server myServer1 192.168.1.3:80 cookie aspnet01 check
backend tomcat_farm
balance roundrobin
option redispatch
option httpclose
cookie SERVERID insert nocache indirect
server farmsvr1 192.168.1.4:8080 cookie tomcat01 check
stats uri /stats
stats realm haproxy
stats auth admin:P@ss0wrd
stats scope .
stats scope aspnet_farm
stats scope sharepoint_farm
frontend httpid
bind *:80
acl is_sharepoint_farm hdr_end(host) -i sharepoint.itcontoso.com
acl is_aspnet_farm hdr_end(host) -i aspnet.itcontoso.com
use_backend sharepoint_farm if is_sharepoint_farm
use_backend aspnet_farm if is_aspnet_farm
default_backend tomcat_farm
Now what is really nice about HAProxy is that we can have a zero-downtime restart/deploy of all servers. For a detailed guide you can see the below article. How I miss Linux sometimes…
Zero-Downtime Restarts with HAProxy
SSL Termination with Pound + HAProxy
One of the things that might cause a problem is if you need to insert cookie but are running 443. If this is the case and you really have no other alternative (risk vs. reward, total cost of ownership, etc) than you can configure HAProxy + Pount to deliver SSL Termination and then forward calls down to the web-servers as HTTP and the SSL decrypt/crypt happens on the load balancer. This can increase CPU, but last checked some Intel Celeron CPUs can handle 700+ transactions per second with this configuration.
We can test this by generating a self-signed certificate…
# install ssl
sudo apt-get install openssl
# create self signed cert
cd /etc/ssl
sudo openssl req -x509 -newkey rsa:1024 -keyout local.server.pem -out local.server.pem -days 365 -nodes
And then updating the /etc/pound/pound.cfg to include an SSL terminator:
ListenHTTPS
Address 192.168.1.4
Port 443
Cert "/etc/ssl/local.server.pem"
End
Full Layer-7 Load Balancer Scenarios
One of the huge benefits we get by having a layer-7 load balancer is that we can perform intelligent routing and have it be seamless to our applications.
Examples
- Load balance all png, jpg, gif requests to a specific server(s).
- Load balance all mov, avi, mkv requests to a specific server(s).
- Load balance users in South East US IP range to southeast.yoursite.com
- Load balance users by ranking (gold customers, silver, bronze)
Additional Resources
Transparent proxy of SSL traffic using Pound to HAProxy backend patch and howto
Linux install and configure pound reverse proxy for Apache http / https web server
Pound Configuration Information