Nginx… Load Balancer… Reverse Proxy… Jack of all Trades

Posted by Chris on October 28, 2009

Lately Nginx has become my new best friend. It seems like every time I think “Hey I want to do …” Nginx is the solution. Recently I’ve setup a new “high availability” cluster for Plus 3 Network and needed a easy-to-configure load balancer to route the traffic between the web servers.

In previous architectures I’ve resorted to wrestling with Linux Virtual Server to handle this job. I was dreading having to go through that painful process again. So this time I searched the interweb and turned up this blog post which sparked the idea of using Nginx to solve this problem.

Along with the load balancing I also wanted to setup a reverse proxy for some of the dynamically created images on the site and cacheable content that we needed to move off the file system and into the database to share across each web server. Plus there were some other dynamically generated bits and pieces that I didn’t want slowing down the our web servers.

The load balancing/reverse proxy part in Nginx is pretty simple since Nginx has the HTTP Proxy module built in. The configuration was similar to setting up the upstream servers for a Mongrel/Thin installation except it would just round robin the requests to the web servers instead the local instances. It also allows you to setup different proxy schemes for different locations… ie. cache the avatars that are stored in the database or the Google map thumbnails.

One additional nugget of goodness I found was the ability to scale and crop images on the fly using Nginx’s Image Filter module. This completely removes the burden from the Rails servers and makes creating square avatar images a snap.

Form a performance standpoint, it blows a Rails only solution out of the ballpark. During my perf tests I was getting about 700 - 900 request/sec on the avatar images (which is comparable to a static image on the site). The Rails server can serve the source image at around 190 request/sec with caching enabled. Not bad for an image that is stored in the database, served via Rails and dynamically scaled/cropped on the fly.

Here is my configuration for the load balancer. It has all the usual goodies that a good reverse proxy needs… dynamic image scaling, expires tags, gzipping, the works! If you have any question feel free to put them in the comments.