mod_gridfs performance

In my previous post, I announced mod_gridfs. Now, it's time for some numbers. Serving a 3KiB file over a gigabit network on modern hardware, 100 concurrent requests, MongoDB replica set of 3 machines as a backend:

• NGINX + nginx-gridfs: 1.3krps
• Apache + mod_gridfs: 6.6krps
• Apache + mod_gridfs with SlaveOk and one slave: 12.2krps

Not testing with larger files, because this way I'll be benchmarkng OS I/O performance instead of user-mode code.

Horizontal scaling

Today we finished conversion of a medium-load (~1.2k requests for dynamic content per second) application (frontend, C#+ASP.NET Web Forms+ASP.NET Web Pages) to work on two separate machines. While moving databases around and adding memcached memory was relatively easy, splitting the application that contained state (lots of internal caches) proved a bit hard, even we already have a plan for proper user affinity (cookies+ip hashing via haproxy). PHP users, for example, do not have the luxury of large-scale persistent internal state, so they do not plan for it and use external devices like memcached, message queues, etc. Our application was written in ASP.NET and used about 70 internal caches when I started working on it. Well, now it perfectly runs on two machines behind NGINX+haproxy, and where's two, there's three and more =)

NGINX and keepalives to backend

I wonder when NGINX will have keepalive connections to backend? Patch by Maxim Dounin was floating around for about half a year, then it made into beta, but it's still in beta now, while our media frontend server is busy creating/destroying several thousands of connections to backends instead of reusing existing ones, there are only about 20 connections needed to serve ~3.5kreq/s (current load). Well, maybe G-WAN will be better, although AFAIK it doesn't come with a prebuilt proxy caching.