mod_gridfs is dead, long live gridfs_server

Long time no write, but mod_gridfs is no longer going to be developed, it's being superseded by gridfs_server. It's written in C#, based on ASP.NET Core, Kestrel, runs on .NET Core 2.0 Preview 2, and successfully serves about 25 million files per day from our MongoDB GridFS installation, as a backend to a front-end cache (that serves about 450 million files per day).

mod_gridfs

As we were planning to move our terabytes of files into MongoDB GridFS, it occured to us that there is no readily available way to efficiently serve these files over the web, without resorting to using an ASP.NET GridFS IHttpHandler we implemented for local debugging some time ago.

After much hassle while developing GridFS handler for G-WAN (it is certainly fast, but keeps crashing even on Ubuntu 10.04.4 LTS, both x86 and x64 versions, even when I removed all non-boilerplate code from the module itself), performance measuring https://github.com/mdirolf/nginx-gridfs (until there is an asynchronous MongoDB driver, any GridFS module for NGINX is doomed), I decided to write an Apache 2.x (actually tested on 2.4) module to serve files from GridFS.

I decided to release it as open source here: https://bitbucket.org/onyxmaster/mod_gridfs/. It runs faster than the NGINX one, even with multiple workers, so we're going to use it as a backend (with NGINX as a caching frontend of course).

Configuration example:
GridFSConnection rsTest/db1,db2
GridFSDatabase my_database

Horizontal scaling

Today we finished conversion of a medium-load (~1.2k requests for dynamic content per second) application (frontend, C#+ASP.NET Web Forms+ASP.NET Web Pages) to work on two separate machines. While moving databases around and adding memcached memory was relatively easy, splitting the application that contained state (lots of internal caches) proved a bit hard, even we already have a plan for proper user affinity (cookies+ip hashing via haproxy). PHP users, for example, do not have the luxury of large-scale persistent internal state, so they do not plan for it and use external devices like memcached, message queues, etc. Our application was written in ASP.NET and used about 70 internal caches when I started working on it. Well, now it perfectly runs on two machines behind NGINX+haproxy, and where's two, there's three and more =)

NGINX and keepalives to backend

I wonder when NGINX will have keepalive connections to backend? Patch by Maxim Dounin was floating around for about half a year, then it made into beta, but it's still in beta now, while our media frontend server is busy creating/destroying several thousands of connections to backends instead of reusing existing ones, there are only about 20 connections needed to serve ~3.5kreq/s (current load). Well, maybe G-WAN will be better, although AFAIK it doesn't come with a prebuilt proxy caching.