29 Dec Scaling WordPress MU with multiple databases
Our larger WordPress MU install, Homeschool Journal has been running on multiple databases for about 3 years. Technically, it's possible to have thousands of blogs in a single database install. Usually where people running MU start to have issues with the number of blogs is with getting complete backups of the database.
Upgrades with multiple databases have always been more work and many people running multiple databases typically did not upgrade until the .1 release of whatever version of MU. When I upgraded Homeschool Journal to MU 2.7.1, it was two weeks before I had found and repaired all of the database issues (via several plugins). I decided I wasn't going to go through that again.
With the number of blogs at WordPress.com, Automattic must be using a database plugin that supports WordPress MU better. So, over a couple of weekends, I tweaked HyperDB to use the same MD5 hash based sharding structure that we were using.
I released SharDB (Download) last week as a beta release intended for larger WordPress MU installs using an existing 1-3 character MD5 hash (by blog id) based multi-database sharding structure. (Other structures will be added in early 2010.) The alpha testing of SharDB was carried out this fall with the help of a few people who I knew were using the same multi-database that we were. SharDB has been powering live sites since early October.
SharDB has been tested with over 50 plugins including BuddyPress 1.1 through 1.1.3, domain mapping (both Donncha's and my version) and my multi-site plugin. I have not found any issues with any of the tested plugins. It should support any plugin that accesses data via the $wpdb object. It has been used to upgrade live sites from 2.7.1 through to 2.8.6 and I've upgraded a test site to the WordPress MU 2.9 release candidate tagged 2009/12/21. (Note: I used the plugin in this post to create the commentmeta tables before updating MU from 2.8.6 to 2.9RC.)
One of the features that I really like about SharDB is that I don't need to edit the database config when adding plugins (a feature inherited from HyperDB). Also, I haven't had issues with plugins failing to create tables. As an added bonus, I added a handy column to the site admin blog screen so that you can see which database a blog's tables are in.