We'd been talking about doing this "sometime soon" for weeks before the power outage. Guess we learned our lesson.
Most machines we can just reboot and the load balancer won't give them traffic while they're dead. However, now it's time to do the master database server. The site will be down for 5 minutes at most in the next hour here sometime. (Waiting for
sherm to call me from the NOC ... he's going to be there just in case there's a problem with the machine not coming back up.)[1] I like ReiserFS also, but converting an existing machine from ext2 to ext3 is just so painless: only one command.