Anyway, we suck. We should have been watching this. We noticed it was getting close a few months ago, but we should've written a script then to notify us whenever a table got near 80% capacity so we could modify it to use a larger pointer size (which is bad to do if you don't need it, since Pentium processors are only 32 bit machines... addressing more than that is slower).
We did the normal maintenance we were going to do, though, and we know a lot more about the MyISAM table format now.
I tried to pass the blame onto Dormando, but that's not fair... we both suck. (hey, I've never claimed to be a great sysadmin/dba. :))
Hey, at least this all happened Sunday night, the lowest LJ traffic time of the week, right?
My apologies to everybody in Europe that was sitting around bored at work on Monday without LiveJournal to keep you entertained. Monday is one of the highest traffic times.
This sucked. Record downtime? I think so.
Oh yeah: Any entry you submitted from 11:15 or whenever the problem started to 11:30 when we took the site down is lost. That sucks even more. :-/