January 9th, 2002


Recent issues.


The last two days of slowness have been caused by another compound problem.

So, in English: Site was down, and there may still be problems for the next few days.

In Geek: One of the major routers our uplink goes through started "flapping" sometime yesterday. Our co-lo facility, Internap, was excellent in helping us through this. They traced the problem down, and talked with their peers on how to fix the problem. It might still be a little iffy, but that's life.

Also, yesterday, two machines outright crashed. One of them was the last machine doing internal DNS, so a lot of other things broke. Mail couldn't resolve hostnames, and therefore couldn't get sent, some internal systems slowed down or stopped working while waiting for DNS timeouts, etc. If anyone recalls, one of our misc machines had a harddrive failure during the holidays. It was the other machine doing internal DNS. That machine just got fixed now, so a single failure like that shouldn't hurt now. All machines are currently running ok. I am also going to be doing lots of configuration stuff today, and adding two new webslaves to the free pool. This should not incurr any extra downtime.