August 5th, 2003

  • lisa

Maintenance work tonight

During tonight's regularly scheduled maintenance window (Tuesday nights at midnight PDT (GMT -0700)) we will be replacing the last of the buggy scsi raid controllers mentioned in previous posts. We do not expect downtime of more than a few minutes and this server is not part of any of the database clusters. I will update here when we are finished.

Update (6:30 am, PDT): All of the planned work for this evening is now complete.

We unexpectedly had to reboot all of our webservers (at once) during the work tonight. This caused the "Forbidden" and "404" errors. Also, some late text messages and emails were sent as the result of some of the older webservers cleaning out their mail caches upon reboot. (Our webservers all netboot, and typically run on the same configurations, but one of them had been up for long enough that it wasn't using new mail relay information until it rebooted tonight) This was not intended and we do apologize for the inconvenience.