Brad Fitzpatrick (bradfitz) wrote in lj_maintenance,
Brad Fitzpatrick
bradfitz
lj_maintenance

santa - read only

We just lost a drive in Santa's disk array. It's rebuilding from a hot spare, but that database is now a little slow during the rebuild. (this is one of the DB clusters that's not yet in master-master, so we can't fail over to the other pair.... we're still working on upgrading all the old ones)

Anyway, during the slow period until it rebuilds (30 min - 1 hour or so), most users on the Santa cluster are in read-only mode. [Where am I?]

BTW, losing disks like this isn't a big deal. The best disks are rated (warrantied) for 3-5 years. Let's assume 4 years. If we have 100 disks (I'm just guessing), that's one failing every 2 weeks if it were spread out. But a lot of these disks we've had for years now. Fun fun. That's why you should never trust a single disk.... always use RAID, boys and girls! :-)
Subscribe

  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Help
  • 224 comments
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →