Brad Fitzpatrick (bradfitz) wrote in lj_maintenance,
Brad Fitzpatrick
bradfitz
lj_maintenance

Moves

The site was slow today, but we expected that. The problem is one of our database clusters (Ribeye) is massively overloaded and it's been getting worse daily. While it doesn't contain many more total users than other clusters, we let it be "new user cluster" for too long, and it's now overflowing with active users (2-3x any other cluster).

We just started moving a bunch of Ribeye's users to a new database cluster.

Many users on Ribeye (where am I?) will find their journal is read-only mode for the next couple hours. Paid users are excluded from being read-only, except while your actual journal is being moved. Everybody else (well, 20% of people on ribeye) are just in read-only to keep that machine faster while we do the moves, otherwise the moves would take too long. Things will speed up tonight, and the moves should fly.

Hopefully by tomorrow or tomorrow afternoon enough active users will be moved over that things will be more balanced out.

Everybody will go read-write in a couple hours, once load goes back to reasonable levels where the machines can keep up.
Subscribe
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

  • 106 comments
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →