Log in

No account? Create an account

Comments on unclustered
bradfitz wrote in lj_maintenance
Comments on unclustered journals are temporarily broken.

It looks like one of the comment tables on the legacy cluster is confused, so it's refusing all new inserts. I'm going to take it down and repair the index from the master data file.

I'll post when I know more.

Corrupt index talk index; an explanation.
bradfitz wrote in lj_maintenance
** NOTE: Don't ask tech support questions here. Go here. ***

The index on the legacy talk table went corrupt. That's why comments are down on non-clustered journals. At 3 GB, it's taking awhile to repair.

When you hear the word "corrupt" you probably get paranoid about data loss, but if you know a bit about how databases work you won't be scared.... so I'll explain:

When data is sent to the database it's stored in a data file. This never goes corrupt (at least, I've never seen it in ~7 years of doing this crap). The problem with a data file by itself is that it's slow finding stuff. To find, say, your journal comments you'd have to scan over the whole thing until you found what you wanted. Typical textbook example: Imagine looking up your friend's phone number in the phone book if it were sorted by "time person moved to the city" and not by last name.

So the "magic" part of the databases are the indexes. The key to making any database application fast is creating, maintaining, and using the right indexes.

Problem with indexes is that they more often go corrupt. I've seen this happen over time on a number of popular database engines ... it shouldn't ever happen, but it does, simply because there's so much optimization in there, and tons of corner cases.

Luckily, when an index is corrupt you can either fix it or just delete it and have it rebuilt. This just takes a lot of time. (Imagine sorting 3.1 million rows by hand, 5 times, each time sorting on something different)

So that's what's happening now. It's fixing the last index now.... should be done somewhat shortly.

Another benefit to clusters is that instead of having one massive 3 GB table, we have, say, three 1 GB tables, and if one's index goes corrupt, the fix is 3 times faster and only affects 1/3 of the users. So I guess that's 9 times better, eh? :-)

The cluster conversion is over 1/5th done, but that'll move a ton faster now ... the last few days I've been speeding the conversion process up a ton. It's flying now. I'll post more updates in the future.

** NOTE: Don't ask tech support questions here. Go here. ***