Today intermittent code errors like "Can't call method "selectrow_hashref" were displayed as our application timed out to certain databases. This did not affect any data or cause any permanent issues -- it's just the result of a regular maintenance tool that wasn't running properly.
As of now, everything should be back to normal. We like to thank everyone for your patience, as well as for writing in reports of the site accessibility issues.
There was a problem earlier which prevented some people from being able to reach the site. We've made some temporary changes to address this, and everyone who was unable to load the site should be able to now.
LiveJournal uses multiple providers to determine how to route traffic from the Internet to our data center. One of those providers went down, which meant anyone who was being routed through this provider couldn't reach LiveJournal. We've temporarily stopped routing any traffic through this provider until they've fixed the problem. Whenever things are back to normal on their end, we'll start routing traffic through them again.
We don't expect there to be any further accessibility issues from this, but if you find yourself unable to reach the site over the next few hours, please send an email to firstname.lastname@example.org and let us know.
Hello LiveJournal users!
In about 40 minutes (1:00 PM Pacific Time), we are going to make system changes. This may disrupt service for some users for only a few seconds. We do not anticipate more than 1 minute of downtime due to this maintenance.
We apologize for the inconvenience.
This afternoon we experienced a technical failure of our Memcached system. Memcached encountered some hardware failures in just the right places (Murphy's Law), which caused issues with some related internal systems and a cascade effect that affected site availability.
Fortunately, we managed to come up with a permanent solution and everything is back up again. Memcached only affects temporary cached data, so no user data was lost or corrupted in any way.
In addition to dealing with these issues, our engineers are working with us to eliminate the few, extremely unlikely points of failure that we managed to encounter today. Better quality "downtime" pages and status updates are now a priority, and I apologize we didn't have something for this situation, as failures on this level are highly unexpected.
Thanks for your patience,
Andreas - LJ Operations