bt (dwell) wrote in lj_maintenance,
bt
dwell
lj_maintenance

DoS update

Everyone has been very patient and understanding and we thank you during this frustrating time. We're still being attacked right now, but we're trying to get it so that every human (no attack bots please), no matter where you are or which computer you're on, can get to us.

Here's a good explanation as to what a DDoS or DoS attack is.

If you're interested in our answers to the following questions, then read this super duper long post.

1) Why can't I reach LiveJournal from [home|work|school] but it works at another place? Are you blocking my IP?
2) How come long entries/comments don't go through but short entries or replies do?
3) What happened to our email notifications?
4) How long is this going to last?



1) Why can't I reach LiveJournal from [home|work|school] but it works at another place? Are you blocking my IP?

2) How come long entries/comments don't go through but short entries or replies do?

3) What happened to our email notifications?

4) How long is this going to last?

Are you blocking ME?
We're not manually blocking your IP but there *is* dynamic rate limiting going on. Basically, if a certain computer is sending a LOT of packets within a short amount of time, that IP or network gets blocked. Unfortunately that could also mean that innocent LJers in the same network could get blocked because there might be an infected computer around you that's messing it up for everyone else.

The block is automatically removed after 600 seconds, but if there really is a computer that is attacking LiveJournal, they wll keep going over the limit and keep getting blocked.

If you have this problem, one thing that will *not* help is hitting refresh a lot! No, really. Unfortunately, we can't guarantee that waiting 600 seconds will help either, especially if someone else at your school or work is hitting refresh (or has an infected computer).

So what happens if you're at home and you're the only LJer? And your computer isn't infected or attacking us? Some have found that using a different browser, like IE instead of Firefox, helped them, especially if they're on wireless. Using a proxy or changing their IP through products like HotSpot Shield helps, simply because it might be your DSL or cable provider's network that is being blocked because of another user/computer.

Length of posts?
Some attacks keep sending 100's or 1000's of requests over and over -- like, "hey, give me this journal entry" -- but don't bother to wait for a response! Extremely rude, right? So we're trying to figure out what is a good request and what is a bad request by intercepting the middle of some of these requests and just saying, "if you REALLY want this journal page, do you mind asking again?", because most automated attacks don't bother asking again but a real human would. Most of the time hitting "refresh" 1 time is good enough and you'll get your page. Some times it breaks and you don't get anything no matter what. This is obviously not what we want to happen and we're working to tweak this.

The other thing is that if we see a WHOLE bunch of data come at once, like posting a big entry or comment, then because our DoS mitigation is so nervous and paranoid it might interpret it as an "attack" (even though it isn't) and reset the connection. That would usually make someone frustrated and we would keep hitting "submit" again, right? But then that makes the DoS Mitigation even more paranoid and it'll keep blocking it because now it's seeing it come again, right away. We're trying to get it off the caffeine and stop being so jittery but as long as the attacks continue it'll keep being easily paranoid. Again, we're trying to find the right balance of letting our LJ'ers through while stopping the bad traffic.

We would totally suggest copying your important entry or comment to Notepad or Textedit before you hit "submit" just in case you run into this problem. We know how "ARRRGH!!!!" it is when you lose something you spent a lot of time writing.

Late emails!

We've noticed that emails flow through when the bad traffic slows down. For a lot of people, timely notifications are important. Even more so if you're an RP'er here. As of RIGHT NOW, emails are going normally, but this unfortunately could change if the attack is resumed or increased.

EDIT: As azurelunatic noted, *if* you're able to get to the website, you can always check your inbox as that will be current even if your emails are still sitting on our servers waiting to go out.

Stop pls?
Finally, when is this all going to stop?? We don't know. We know there are some things we can do to help. We know our DoS mitigation works to some degree, and we think we can tune it so that it works even better. But things could change minute-by-minute.

We are working together with US-CERT, with Facebook, with Twitter and with Google since this all started. They've been awesome in sharing what information they can. Each site is being targeted in slightly different ways and each site and company has different capabilities and completely different architectures; what works for LJ might not work for Google and so on.

All of us want this to stop and we're working together to do so. Some of you have barely noticed any hiccups, some of you can't even get to our site and some of you can get through after you jump through what seems like a million hoops. What each of us can do though is make sure that our computers are always up-to-date with the latest system patches from Windows, Apple and whatever OS you are running; that we run up-to-date anti-virus software (there's tons of good free ones out there too!); firewalls; watch what we click on and install and that we try to do our part in making sure our computers aren't turned into zombies/bots so that we don't allow the bad guys into using OUR computers and networks to attack others.



bt
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

  • 332 comments
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →