Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Really slow...
Options
-
24-10-2010 9:13pm#1
Comments
-
-
-
Seems to alternate between slow-fine-slow-503s-fine constantly for over a month now. Is this a bandwidth/usage issue or do we have a software incompatibility with Google Rehashed (aka Google More Annoying Edition)'s new spiders? (if they changed those that is, certainly everything else owned by them has changed and become more sluggish and poo!
) 0 -
Seems to alternate between slow-fine-slow-503s-fine constantly for over a month now. Is this a bandwidth/usage issue or do we have a software incompatibility with Google Rehashed (aka Google More Annoying Edition)'s new spiders? (if they changed those that is, certainly everything else owned by them has changed and become more sluggish and poo!)
There are loads of different issues which cause 503s and one specific reason which has caused slowness.
We're working on it, but trust me, I barely have enough information to diagnose the problems so you'll have a very hard time guessing with the amount of data you have.
To give you an example of one thing I've had to chase down to alleviate one source of 503s (not a particularly common one as it turned out
) see my answer to question here: http://serverfault.com/questions/190470/varnish-503-when-delivering-a-slow-page/. That took me about 90 minutes of hard work to figure out and it yielded almost nothing. 0 -
I'll throw in another symptom I forgot to put in earlier: ever since Boards started to come apart at the seams I've noticed that after posting often my new post won't show until the thread has been refreshed a few times (even if its an OP - resulting in a weird blank thread
). I've never seen this glitch before we were drowned in the Sea of 503 a few months back. 0 -
Advertisement
-
I'll throw in another symptom I forgot to put in earlier: ever since Boards started to come apart at the seams I've noticed that after posting often my new post won't show until the thread has been refreshed a few times (even if its an OP - resulting in a weird blank thread). I've never seen this glitch before we were drowned in the Sea of 503 a few months back.
They're unrelated. If you're technical, the phrase "MySQL replication is asynchronous" should tell you everything you need to know about why a new post doesn't always appear immediately.
And "Sea of 503" is over-egging the pudding a little bit. Apart from times when the site is down altogether, 503s make up between 0% and 0.05% of page requests.0 -
They're unrelated. If you're technical, the phrase "MySQL replication is asynchronous" should tell you everything you need to know about why a new post doesn't always appear immediately.
I was just curious about how that never ever happened until a short while ago
And "Sea of 503" is over-egging the pudding a little bit. Apart from times when the site is down altogether, 503s make up between 0% and 0.05% of page requests.
Simple answer to that - Boards must have gone down completely several times and for some reason we saw 503s instead of the "usual" Something's Borked Lol! page usually seen upon occasions of mass-hamster-suicide
BUMP for a reason - its happening again, only this time for two days solid! :eek::( No 503's or outages, just crippling slowness 24/7 Have the hamsters been taken out of their cages for scheduled maintenance or has Boards been hit with a Hamster Flu epidemic? 
EDIT - The first time I tried to post this... WHAM! Outage. Me and my big mouth... 0 -
-
I was just curious about how that never ever happened until a short while ago
It's been happening for about 12 months or so, it's just getting more frequent thanks to very heavy growth in adverts.ie which shares the same infrastructure (and it doesn't use the database particularly efficiently so it "punches above its weight" when it comes to screwing stuff up). Adverts is moving off to its own infrastructure soon, so that will help immensely for that particular problem.Simple answer to that - Boards must have gone down completely several times and for some reason we saw 503s instead of the "usual" Something's Borked Lol! page usually seen upon occasions of mass-hamster-suicide
This is (roughly) how you get stuff from boards:[You] <-> [Load balancer] <-> [Web Server] <-> [Database]
If there's something wrong with a load balancer, you'll either see nothing if it's completely knackered or if the network is gone, slowness if there's network issues or if all the web servers are slow and are causing a backlog or 503s if the load balancer is completely backlogged and can't send any more traffic to the web servers.
If there's something wrong with a web server, you usually don't see it since the load balancer will retry another web server until it finds a working one (it currently tries up to 4 web servers before showing you a 503). If the web server is slow, then that counts as broken, so the load balancer will retry until it either finds a faster one or gives up with a 503. This is why you'll sometimes get 503s for searches, since search is very slow.
If the database is broken, you usually don't see it since the web server has multiple databases it can connect to and is usually able to pick non-broken ones. In the event that you do get stuck with a broken database machine you'll either get a vBulletin error message if the DBs are completely down or a 503 if the DB is just slow and hence makes the page run over its allotted time.
As you can see, you can get a 503 as a result of pretty much any problem, and I haven't covered anything to do with misconfigurations above either. The reason you're seeing 503s in the last 8 months is that before (with our old load balancer), if any of those problems happened, you'd see a blank white page or your browser would show its flavour of "this page is responding too slowly" message.BUMP for a reason - its happening again, only this time for two days solid! :eek::( No 503's or outages, just crippling slowness 24/7 Have the hamsters been taken out of their cages for scheduled maintenance or has Boards been hit with a Hamster Flu epidemic?
EDIT - The first time I tried to post this... WHAM! Outage. Me and my big mouth...
The current issues are a different problem. It's not web server capacity (we're at about 40% at peak). It's not database capacity (we're at about 60% at peak). It appears to be some sort of network-related problem at our load balancers. Unfortunately, it's very difficult to troubleshoot since the load balancers are at about 10-20% capacity by most measures. The machines just go through periods of dropping the first few packets of every connection sent to them (which shows up as a "slow site"). Our investigation into it is still in progress, but there's very little evidence as to why it's happening.
If you're seeing slowness 24/7 then you have separate issues also, since the site is actually quite fast for about 16-18 hours a day. We have pretty good measurements, so I'm pretty confident about those numbers. Also, I use the site about 10 hours a day and most of the time it's pretty fast.
There's one thing you (all - not just you Solitaire) can do to help: don't post messages asking if the site is down or slow. If the site is down or slow then we already know about it and we're probably busy fixing it or en route to the nearest computer. Responding to your queries slows us down (this post has taken the guts of 30 minutes - there's no way I could have posted it last night) and not responding results in people jumping to conclusions, making stuff up and getting angry - all of which stresses us out more than we already are. If you're not sure if the site is down or not, try http://downforeveryoneorjustme.com/, it's usually right. Please don't post speculation about the problem either, especially if you're a mod, since people will seize upon your explanation and repeat it everywhere. I know it's frustrating, but all you can do to help is wait until the problem is fixed.0 -
There are loads of different issues which cause 503s and one specific reason which has caused slowness.
We're working on it, but trust me, I barely have enough information to diagnose the problems so you'll have a very hard time guessing with the amount of data you have.
To give you an example of one thing I've had to chase down to alleviate one source of 503s (not a particularly common one as it turned out) see my answer to question here: http://serverfault.com/questions/190470/varnish-503-when-delivering-a-slow-page/. That took me about 90 minutes of hard work to figure out and it yielded almost nothing.
Yeah, that's great, but I have absolutely no idea what you are talking about there.
If memory serves (and I'm not talking about the pc-4200 on my pc here), this is where we come to report problems on the site.
The thing is, most people do not know anything beyond their own computer specs.
Could you please explain the problems in a language we can all understand?
The vast majority of people here do not speak in binary. A simple explanation would be appreciated.0 -
Advertisement
-
-
-
Leave a Comment
Advertisement