View Full Version : Is anyone experiencing site performance issues as of right now?
APUG--brought down by the "deleted" and joke threads. Well, those would be no great loss.
Could be. The server still seems twitchy but since removing those big threads it's been somewhat usable. If you stumble across any giant threads might be good idea to lock them for now or move them to the "compost heap" so we can keep track of them. When the dust settles we'll try moving them back in, locking them and making a "part2" for them.
Sirius Glass
04-21-2012, 09:36 AM
Things seem improved after moving a few mega-threads from view. What I noticed was the site running fine, then a big slow down. After gaining access to the mysql database process log, every time the site went slow there would be a thread id in the queued processes. Each id corresponded to an extremely large thread. Threads like "your latest 35mm camera purchase" or the "deleted" thread. There were several Kodak ones as well. I'll get these threads back into view soon but locked down and we can create a "part2" for them. For now seems safest to stash them away. It's still hard to tell 100% if that was the issue.
Oh no! Say it ain't so! Not the DELETED thread!
one more find. this sql query is causing the remaining lag:
[quote] Copying to tmp table | SELECT DISTINCT t.threadid,t.title
FROM post AS p
LEFT JOIN thread AS t ON (p.threadid=t.threa |{/quote]
Trying to find out what component in our software is generating this. I think once we can get rid of that things will be a lot better.
The above may have been generated by the sidebar. I have disabled the sidebar for now to see how it goes.
papagene
04-21-2012, 10:10 AM
Site performance seems spotty this morning, sometimes taking a minute or so to load a page.
Site performance seems spotty this morning, sometimes taking a minute or so to load a page.
How is it as of right now?
flatulent1
04-21-2012, 10:26 AM
Seems okay here.
Hopefully things will run ok for now. It seems the main culprits have finally been identified. Gotta get some sleep, then the next couple days see if we can get things back to normal.
Been here for about ten mins, all is nice n fast
markbarendt
04-21-2012, 11:47 AM
I can see things again, yea.
BrianShaw
04-21-2012, 11:50 AM
Oh no! Say it ain't so! Not the DELETED thread!
What deleted thread???
If your site is affected by gibberish in a 300 page Deleted thread, kill it, we will be happy to start another one....
Sirius Glass
04-21-2012, 12:13 PM
If your site is affected by gibberish in a 300 page Deleted thread, kill it, we will be happy to start another one....
That is not gibberish and it makes more sense than the 300 pages about making Kodachrome in grandma's chamber pot.
Siriusly, the problem will not be the long threads, but database coding problems. I have dealt with these type of software problems for many decades. It is in the database software and not the records.
Steve
I know just letting Sean know that quarantining or deleting that thread really wd not matter to us. Indexing, parallelism all fun things in DBA land are what are at play....
ras351
04-21-2012, 05:21 PM
one more find. this sql query is causing the remaining lag:
[quote] Copying to tmp table | SELECT DISTINCT t.threadid,t.title
FROM post AS p
LEFT JOIN thread AS t ON (p.threadid=t.threa |{/quote]
Trying to find out what component in our software is generating this. I think once we can get rid of that things will be a lot better.
Left/Right/Outer joins can be expensive, more so if the tables aren't indexed for the columns being compared. Sometimes things can be improved by changing the query or by creating/managing an alternative table with less data (which can break normal form rules but I won't tell). :)
ok found the bad code in the sidebar. It was related to "My recent posts" block. I have re-enabled the sidebar but disabled that one block for now. Things seem to be running ok. I am now in contact with vBulletin, turning over all of our config files so they can help us optimize the server. The next couple days I'll be focusing on that. To give you an idea of how the large threads+bad code in the sidebar issue affected us:
http://i.imgur.com/2NziK.jpg
The above chart is for the last 3 days. You can visually see the massive problem arise and fade away once it was resolved. I certainly learned a lot during this drama. I had never really needed to go into the database console to troubleshoot. I have a much better feel now for drilling down to issues from the server application level.
Reading SQL logs is like reading sandscript, thanks for all your 'data gnome catching'. Those 2 min loads were something. Thanks for your efforts, all seems to be zippy and smooth /*knock on wood*/
brucemuir
04-21-2012, 07:30 PM
running well here Sean
thanks for the due diligence!
Still running ok. Here is another image of the performance, last 24hrs (the spike on the left is when we had the dramas), you can see on the right how it has settled and response times are fast.
http://i.imgur.com/GwPa0.jpg
There could still be a stray monster thread lurking around that will hit us again, but now that I am familiar with tracking them down the impact should be minimal.
Molli
04-21-2012, 11:56 PM
It's running perfectly here in Victoria, Australia now, Sean. When I saw the post earlier this morning (3-4am-ish) about database issues, I just steered clear. I can't believe how quickly you've got it up and running again. I "broke" my website numerous times when I was coding WordPress plugins and, even with just a few hundred pages to worry about, that was a nightmare. You have my sincerest commiserations. Digging through code of any type really isn't much fun and this site is massive. Thanks for all of your incredibly hard work - it's truly appreciated.