Hey, what happened? For a while, we either got 403-Forbidden, 503, or 404-Page not found errors. I actually set up a monitor on my local instance of Uptime Kuma to see when the forum came back up.
Quote from: ZLoth on May 16, 2025, 10:57:08 AMHey, what happened? For a while, we either got 403-Forbidden, 503, or 404-Page not found errors. I actually set up a monitor on my local instance of Uptime Kuma to see when the forum came back up.
The forum server got absolutely clobbered by bots.
Why would anyone DDOS this forum? Idiots.
The same thing happened to me. I had feared I had been either banned from posting, or AAroads was blocking me for some reason. I'm glad I wasn't the only one who experienced this and can post again.
Quote from: swake on May 16, 2025, 11:16:57 AMWhy would anyone DDOS this forum? Idiots.
Companies downloading the entire forum to feed it into their AI. Except instead of going through the pages one by one they have 136,000 computers connect all at once to download everything.
Quote from: The Ghostbuster on May 16, 2025, 11:49:50 AMThe same thing happened to me. I had feared I had been either banned from posting, or AAroads was blocking me for some reason. I'm glad I wasn't the only one who experienced this and can post again.
Alex had said earlier this week he thought he would be forced to take the forum down pending a solution. I wondered whether it would come back up. Glad to see it's back.
Quote from: Scott5114 on May 16, 2025, 11:57:42 AMCompanies downloading the entire forum to feed it into their AI. Except instead of going through the pages one by one they have 136,000 computers connect all at once to download everything.
136,000 computers connecting at once? Doubtful. But, 136,000 virtual instances connecting at once to download everything in a server farm? That's more realistic.
Quote from: ZLoth on May 16, 2025, 12:11:48 PMQuote from: Scott5114 on May 16, 2025, 11:57:42 AMCompanies downloading the entire forum to feed it into their AI. Except instead of going through the pages one by one they have 136,000 computers connect all at once to download everything.
136,000 computers connecting at once? Doubtful. But, 136,000 virtual instances connecting at once to download everything in a server farm? That's more realistic.
I dunno, that's what the forum software said before it went down.
If they're all connecting from different IP addresses and displaying different user agents, then it really doesn't make all that much difference on our end whether they're running on different hardware or not. (And with IT stuff, I find it usually makes more sense to speak imprecisely when that conveys the gist of what's going when you have no idea to what degree the person you're talking to cares about the technical detail.)
Modern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
I'm getting those same 502 errors, about 20% of the time that I click on a topic.
When the forum first came back up, the "users online" and "most online today" seemed to hover at around 300, which seemed reasonable, but I now see in the last hour that the "most online today" spiked again to 28K - so I'm concerned the bots might be trying to come back?
Quote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
Quote from: swake on May 16, 2025, 02:52:30 PMQuote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
AI scrapers are the property of AI companies. Don't whitewash what they are.
Quote from: LilianaUwU on May 16, 2025, 03:44:10 PMQuote from: swake on May 16, 2025, 02:52:30 PMQuote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
AI scrapers are the property of AI companies. Don't whitewash what they are.
Isn't this the Sonic the Hedgehog Bad Future that K12 was trying to convince us would be great?
Quote from: Max Rockatansky on May 16, 2025, 03:48:42 PMQuote from: LilianaUwU on May 16, 2025, 03:44:10 PMQuote from: swake on May 16, 2025, 02:52:30 PMQuote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
AI scrapers are the property of AI companies. Don't whitewash what they are.
Isn't this the Sonic the Hedgehog Bad Future that K12 was trying to convince us would be great?
If that's the guy who talked about a Minority Report-style future where everything is automated, but there is no manual override or concept of "things need to be maintained," then maybe.
My take is AI is a tool like anything else. It's neither good nor bad. It's what you do with it.
Quote from: Quillz on May 16, 2025, 04:32:42 PMQuote from: Max Rockatansky on May 16, 2025, 03:48:42 PMQuote from: LilianaUwU on May 16, 2025, 03:44:10 PMQuote from: swake on May 16, 2025, 02:52:30 PMQuote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
AI scrapers are the property of AI companies. Don't whitewash what they are.
Isn't this the Sonic the Hedgehog Bad Future that K12 was trying to convince us would be great?
If that's the guy who talked about a Minority Report-style future where everything is automated, but there is no manual override or concept of "things need to be maintained," then maybe.
My take is AI is a tool like anything else. It's neither good nor bad. It's what you do with it.
All these AI-branded algorithm tools have been resulting in negative outcomes along all frontiers of the road enthusiast hobby.
Off forum today I had to delete an automatically generated AI from hijacking a California Road group I manage on Facebook. This is one of several groups where I've had to make similar intervention. That doesn't even get into the malignant mass of AI Road pages that keep stealing from hobby pages.
But yes, K12 is very big on the supposed benefits of modern algorithms. He and others seem to misunderstand that none of what is out there is actual artificial intelligence.
Quote from: Quillz on May 16, 2025, 04:32:42 PMQuote from: Max Rockatansky on May 16, 2025, 03:48:42 PMQuote from: LilianaUwU on May 16, 2025, 03:44:10 PMQuote from: swake on May 16, 2025, 02:52:30 PMQuote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
Thank you so much.
People call these chatbots AI. THEY ARE NOT AI. It's a sham.
AI scrapers are the property of AI companies. Don't whitewash what they are.
Isn't this the Sonic the Hedgehog Bad Future that K12 was trying to convince us would be great?
If that's the guy who talked about a Minority Report-style future where everything is automated, but there is no manual override or concept of "things need to be maintained," then maybe.
My take is AI is a tool like anything else. It's neither good nor bad. It's what you do with it.
It's a tool, but I just don't need that left-handed screwdriver and nine-point socket all that often.
K12 reminds me of Dyson from Terminator 2. Where he was making his design with nothing but good intentions in mind ("a pilot that never gets tired or makes mistakes"), and it didn't even seem to cross his mind that his design could be exploited and/or used for nefarious purposes.
I wasn't surprised the forum was taken offline (there were warnings about it). I was a bit surprised the forum was down for that many days. Hopefully the "AI" scraper bots won't return any time soon.
The first time I tried to access the forum during the outage, I actually saw a note from Alex saying he was going to be forced to take it offline until he could figure out something. After that, I started getting 403 errors.
I saw the same note hbelkins mentions.
Honestly, with all the messaging to the effect of the forum being on borrowed time, I'm pleasantly surprised that it returned at all. I was starting to wonder how I'd keep up with road-related developments around the country without it.
Quote from: DTComposer on May 16, 2025, 01:21:03 PMI'm getting those same 502 errors, about 20% of the time that I click on a topic.
And what's worse, once you get it on a topic, I've found that you'll get it again and again when trying to click the same link (reload only works 1/10 of the time). It takes me 3-5 tries to load topics that get the error, and when they finally do load, the load to the start of the page, not whatever post they were supposed to start on. Combine it with the lack of "new" banners on posts in threads, and browsing the forum isn't fun right now.
So many 'Loading' hangups now. Almost every time I go to quote a post, it fails to populate the quoted text and just throws a 'Loading' banner at the top of the screen instead. Refreshing the page and trying again works sometimes, but other times it takes several attempts and few mild profanities.
Eh, could be worse. Perhaps the struggle to post will increase the quality of posts on the forum. Only the strong and persistent can participate!
I'm glad it's back up. I can work around 502's and the Loading... on quotation issue.
("Back in my day, all we had was this: > ... or this: ^^^^^^^")
Reading this thread explains the 402 gateway error I got when opening this thread in particular. The irony is palpable however.
Quote from: vdeane on May 16, 2025, 10:07:32 PMHonestly, with all the messaging to the effect of the forum being on borrowed time, I'm pleasantly surprised that it returned at all. I was starting to wonder how I'd keep up with road-related developments around the country without it.
I got a Discord server.
Alerts don't appear to be working presently. It's showing I have one, but I cannot read it.
(EDIT) When I open up alerts as a separate page, I can access them.
Quote from: DTComposer on May 16, 2025, 01:21:03 PMI'm getting those same 502 errors, about 20% of the time that I click on a topic.
When the forum first came back up, the "users online" and "most online today" seemed to hover at around 300, which seemed reasonable, but I now see in the last hour that the "most online today" spiked again to 28K - so I'm concerned the bots might be trying to come back?
Quote from: Max Rockatansky on May 16, 2025, 12:16:38 PMModern algorithms (misbranded AI) are a cancer.
Also, I'm getting a bunch of 502 errors if I refresh pages too quickly. I'm taking it that this some form of mitigation against the bots?
The 502's are a result of me trying to correct some errors with the forum. I had to cut short my work on this yesterday as I had plans and didn't want to make everyone wait too much more. Hopefully I will clean up the 502 error problem this morning.
Quote from: Bobby5280 on May 16, 2025, 05:15:05 PMI wasn't surprised the forum was taken offline (there were warnings about it). I was a bit surprised the forum was down for that many days. Hopefully the "AI" scraper bots won't return any time soon.
I was prepared to leave it offline longer...
I needed time to approach the bot problem from another angle, and eventually surmised leveraging Fail2Ban through Plesk to actively track and block problematic IP's. However, I had major problems with restoring the forum due to the persistent 403's generated by nginx and with software depreciation. They are working on the next SMF finally, but until that is released, I am on my own for correcting code problems such as with the open_basedir settings.
FWIW I've also been having to learn on the fly, as nginx/Apache2 were pretty much voids for me in my knowledge wheelhouse. Since I've been running commands through shell extensively on this, I at least have a better grasp on using it and its capabilities.
Quote from: Alex on Today at 07:59:04 AMQuote from: Bobby5280 on May 16, 2025, 05:15:05 PMI wasn't surprised the forum was taken offline (there were warnings about it). I was a bit surprised the forum was down for that many days. Hopefully the "AI" scraper bots won't return any time soon.
I was prepared to leave it offline longer...
I needed time to approach the bot problem from another angle, and eventually surmised leveraging Fail2Ban through Plesk to actively track and block problematic IP's. However, I had major problems with restoring the forum due to the persistent 403's generated by nginx and with software depreciation. They are working on the next SMF finally, but until that is released, I am on my own for correcting code problems such as with the open_basedir settings.
FWIW I've also been having to learn on the fly, as nginx/Apache2 were pretty much voids for me in my knowledge wheelhouse. Since I've been running commands through shell extensively on this, I at least have a better grasp on using it and its capabilities.
Glad to hear that you've found what seems to a solution for the time being, and that you're getting more confident with the tools you're working with.
If we run into further bot problems, Anubis seems to be working pretty well on the wiki, so that would be the obvious next thing to try on the main site.
I keep getting a 502 error when I load certain pages. Still some bugs on my end.
Quote from: MATraveler128 on Today at 09:55:02 AMI keep getting a 502 error when I load certain pages. Still some bugs on my end.
I'm not sure it has anything to do with the page. I can't confirm, but it seems to be a constant 40% for me, reloading randomizes the "roll" again. If anything, and I can't confirm, loading a particular page for the first time that's current has a higher chance than reloading any page or viewing a page in a thread where the last post is more than a few days old.
If you get an error after posting, the post
does go through. It happened with this one. It was a different error message than the standard 502, though.
The 502 errors are due to this error:
2: Unknown: Cannot call session save handler in a recursive manner
Really getting frustrated trying to solve this. :banghead:
Quote from: Alex on Today at 10:51:10 AMThe 502 errors are due to this error:
2: Unknown: Cannot call session save handler in a recursive manner
Really getting frustrated trying to solve this. :banghead:
You got this. We are all counting on you. Thank you for your efforts.
Quote from: Bobby5280 on May 16, 2025, 05:15:05 PMI wasn't surprised the forum was taken offline (there were warnings about it). I was a bit surprised the forum was down for that many days. Hopefully the "AI" scraper bots won't return any time soon.
I was pleasantly surprised the forum came back up what I think was quickly. I imagined it would be down a week or more.
I join the chorus of those thanking Alex for his time and effort in resolving the issue.
I've gotten a few random 502 errors. If I refresh it usually clears it up. But sometimes it points me to the top of the page rather than the first unread post.
I had seen the reports of the problems, and then when I got the "forbidden" error I thought the forum had been discontinued. I thank Alex and any others who helped for getting us back online.
EDIT: And alerts are working again for me.
Quote from: GaryV on Today at 12:56:17 PMI've gotten a few random 502 errors. If I refresh it usually clears it up. But sometimes it points me to the top of the page rather than the first unread post.
I had seen the reports of the problems, and then when I got the "forbidden" error I thought the forum had been discontinued. I thank Alex and any others who helped for getting us back online.
EDIT: And alerts are working again for me.
I set the forum to 403/forbidden as the bots were still hammering the server with GET requests even after I renamed the forum subdirectory so that the URL would not work. The 403 method was a surefire way to block them.
I think I have the 502 error resolved... Working on one last error now that mostly effects the script processing in the background.
Quote from: Alex on Today at 02:01:33 PMI think I have the 502 error resolved... Working on one last error now that mostly effects the script processing in the background.
Confirmed. I haven't gotten any in a few hours.
No 502 errors today for me (but only one Connection Error message) today so far, compared to the last two days.
Minor things, but noticed too that the "Like" button, as well as some information (like personal messages, date joined, last active, etc) about a user on posts are also unavailable for the timebeing, assuming that is linked to current efforts to combat the bot issues on the server.
Thanks to Alex and everyone else working to resolve such issues.
Quote from: fwydriver405 on Today at 02:49:43 PMNo 502 errors today for me (but only one Connection Error message) today so far, compared to the last two days.
Minor things, but noticed too that the "Like" button, as well as some information (like personal messages, date joined, last active, etc) about a user on posts are also unavailable for the timebeing, assuming that is linked to current efforts to combat the bot issues on the server.
Thanks to Alex and everyone else working to resolve such issues.
If you encountered a 500 error around 2:30 EDT, that was because of a syntax error I made in a file related to the database that took down the server. :-o
Otherwise I did deactivate the Like button, as it was causing scripting problems. Some of the other aspects may be because I did not have the original files when I did reinstallations. James or Scott probably has a better grasp of what SMF packages were installed or settings were made such as for date joined and last active.
With that said, I have addressed each of the SMF errors reported. There are still times that the Forum hesitates to where a query exceeds 5 seconds. The jam has something to do with the Display.php script from SMF. Since I've been working on this almost nonstop since 7:30 this morning, I am going to say that is an acceptable issue considering that it seems that functionality is vastly restored. That plus that Fail2Ban and a modification to the nginx directives to block all bots seems to be addressing the thousands of bot GET requests from before.
If you got a 502 error just now, that is because I restarted the server. The memory usage was again topped out. Another thing for me to monitor...
I miss the like button :-(
Quote from: Alex on Today at 03:08:00 PMIf you got a 502 error just now, that is because I restarted the server. The memory usage was again topped out. Another thing for me to monitor...
I saw it, but I thought it was because I was testing if forum search was back (it's not) and it was blocking me specifically for high usage.
Idea: What about a server restart every X hours on the hour (X would be chosen based on how long a restart takes) so the outages are expected rather than unexpected?
I appreciate the hard work trying to keep the forum up. It's been working just fine for me today.
Quote from: fwydriver405 on Today at 02:49:43 PMas well as some information (like personal messages, date joined, last active, etc) about a user on posts are also unavailable for the timebeing
I thought the left columns with user information looked unusually short... I guess that's why.