Too many guests

[LilianaUwU]

It's to a point where every night the forum is essentially DDOSed. What should be done to counter this?

[WillWeaverRVA]

It's to a point where every night the forum is essentially DDOSed. What should be done to counter this?

I've noticed that too. I keep getting "this site is not secure" errors pretty much every night and every morning.

[SectorZ]

I haven't received an error accessing this site in a few weeks. Interesting that I haven't been affected like some others have.

[Rothman]

I haven't received an error accessing this site in a few weeks. Interesting that I haven't been affected like some others have.

Happened to me briefly early this morning.

[1995hoo]

Oddly, I got an error on my iPad this morning, but it worked fine on my PC a few minutes later.

[Max Rockatansky]

I was thinking the same thing last night when I got home and my wife had twelve people over. 

[1995hoo]

I was thinking the same thing last night when I got home and my wife had twelve people over. 

You're making me remember an old joke:

A wealthy but very rednecky man marries a hot young wife. The first day after the marriage, he says, "OK, look, I want to make sure you understand the rules right off. Every day I get home from work at 5:00 PM. I want the TV to be tuned to ESPN, I want my cold beer waiting, and I want to sit like a lump until you have dinner ready at 6:00."

His wife looks at him and says, "That's fine. And I want you to understand my day. I will have sex in this house every afternoon at 4:00 whether you're here or not."

[freebrickproductions]

I think the "guests" are bots scraping the site for stuff to train "AI" models on. Apparently those just straight-up ignore robots.txt on the server, and don't try to avoid overloading servers. I know the forum I help run and SABRE have also been having some issues with the same things lately. I think on my end, I just banned a bunch of IPs until the number of "guests" got to a reasonable level while I think SABRE's implemented more robust anti-bot measures.

[oscar]

I'm getting a completely different error. When I try to access the forum directly, I get redirected to a Plesk site, via https://forum.aaroads.com/login_up.php?success_redirect_url=%2Findex.php When I go to the aaroads.com main page, its link to the forum takes me here. What gives?

I don't know if the Plesk site is a new and improved version of the forum, a bogus phishing site, or malware. I'm not trying it out until I'm sure it's safe.

[Henry]

All I get is a blank screen when I try to click on a link, and sometimes the site even looks funny, as in the normal design isn't there, but instead, it looks more like a poorly-formatted Word document.

[Rothman]

Just got an "offline" message about ten minutes ago -- said the site was down.

[Alex]

The code for the forum is depreciating and the downtime on the server and AARoads website is a result of GET requests executing too slow. A snip of the error log just now shows continuous problems:

Code Select Expand

[06-May-2025 07:31:17] WARNING: [pool aaroads.com] child 1966, script '/var/www/vhosts/aaroads.com/httpdocs/forum/index.php' (request: "GET /forum/index.php?PHPSESSID=6049a912ca767dfb97152ad3c7f8f774&topic=11718.msg2653033") executing too slow (1.066481 sec), logging
[06-May-2025 07:31:17] WARNING: [pool aaroads.com] child 1929, script '/var/www/vhosts/aaroads.com/httpdocs/forum/index.php' (request: "GET /forum/index.php?PHPSESSID=df4020088fabc59cea256ba595c36904&topic=99.msg2903310") executing too slow (1.050513 sec), logging
[06-May-2025 07:31:17] WARNING: [pool aaroads.com] child 1967, script '/var/www/vhosts/aaroads.com/httpdocs/forum/index.php' (request: "GET /forum/index.php?PHPSESSID=9243f639c20e1223d42ac858e8095c99&action=printpage&topic=7920.0") executing too slow (1.307300 sec), logging
[06-May-2025 07:31:18] WARNING: [pool aaroads.com] child 1876, script '/var/www/vhosts/aaroads.com/httpdocs/forum/index.php' (request: "GET /forum/index.php?PHPSESSID=ad8238489b8b8ffcbed8d0400e616855;topic=32158.msg2770981") executing too slow (1.330181 sec), logging

I enabled a slow log showing which PHP transactions are problematic and 95% of them are from the forum index.php script. Here's a snip from that:

Code Select Expand

[06-May-2025 07:37:57]  [pool aaroads.com] pid 2834997
script_filename = /var/www/vhosts/aaroads.com/httpdocs/forum/index.php
[0x000075f42ca13c30] shell_exec() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Subs.php:5270
[0x000075f42ca13b50] host_from_ip() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Security.php:191
[0x000075f42ca13940] is_not_banned() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:114
[0x000075f42ca13120] smf_main() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:90

[06-May-2025 07:37:59]  [pool aaroads.com] pid 2835999
script_filename = /var/www/vhosts/aaroads.com/httpdocs/forum/index.php
[0x000075f42ca13c30] shell_exec() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Subs.php:5270
[0x000075f42ca13b50] host_from_ip() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Security.php:191
[0x000075f42ca13940] is_not_banned() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:114
[0x000075f42ca13120] smf_main() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:90

[06-May-2025 07:38:00]  [pool aaroads.com] pid 2834998
script_filename = /var/www/vhosts/aaroads.com/httpdocs/forum/index.php
[0x000075f42ca13c30] shell_exec() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Subs.php:5270
[0x000075f42ca13b50] host_from_ip() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Security.php:191
[0x000075f42ca13940] is_not_banned() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:114
[0x000075f42ca13120] smf_main() /var/www/vhosts/aaroads.com/httpdocs/forum/index.php:90

I have tried implementing the recommended level of caching, forcing PHP to downgrade to 7.4 (which is what the latest SMF software is compatible with), working with AI to try to address broken code, etc. By no means is this my area of expertise, and the level of frustration and time invested into this is wearing on me.

The Plesk issues are partly because I have not finalized settings, such as 404. I'm nearly complete with my object-oriented overhaul of the AARoads back end, which has been the priority of my focus for the last 4 months. I want to get through this, but the forum downtime on the server has been problematic with both script debugging and data entry. Also as a result, I have the server rebooting daily at 2 AM now, as when you get the "this site is offline" message, that seems to be the one solution that works.

[LilianaUwU]

Today, the record for most people online was DOUBLED from 48k to 95k. The forum is very slow, and it threw me a database error once.

[Rothman]

We're just too friendly and welcoming.

[Alex]

BOTs are flooding the forum with GET requests. 32 in the last 21 seconds. I have consulted AI to try to find a solution, but found nothing. This is also affecting the rest of the server, and I may resort to taking the Forum offline again unless someone can provide me with guidance on how to address this.

[Max Rockatansky]

We're just too friendly and welcoming.

I think from previous conversation that is actually part of the problem.  Non-registered guests can view much of the forum.  Problem is that it is causing chaos because of all the bots. 

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

[formulanone]

We're just too friendly and welcoming.

I think from previous conversation that is actually part of the problem.  Non-registered guests can view much of the forum.  Problem is that it is causing chaos because of all the bots. 

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

That also makes it very difficult to join this forum.

[Max Rockatansky]

We're just too friendly and welcoming.

I think from previous conversation that is actually part of the problem.  Non-registered guests can view much of the forum.  Problem is that it is causing chaos because of all the bots. 

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

That also makes it very difficult to join this forum.

Time and other social media platforms seem to be chipping away on the demand for forums such as this anyways.  If I had to choose between forum death by bots or increasing entropy the latter is far more appealing.

[LilianaUwU]

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

I've BEEN suggesting exactly that.

[Alex]

We're just too friendly and welcoming.

I think from previous conversation that is actually part of the problem.  Non-registered guests can view much of the forum.  Problem is that it is causing chaos because of all the bots. 

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

I tried changing the settings to make the forum viewable to only registered users. Didn't stop the GET requests at all.

[Max Rockatansky]

We're just too friendly and welcoming.

I think from previous conversation that is actually part of the problem.  Non-registered guests can view much of the forum.  Problem is that it is causing chaos because of all the bots. 

Might be high time to make the forum viewable to only registered users.  I haven't seen anyone propose a viable solution and this bot stuff keeps happening.

I tried changing the settings to make the forum viewable to only registered users. Didn't stop the GET requests at all.

Ah, that is an element I wasn't aware of. 

[Alex]

I think I am about done hosting/running the Forum...

[Jim]

I'm looking for similar ideas to fight this on a much smaller scale for the TM forum and the TM site in general.  I added some more robots.txt files but the problematic bots are unlikely to honor them.  I'm thinking about whether to put on a site-wide apache username/password that I'd have to share somehow with legitimate users without having it get brought into the automated requests by the bots.

[freebrickproductions]

IIRC, there are services out there that should help hinder/limit the number of requests from bots/scrapers?

[Alex]

IIRC, there are services out there that should help hinder/limit the number of requests from bots/scrapers?

I implemented code AI gave me with an .httaccess file earlier today. Didn't reduce the requests.

Code Select Expand

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (bot|crawler|spider|scraper) [NC]
RewriteRule ^ - [F]