Server Downtime - May 13th-16th, 2025

[vdeane]

Country flags are gone too, though I think a lot of us stopped using them by this point.  Also, changing your profile details apparently nukes your avatar, with no way to get it back unless you have external hosting.

EDIT: Managed to find it in the gallery listing.  Also, it's hilarious that there's still a spot for Skype usernames with Skype no longer being a thing.

[cahwyguy]

Just a note to say THANK YOU to Alex for all his hard work on the forum. The Pacific Southwest forum is one I depend upon as a primary source of information for my pages, as well as a primary place to promote when I do California Highway Headline posts, posts of updates to my pages, and posts about the California Highways podcast. It has always served as a relatively high quality source of information.

I know the incredible amount of work that goes into maintain all of this, and I just want to let @Alex know that I appreciate it.

The one suggestion I might make is to create a read-only, non-forum page off AARoads (e.g. .../forumstatus.html) that just provides a status update on the forum for downtimes like this. The AARoads FB page, which would be a logical place for that information, isn't a good source.

Thank you again.

[Molandfreak]

I miss the like button 

Like

See, this is why it's handy. If someone posts something thoughtful or interesting that I have no further comment on, I like using it since we can no longer just post an emoticon in response to someone's post.

[Alex]

Just a note to say THANK YOU to Alex for all his hard work on the forum. The Pacific Southwest forum is one I depend upon as a primary source of information for my pages, as well as a primary place to promote when I do California Highway Headline posts, posts of updates to my pages, and posts about the California Highways podcast. It has always served as a relatively high quality source of information.

I know the incredible amount of work that goes into maintain all of this, and I just want to let @Alex know that I appreciate it.

The one suggestion I might make is to create a read-only, non-forum page off AARoads (e.g. .../forumstatus.html) that just provides a status update on the forum for downtimes like this. The AARoads FB page, which would be a logical place for that information, isn't a good source.

Thank you again.

Much appreciated Dan. I like your idea and will add that as an addition to my nearly complete site reprogramming.

I miss the like button 

Like

See, this is why it's handy. If someone posts something thoughtful or interesting that I have no further comment on, I like using it since we can no longer just post an emoticon in response to someone's post.

I don't disagree with you on this. It is that during the course of my debugging and code adjustments, the Likes script was part of a recursion error that generated incessantly:

Code Select Expand

2022 | 1747489305 |      6594 | D
▒g  | ?topic=18020.1750             | 2: Unknown: Cannot call session save handler in a recursive manner | 9bf262bec0956a41b08875ca9536d734 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2021 | 1747488610 |         0 | (M▒▒   | ?topic=10906.msg261197        | 2: Unknown: Cannot call session save handler in a recursive manner | 959f589879e9afad53248ff3c71675f9 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2020 | 1747487382 |         0 | 4▒▒▒    | ?topic=22572.msg2317834       | 2: Unknown: Cannot call session save handler in a recursive manner | f3e7f1bfb28018d2e3335073d4650b24 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2019 | 1747487209 |         0 | ▒˾▒    | ?topic=12138.0                | 2: Unknown: Cannot call session save handler in a recursive manner | 9a69812234fb96c3683991378bf0ce6d | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2018 | 1747487208 |         0 | ▒˾▒    | ?topic=12138.0                | 2: Unknown: Cannot call session save handler in a recursive manner | 7c13142ee7c9d82c8af59de1aef4a8c5 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2017 | 1747487208 |         0 | ▒˾▒    | ?topic=12138.0                | 2: Unknown: Cannot call session save handler in a recursive manner | a26d2231f1f3f6808fb37bd747c8c851 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2016 | 1747487198 |         0 | 4▒▒▒    | ?topic=1224.msg2979417        | 2: Unknown: Cannot call session save handler in a recursive manner | b85130c98773c858c7f1a81cd84a7fce | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2015 | 1747487102 |         0 | B▒D  | ?topic=11590.0;prev_next=prev | 2: Unknown: Cannot call session save handler in a recursive manner | e1bc74829e96601cb5c14bc013d45ed7 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2014 | 1747486515 |         0 | 4▒▒▒    | ?topic=22799.msg2541607       | 2: Unknown: Cannot call session save handler in a recursive manner | 793979d9cc5cb5d17f792a2caa87b584 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |
|     2013 | 1747486371 |         0 | 3▒▒   | ?topic=1492.msg39701          | 2: Unknown: Cannot call session save handler in a recursive manner | fa88003ecde62d7c793a8481b5f76c26 | general    | Unknown |    0 | [{"function":"smf_error_handler"}] |

Error Log (09:55:02 and later)
Stack traces show two errors related to checkSession() in Security.php, called from Likes.php:147:

Code Select Expand

[0x00007d2bd8013340] fatal_lang_error() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Errors.php:237
[0x00007d2bd80131c0] checkSession() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Security.php:710
[0x00007d2bd8013130] call() /var/www/vhosts/aaroads.com/httpdocs/forum/Sources/Likes.php:147

The stack trace points to Likes.php:147, where checkSession() is called during a "Like" action. SMF's checkSession() function (in Security.php) ensures a valid session before processing user actions. If an AJAX request (e.g., liking a post) triggers a session save while another session operation is active, PHP throws the recursion error.
This can happen if:
Multiple AJAX requests fire simultaneously (e.g., a user rapidly clicks "Like").
SMF's session handler is misconfigured or interfered with by another process.

Immediate Fix:
Modify Likes.php to avoid recursive session calls:

I attempted some code modifcation to address this, but the problem continued. I eventually decided to just deactivate Likes so I could move forward with getting the Forum to a stable state.

Session Recursion Error Resolution
Deactivating "Likes" (via Admin > Posts and Topics > Likes) was a smart move. The stack traces in the slow log (e.g., Likes.php:147 triggering checkSession()) confirm the error was tied to that feature. Since no new errors appear in the logs after 09:47:46, the issue is likely resolved.

I believe James was who activated the Likes aspect of SMF, so maybe he can restore it when he has some time.

[Alex]

The one suggestion I might make is to create a read-only, non-forum page off AARoads (e.g. .../forumstatus.html) that just provides a status update on the forum for downtimes like this. The AARoads FB page, which would be a logical place for that information, isn't a good source.

I have a script ready, and it will debut once I finish the AARoads backend reprogramming. It will query the SMF database and output users online and the most recent five posts in addition to a status message regarding if any SMF errors are currently logged. I'll adjust the text above it when forum downtime will occur of if I have to take it offline to address an ongoing problem with bots and the like.

I set up a cron job so that this will not be an actual live look at the forum, but instead be a snapshot generated every 5 minutes. This way the script is not actively querying the database anytime someone is accessing the forum status webpage. Programmingwise, this is encapsulated as a method to make it more efficient as well.

[MaxConcrete]

Thanks for the efforts to get the site back up.

Now I need to dedicate effort to restoring one of my web hosts.

Early this morning I received a message from the host of my DallasFreeways.com domain that "User dallasfr has used up 460% of their bandwidth and 88.4% of their allocated disk space" and "User dallasfr has been suspended for bandwidth overusage". This situation has never occurred in the 10 years I've used this host (webhostingpad). The long term trend for bandwidth on my sites is flat or down, so a bandwidth surge is unexpected.

So all content hosted on that site is unavailable. My host is totally suspended - I cannot even log in with FTP.

I use a directory on DallasFreeways.com for images which I posted on AA Roads. It appears that the bot is following links in posts on AA Roads and also instigating a DDoS attack on those links.

I have a few items on AA roads which are linked to files hosted on Azure. On May 5 and 6 file requests were 5 to 10 times normal rates. There are also spikes about 5x normal on May 12 and May 14. I can't say for a fact that these spikes were caused by the AA Roads DDoS bot, since I have only a few files linked on AA Roads and the overwhelming majority of requests to Azure are direct from dfwfreeways.com and houstonfreeways.com.

I've been in the process of migrating to Azure blob hosting, so I think this will force me to complete that process.

UPDATE: my host says all the requests are coming from a single ip, 166.194.200.53, which is an ATT/Cingular ip in Allen, Texas. Is that consistent with the attack source on AARoads, or is my situation separate and unrelated?

Hello,

I have reviewed your request and found that your account has consumed approximately 450 GB of bandwidth so far. Upon analyzing the account statistics, I discovered that Apache was responsible for the bandwidth consumption.

I further examined the domain access logs and identified that the IP caused the issue.
166.194.200.53

Are you aware of this activity?

Please let us know if there's anything else we can assist you with.

[wxfree]

I was unable to get the forum on my mobile browser for days.  On the computer, it came up most of the time.  It would be available on the computer and not available on mobile at the same time.  Even recently, when the number of guests has been reasonable and the site is working, I couldn't get it on mobile.  It was redirecting from aaroads.com/forum to forum.aaroads.com.  The problem was on my side.  I cleared the browser cache (I'm old enough to remember when that sometimes fixed things, although I haven't done it in a long time) and it works now.  Also, the redirect is gone.

[vdeane]

Interesting that the like button and the 502 bad gateway errors seem to have been related, as the absence of both makes the lack of "new" indicators on posts a less pressing issue.  Without the random chance of starting at the top of a page due to an error or scrolling up to find and look at post likes, I can just start browsing from the new post without worrying about needing to keep track of where it is.

[hotdogPi]

I was unable to get the forum on my mobile browser for days.  On the computer, it came up most of the time.  It would be available on the computer and not available on mobile at the same time.  Even recently, when the number of guests has been reasonable and the site is working, I couldn't get it on mobile.  It was redirecting from aaroads.com/forum to forum.aaroads.com.  The problem was on my side.  I cleared the browser cache (I'm old enough to remember when that sometimes fixed things, although I haven't done it in a long time) and it works now.  Also, the redirect is gone.

What trick did you use to get onto the forum, since it was down serverside?

[wxfree]

I was unable to get the forum on my mobile browser for days.  On the computer, it came up most of the time.  It would be available on the computer and not available on mobile at the same time.  Even recently, when the number of guests has been reasonable and the site is working, I couldn't get it on mobile.  It was redirecting from aaroads.com/forum to forum.aaroads.com.  The problem was on my side.  I cleared the browser cache (I'm old enough to remember when that sometimes fixed things, although I haven't done it in a long time) and it works now.  Also, the redirect is gone.

What trick did you use to get onto the forum, since it was down serverside?

I think what happened was that the forum.aaroads.com redirect got cached and the mobile browser kept going to it.  It seems the redirect wasn't active very long, so it lead to a dead end.  Once I cleared the cache today, it started working normally.  My computer browser never got that redirect and it either loaded the forum or didn't, based on the server conditions.

[kphoger]

Alex:  Do you ever feel like you're playing Whack-A-Mole with a Band-Aid?

I miss the like button

Yes, I had noticed that you were 'liking' approx. 95% of all forum posts...  [/sarc]

If someone posts something thoughtful or interesting that I have no further comment on, I like using it ...

Oh, see, now, that makes total sense.  Instead of either posting or just leaving the author wondering if anyone cared what he wrote, you could do something in between.  I see the light now.

[Molandfreak]

I guess spreading a bit of positivity wasn't quite as well-received as I thought it would be.

[Scott5114]

I was one who wasn't exactly enthusiastic about the idea of the like button, until it was implemented. Indeed, it served a useful purpose as a form of light engagement that doesn't take space up on the page but still indicated appreciation for the post. It was always nice getting a bunch of likes whenever I made a stupid joke.

I never really knew what to think when I purpletexted at people and got likes on that, though.

[GaryV]

what to think when I purpletexted at people and got likes on that

People liked how you were maintaining control of the forum.

[JayhawkCO]

I was one who wasn't exactly enthusiastic about the idea of the like button, until it was implemented. Indeed, it served a useful purpose as a form of light engagement that doesn't take space up on the page but still indicated appreciation for the post. It was always nice getting a bunch of likes whenever I made a stupid joke.

I never really knew what to think when I purpletexted at people and got likes on that, though.

People wish they had the power to shut others up.

[rickmastfan67]

Working fine for me so far; no "NEW" icons is tolerable.

This has been fixed.