March 11th-13th Downtime

Started by ZLoth, March 13, 2025, 10:28:32 AM

Previous topic - Next topic

ZLoth

What highway incident caused the forum and website to be down from Tuesday, March 11th to Thursday, March 13th?
Welcome to Breezewood, PA... the parking lot between I-70 and I-70.


JayhawkCO

Quote from: ZLoth on March 13, 2025, 10:28:32 AMWhat highway incident caused the forum and website to be down from Tuesday, March 11th to Thursday, March 13th?

See https://www.aaroads.com/forum/index.php?topic=35763.0

kphoger


He Is Already Here! Let's Go, Flamingo!
Dost thou understand the graveness of the circumstances?
Deut 23:13
Male pronouns, please.

Quote from: PKDIf you can control the meaning of words, you can control the people who must use them.

JayhawkCO


Alex

Quote from: ZLoth on March 13, 2025, 10:28:32 AMWhat highway incident caused the forum and website to be down from Tuesday, March 11th to Thursday, March 13th?

Leading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out. My options to boost security were limited due to it being a managed server. All I could do was try Cloudflare and implement .htaccess codes. Both of these efforts were futile.

An option presented to me was to switch the site to a VPS server with root access (where I can block IP's, set up firewalls, utilize other security protocals). Since they offer a 30-day money back guarantee, I opted to proceed with this, which ultimately led the site to go down due to the transition and reset-up process.

kphoger

Quote from: Alex on March 13, 2025, 11:01:52 AMLeading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out. My options to boost security were limited due to it being a managed server. All I could do was try Cloudflare and implement .htaccess codes. Both of these efforts were futile.

An option presented to me was to switch the site to a VPS server with root access (where I can block IP's, set up firewalls, utilize other security protocals). Since they offer a 30-day money back guarantee, I opted to proceed with this, which ultimately led the site to go down due to the transition and reset-up process.

My bad.  It was this one:

QuoteOfficers pinned a driver to the ground on a busy Hong Kong street on Tuesday evening after his car hit four other vehicles, including a police van.⁠ The drama unfolded at about 9.19pm when a police patrol van spotted a suspicious Mercedes-Benz parked near 178 Portland Street in Mong Kok.⁠ Officers approached to investigate but the car suddenly sped off. Police gave chase.⁠ The driver attempted to manoeuvre the Mercedes out of the jam but hit four other vehicles, including the police van in pursuit.⁠

He Is Already Here! Let's Go, Flamingo!
Dost thou understand the graveness of the circumstances?
Deut 23:13
Male pronouns, please.

Quote from: PKDIf you can control the meaning of words, you can control the people who must use them.

webny99

Quote from: ZLoth on March 13, 2025, 10:28:32 AMWhat highway incident caused the forum and website to be down

Quote from: kphoger on March 13, 2025, 11:17:08 AMIt was this one:

QuoteOfficers pinned a driver to the ground on a busy Hong Kong street on Tuesday evening after his car hit four other vehicles, including a police van.⁠ The drama unfolded at about 9.19pm when a police patrol van spotted a suspicious Mercedes-Benz parked near 178 Portland Street in Mong Kok.⁠ Officers approached to investigate but the car suddenly sped off. Police gave chase.⁠ The driver attempted to manoeuvre the Mercedes out of the jam but hit four other vehicles, including the police van in pursuit.⁠

Well, it's good to see someone has made sense of it all.  :D 

ZLoth

#7
Quote from: Alex on March 13, 2025, 11:01:52 AMAll I could do was try Cloudflare and implement .htaccess codes. Both of these efforts were futile.

Ah. My primary use of Cloudfare is as a DNS provider only, no additional services. This is so that I can access certain services on my home server through a reverse proxy (Nginx) using a fully qualified domain name and a SSL connection using a LetsEncrypt certificate.
Welcome to Breezewood, PA... the parking lot between I-70 and I-70.

NE2

pre-1945 Florida route log

I accept and respect your identity as long as it's not dumb shit like "identifying as a vaccinated attack helicopter".

Max Rockatansky

Quote from: JayhawkCO on March 13, 2025, 10:49:39 AM
Quote from: kphoger on March 13, 2025, 10:47:04 AM
Quote from: JayhawkCO on March 13, 2025, 10:34:25 AMSee https://www.aaroads.com/forum/index.php?topic=35763.0

He's asking about the highway incident.

This was the one

Some people's kids, man.

Why am I not surprised this was something from Victorville?  Victorville has a motley assortment of desert dwellers that would give Florida Man a run for his money.

1995hoo

Quote from: kphoger on March 13, 2025, 11:17:08 AM
Quote from: Alex on March 13, 2025, 11:01:52 AMLeading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out. My options to boost security were limited due to it being a managed server. All I could do was try Cloudflare and implement .htaccess codes. Both of these efforts were futile.

An option presented to me was to switch the site to a VPS server with root access (where I can block IP's, set up firewalls, utilize other security protocals). Since they offer a 30-day money back guarantee, I opted to proceed with this, which ultimately led the site to go down due to the transition and reset-up process.

My bad.  It was this one:

QuoteOfficers pinned a driver to the ground on a busy Hong Kong street on Tuesday evening after his car hit four other vehicles, including a police van.⁠ The drama unfolded at about 9.19pm when a police patrol van spotted a suspicious Mercedes-Benz parked near 178 Portland Street in Mong Kok.⁠ Officers approached to investigate but the car suddenly sped off. Police gave chase.⁠ The driver attempted to manoeuvre the Mercedes out of the jam but hit four other vehicles, including the police van in pursuit.⁠

Alex said "GET HEAD," so obviously the driver must have been either receiving, or giving, a blowjob at the time of the incident.
"You know, you never have a guaranteed spot until you have a spot guaranteed."
—Olaf Kolzig, as quoted in the Washington Times on March 28, 2003,
commenting on the Capitals clinching a playoff spot.

"That sounded stupid, didn't it?"
—Kolzig, to the same reporter a few seconds later.

Voyager

Quote from: ZLoth on March 13, 2025, 10:28:32 AMWhat highway incident caused the forum and website to be down from Tuesday, March 11th to Thursday, March 13th?

Wasn't me this time.
AARoads Forum Original

CtrlAltDel

Quote from: Alex on March 13, 2025, 11:01:52 AMLeading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out. My options to boost security were limited due to it being a managed server. All I could do was try Cloudflare and implement .htaccess codes. Both of these efforts were futile.

An option presented to me was to switch the site to a VPS server with root access (where I can block IP's, set up firewalls, utilize other security protocals). Since they offer a 30-day money back guarantee, I opted to proceed with this, which ultimately led the site to go down due to the transition and reset-up process.

Appreciate everything you do for this board, Alex.  :clap:
I-290   I-294   I-55   (I-74)   (I-72)   I-40   I-30   US-59   US-190   TX-30   TX-6

Chris

Quote from: Alex on March 13, 2025, 11:01:52 AMLeading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out.

On the Dutch road forum we also had this issue recently, with a continuous 100% CPU load on the VPS by requests from Singapore IP addresses.

These are probably AI crawlers which ignore robots.txt.

A temporary geoblock of IPs from the originating country may help. Our admin is also looking into a solution which feeds such IPs junk data back instead of loading the server.

rschen7754

Quote from: Chris on March 14, 2025, 03:12:12 PM
Quote from: Alex on March 13, 2025, 11:01:52 AMLeading up to this starting around Wednesday last week was the 900,000+ GET and HEAD requests from Hong Kong, China IP addresses overwhelming the managed server that hosted AARoads. The responsiveness of the site degraded to awful, with long load times and pages timing out.

On the Dutch road forum we also had this issue recently, with a continuous 100% CPU load on the VPS by requests from Singapore IP addresses.

These are probably AI crawlers which ignore robots.txt.

A temporary geoblock of IPs from the originating country may help. Our admin is also looking into a solution which feeds such IPs junk data back instead of loading the server.

I usually feed the IPs into sites like whatismyipaddress.com and ipinfo.io. If it comes back proxy or webhost, there is really no good reason for them to be hitting the site that fast. ipinfo can parse out what block it is from and I block the entire range.

With Microsoft you have to be careful, it is hard to tell what is Bing and what is some VM somebody is running on Azure.



Opinions expressed here on belong solely to the poster and do not represent or reflect the opinions or beliefs of AARoads, its creators and/or associates.