Board Update: On Last Night's Issues at 11:00PM
Hey all,
I wanted to let you all know what went on last night with the boards.
At 10:50PM last night I was notified by our PRTG Network Monitor that the backend API had stopped responding during the game. I immediately reached out to Will Stewart to check if we were getting connectivity reports and emails from subscribers.
At 11:00PM we were serving 1.28 Million requests per 15 minutes, or 85,333 Requests per Minute, 1422 Requests Per Second.
I love our community, but we are not that big. Normally we have 1000-2000 active users during games.
Looking at the data I have at Cloudflare, it looks like there were 1.49 million requests by bots last night between 10:45 and 11:15PM. The servers were responding again by 11:30PM according to PRTG.
I have a set of IP addresses that were each doing 3.5k requests during this time period. I'm going through the process of compiling them and banning them as well as submitting them to the authorities. I will also be cross referencing them with our user database and see if anyone matches up.
Nobody's information has been compromised, the only thing that "happened" was that we received over a million fake requests during the roughly 20 minute period.
Thank you all for being patient with us. I'll be putting extra security in place(Cloudflare offers extra DDoS protection, but we have most of those features disabled as this has never happened before).
Feel free to ask me any questions, or if you have information about the incident please email me at [email protected]
Posted: 10/28/2022 at 01:41 PM ET
Thread Replies (8)
Board Update: On Last Night's Issues at 11:00PM
◄
-- John Donna Choi
Oct 28, 2022 at 01:41 PM ET
John Donna Choi
•
10/28/22 01:41 PM ET
Why would anybody want to screw with this site.**
-- hok1eboy
Oct 28, 2022 at 08:51 PM ET
hok1eboy
•
10/28/22 08:51 PM ET
Thanks for explaining, it was very frustrating. Makes sense, go get em!**
-- Polito1
Oct 28, 2022 at 05:16 PM ET
Polito1
•
10/28/22 05:16 PM ET
Holy moly, John. Thank you for your diligence and expertise, again!**
-- PhotoHokieNC
Oct 28, 2022 at 04:44 PM ET
PhotoHokieNC
•
10/28/22 04:44 PM ET
John I feel for you and the rest of the TSL team, why did someone do this?
-- Stech
Oct 28, 2022 at 04:38 PM ET
Stech
•
10/28/22 04:38 PM ET
I only experienced on the sub board. How do bots attack with a password?**
-- TomTurkey
Oct 28, 2022 at 03:44 PM ET
TomTurkey
•
10/28/22 03:44 PM ET
From what I've been seeing the attack was sub board only
-- John Donna Choi
Oct 28, 2022 at 05:13 PM ET
John Donna Choi
•
10/28/22 05:13 PM ET
It was the Russians?!? Chinese?!? Both?!? We bin hacked!!!**
-- aint2Hokie4me
Oct 28, 2022 at 02:59 PM ET
aint2Hokie4me
•
10/28/22 02:59 PM ET
Yikes!... didn't know we rated bots**
-- OldRetiredHokie
Oct 28, 2022 at 02:15 PM ET
OldRetiredHokie
•
10/28/22 02:15 PM ET