The conversation

bob1029HNSource author
Business & incomeExperience excerpt · English

I have found that attempting to catalog and record all of the naughty behavior my web server sees …

… I have found that attempting to catalog and record all of the naughty behavior my web server sees is the highest risk to DDOS so far. Logging lines like "banned client rejected" every time they try to come in the door is shooting yourself in the foot with regard to disk wear, IO utilization, et. al. There is no reason you should be logging all of that background radiation to disk or even thinking about it. If your web server cant handle direct exposure to the hard vacuum of space, it can be placed behind a proxy/CDN (i.e., another web server that doesn't suck).

Original discussionAggressive bots ruined my weekend104 source comments · at import
2

What was this responding to?

Parent context · original excerpt

IT
ItsBobHN

I had a website earlier this year running on Hetzner. It was purely experimenting with some ASP.NET stuff but when looking at the logs, I noticed a shit-load of attempts at various WordPress-related endpoints. I then read something about a guy who deliberately put a honeypot in his robots.txt file. It was pointing to a completely bogus endpoint. Now, the theory was, humans won't read robots.txt so there's no danger, but bots and the like will often read robots.txt (at least to figure out what you have... they'll ignore the "deny" for the most part!) and if they try and go to that fake endpoint you can be 100% sure (well, as close as possible) that it's not a human and you can ban them. So I tried that. I auto-generated a robots.txt file on the fly. It was cached for 60 seconds or so as I didn't want to expend too many resource on it. When you asked for it, you either got the cached one o…

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 2

Direct replies available in this import. Original-language excerpts.

IT
ItsBobHN

> If a client has an IP in this set, they get a simple UTF8 constant string in the response body "You have exceeded resource limits, please try again later". Would a simple 429 not do the same thing? You could log repeated 429's and banish accordingly.

Full original
MA
marcosdumayHN

> they get a simple UTF8 constant string in the response body "You have exceeded resource limits, please try again later" I imagine they get a 429 response code, but if they don't, you may want to change that. I do think you are on the right place in that it's important to let those requests get the correct error, so if innocent people are affected, they at least get to see there's something wrong.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.