Ask HN: How to stop an AWS bot sending 2B requests/month?

Hire a lawyer and have him send the bill for his services to them immediately with a note on the consequences of ignoring his notices. Bill them aggressively.

Do you receive, or expect to receive any legitimate traffic from AWS Singapore? If not, why not blackhole the whole thing?

You don't even need to send a response. Just block the traffic and move on

If it follows redirects, have you tried redirecting it to its own domain?

Just find a Hoster with low traffic egress cost, reverse proxy normal traffic to Cloudflare and reply with 2GB files for the bot, they annoy you/cost you money, make them pay.

block the IPs or setup an WAF on AWS if you cannot be on Cloudflare.

if it follows redirect, redirct him to a 10gb gzip bomb

Write to aws abuse team

Making the obviously-abusive bot prohibitively expensive is one way to go, if you control the terminating server.

gzip bomb is good if the bot happens to be vulnerable, but even just slowing down their connection rate is often sufficient - waiting just 10 seconds before responding with your 404 is going to consume ~7,000 ports on their box, which should be enough to crash most linux processes (nginx + mod-http-echo is a really easy way to set this up)

Null-route the entirety of AWS ip space.

Have ChatGPT write you a sternly worded cease and desist letter and send it to Amazon legal via registered mail.

AWS has become rather large and bloated and does stupid things sometimes, but they do still respond when you get their lawyers involved.

Maybe add this IP to a blacklist? https://iplists.firehol.org/ It would be easier to pressure AWS when it is there

What kind of content do you serve? 700 RPS is not a big number at all, for sure not enough to qualify as a DoS. I'm not surprised AWS did not take any action.

Take a look at https://github.com/pingooio/pingoo

It's a reverse-proxy / load balancer with built-in firewall and automatic HTTPS. You will be able to easily block the annoying bots with rules (https://pingoo.io/docs/rules)

I had this issue on one of my personal sites. It was a blog I used to write maybe 7-8 years ago. All of a sudden, I see insane traffic spikes in analytics. I thought some article went viral, but realized it was too robotic to be true. And so I narrowed it down to some developer trying to test their bot/crawler on my site. I tried asking nicely, several times, over several months.

I was so pissed off that I setup a redirect rule for it to send them over to random porn sites. That actually stopped it.

> I've tried 30X redirects (which it follows) to no avail

Make it follow redirects to some kind of illegal website. Be creative, I guess.

The reasoning being that if you can get AWS to trigger security measures on their side, maybe AWS will shut down their whole account.

As others have suggested you can try to fight back depending on the capabilities of your infrastructure. All crawlers will have some kind of queuing system. If you manage to cause for the queues to fill up then the crawler wont be able to send as many requests. For example, you can allow the crawler to open the socket but you only send the data very slowly causing the queues to get filled quickly with busy workers.

Depending on how the crawler is designed this may or may not work. If they are using SQS with Lambda then that will obviously not work but it will fire back nevertheless because the serverless functions will be running for longer (5 - 15 minutes).

Another technique that comes to mind is to try to force the client to upgrade the connection (i.e. websocket). See what will happen. Mostly it will fail but even if it gets stalled for 30 seconds that is a win.

> Thankfully, CloudFlare is able to handle the traffic with a simple WAF rule and 444 response to reduce the outbound traffic.

This is from your own post, and is almost the best answer I know of.

I recommending you configure a Cloudflare WAF rule to block the bot - and then move on with your life.

Simply block the bot and move on with your life.

There might be some ideas to dig here: https://news.ycombinator.com/item?id=41923635

If it follows the redirect I would redirect it to random binary files hosted by Amazon, then see if it continues to not require any further action

What kind of website is this that makes it so lucrative to run so many requests?

if they have some service up on the machines the bot connect from then u can redirect them to themselves.

otherwise, maybe redirect to aws customer portal or something -_- maybe they will stop it if it hit themselves...

Use a simple block rule, not a WAF rule, those are free.

Silly suggestion: feed them bogus DNS info. See if you can figure out where their DNS requests are coming from.