HN Explorer

18 comments

[ 2.8 ms ] story [ 46.8 ms ] thread
thepuppet33r
I feel like this is cool, but is also a harbinger of how the Internet is becoming a bunch of bots trying to trick other bots away from the real content, which is how often being written by bots (at least in part).
blantonl
It's bots all the way down...
supportengineer
We should turn all of this shit off, go outside, and breathe some fresh air.
throwway120385
I often dream of working my great grandfather's job as a finish carpenter.
derefr
"Becoming"? Dumb overzealous bots (e.g. broken API clients that retry immediately forever; scrapers that want your 5PB dataset right now; etc) that won't go away — so you have to confuse/stall/crash them to keep your service up — have been a thing since the '90s. They're the reason that some NOC admins block entire ranges of APNIC and AFRINIC, and why DNSRBLs exist.

Here's a fun read on dealing with this sort of thing in the context of a Tor Hidden Service: https://www.hackerfactor.com/blog/index.php?/archives/777-St...

alyandon
I wrote a similar tool years ago to get better acquainted with Go. It was an endless procedurally generated website of linked pages that contained things that looked like email addresses along with username/password combinations.

The logs were fun to watch.

surfingdino
We need this to punish all those AI scrapers... I approve!
Bluestein
Lets call it "AI ay ay!" :)

  PS. Unofficial slogan:
  "PAIn where you need it,
  when you need it!"
metadat
Is there a demo site where we can see this in action?
ChrisMarshallNY
Sounds fun, but it also sounds like something they could learn, quickly.
gessha
Two can play the learning game. Set up an adversarial ML-enabled honeypot and the giant waste of electricity begins.
cratermoon
Years and YEARS ago Ronald F. Guilmette came up with wpoison, a CGI script that generated random web content containing made-up e-mail addresses. It existed to poison spammer's email address databases.

The spirit of that effort lives on in an improved version at https://gitlab.com/gloomytrousers/wpoison

[deleted]
(comment deleted)
OutOfHere
I don't know why this is necessary. If a particular IP is hammering your site, it's sufficient to intelligently give HTTP 429 errors to encourage the IP to discipline and temper its requests. It works as well as anything else. If you don't want clients to access data, put it behind a login, not on the open internet.
Doxin
There's a LOT of misbehaving bots out there that will just relentlessly keep hammering an endpoint that returns error codes.
OutOfHere
Well, that's because the site too is misbehaving there by always erroring. The site should give the correct response based on a bucket algorithm, and this will cause the bot to ease off. Effectively a rate limit is implemented for mutually satisfactory operations.
Doxin
A bot that ignores error codes isn't going to suddenly start behaving itself if your site has a rate limit. It'll simply ignore the rate limiting errors and move on to the next page the second the site does return a non-error response.