I just got this WordPress plugin approved on wp.org.
It lets site owners control automated access from AI crawlers and agents with per-site and per-post rules: allow, deny, teaser previews, and rate limits. It can also return 402 if you want to experiment with paid access.
If a crawler signs requests, the plugin can verify RFC 9421 HTTP Message Signatures. Most crawlers do not sign yet, but I wanted the path to be there.
It also publishes llms.txt, a JSON feed, and per-post Markdown endpoints.
1 comment
[ 3.7 ms ] story [ 15.2 ms ] threadIt lets site owners control automated access from AI crawlers and agents with per-site and per-post rules: allow, deny, teaser previews, and rate limits. It can also return 402 if you want to experiment with paid access.
If a crawler signs requests, the plugin can verify RFC 9421 HTTP Message Signatures. Most crawlers do not sign yet, but I wanted the path to be there.
It also publishes llms.txt, a JSON feed, and per-post Markdown endpoints.
Source and other integration surfaces (Node, Python, proxy): https://github.com/OpenBotAuth/openbotauth https://openbotauth.com/developers
If you run a content site or a crawler team, I would love feedback on signing defaults and replay protection expectations.