HN Explorer

Ask YC: Stopping bots at proxy server level

3 points by comforteagle ↗ HN
I'm trying to block bad bots/spider on my app server. I'm using nginx as a reverse proxy though and so the IP that appears to my app server is always the proxy.

How can I either capture the actual browser's IP on the app server -or- start stopping bad bots/spiders (not Google or adsense's bot) at the reverse proxy level?

1 comment

[ 3.7 ms ] story [ 10.0 ms ] thread
comforteagle
Answer (not YC provided): nginx passes IPs in X-Real-IP HTTP header. If your bot trap is not customizable, you can set up mod_rpaf Apache module on app instance.