Meta Bad Bot – facebookexternalhit/1.1
My website is receiving a massive amount of visits from the Facebook bot.
The user agent varies and can be: facebookexternalhit/1.1 and meta-externalagent/1.1
There are several IPs, but practically all of them are IPV6, as in the example: 2a03:2880:13ff:18::face:b00c
My website is very small and has few pages and few accesses, probably not even 20 people per day. But I must be receiving at least 50 thousand hits from this bot, it seems that something is misconfigured and I don't know how to report this to Facebook/Meta. Maybe they are using this data to train Llama, I don't know, I just know that it is certainly wrong behavior on their part as there is no need to visit the same page several times a day.
2 comments
[ 4.7 ms ] story [ 16.4 ms ] threadhttps://developers.facebook.com/docs/sharing/webmasters/web-...
The article states that these may bypass `robots.txt`, so you may not even bother with that.
There are people who use facebook as a proxy for scraping with the preview functionality.
https://datadome.co/threat-research/how-facebook-was-used-as...
>With technical detection only, the fraudulent requests would have been indistinguishable from legitimate API calls, since they had the Facebook user agent and IP address. However, our heuristic analysis uncovered that certain parameters, unlikely to be used by humans, were overrepresented in the URLs that Facebook requested. Certain use cases were also unlikely: for example, a human user would typically share a link to a specific ad, not to a page of search results.
According to that article, looking at the requests and discerning patterns of bogus traffic would be preferable to more drastic approaches such as blocking or rate limiting indiscriminately, and risking to impact legitimate traffic.