A few days ago they were identifying as “FaceBot” (not "FacebookBot").
When that began to be blocked, they switched to reusing the “Facebookexternalhit” user agent they also use for redirects; one people are less likely to block.
Meta as a company always acts like the most unprofessional idiots if they feel under pressure. They have all the time and ressources to do it right and they never do.
isn't this crawler to generate previews on Facebook? they have others for training and AI stuff. oh well, one never knows with Meta...
——
The facebookexternalhit/1.1 user agent you're seeing in the logs is a Facebook crawler, specifically used by Facebook’s servers to fetch content (like Open Graph metadata) when:
Someone shares a link on Facebook or Messenger
Facebook needs to generate a preview (title, image, description) for that URL
5 comments
[ 0.19 ms ] story [ 24.1 ms ] threadA few days ago they were identifying as “FaceBot” (not "FacebookBot").
When that began to be blocked, they switched to reusing the “Facebookexternalhit” user agent they also use for redirects; one people are less likely to block.
—— The facebookexternalhit/1.1 user agent you're seeing in the logs is a Facebook crawler, specifically used by Facebook’s servers to fetch content (like Open Graph metadata) when:
Someone shares a link on Facebook or Messenger
Facebook needs to generate a preview (title, image, description) for that URL