Does robots.txt not work? Is the issue that you want to be indexed but otherwise not have them use what they crawl for other stuff?
If I'm interpreting this right, I agree that it's scummy of them to set up a kind of social contract where they can index your stuff and then decide actually they can do whatever they want with it. This is effectively what happened with news sites I think. But at this point, they are at the stage of reaping the rewards of the monopoly they built, so antitrust laws are unfortunately the likely recourse.
3 comments
[ 3.3 ms ] story [ 16.5 ms ] threadIf I'm interpreting this right, I agree that it's scummy of them to set up a kind of social contract where they can index your stuff and then decide actually they can do whatever they want with it. This is effectively what happened with news sites I think. But at this point, they are at the stage of reaping the rewards of the monopoly they built, so antitrust laws are unfortunately the likely recourse.