[flagged]
partial answer: the major labs (Anthropic, OpenAI) do respect robots.txt for their named crawlers, so blocking ClaudeBot/GPTBot in robots.txt works for those specific bots. What you can't easily opt out of is the…
Timing's also worth nothing. the investments piece has been reported on for over a year. It becomes a probe right before liquidity, which makes both sides look opportunistic rather than principled.
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
partial answer: the major labs (Anthropic, OpenAI) do respect robots.txt for their named crawlers, so blocking ClaudeBot/GPTBot in robots.txt works for those specific bots. What you can't easily opt out of is the…
Timing's also worth nothing. the investments piece has been reported on for over a year. It becomes a probe right before liquidity, which makes both sides look opportunistic rather than principled.