I did.
Pay me for the content I posted,
including stories posted to some of the most popular subreddits that generated thousands of comments ("engagement")
and reached #1 for the day.
The thousands of comments generating discussion (more "engagement").
The subreddit I moderated.
All this writing,
mine and thousands of others',
is what makes reddit's content valuable.
Pay the value creators.
The idea of ignoring robots.txt when it is abused for greed sounds okay, but it would be much better if the industry was in a spot where we respected each others' requests, and peers don't make requests that put legitimate, honest actors at a disadvantage compared to rouge crawlers
so Bing can create a different entity in India for crawling and use that to show reddit content in search results?
I'm sure they would have already done it it if it was a possibility.
If they bought the index from a company that had compiled it legally according to the law of another country they might be okay.
The terms of service bind the legal entity that is accessing Reddit’s servers, whereas reproduction of the data is governed by copyright law. Microsoft would still have to abide by US copyright law but there’s some wiggle room in terms of who is doing the scraping.
This doesn't make sense. It sounds like reddit made the appropriate robots.txt changes, and Microsoft is respecting them. But the quote is representing that reddit is taking some action beyond that to effectively block Microsoft.
> He specifically named Microsoft, Anthropic, and Perplexity for refusing to negotiate, saying it has been “a real pain in the ass to block these companies.”
Surely updating robots.txt isn't the pain in the ass he is referring to?
A testament to how shit are modern day search results, that a single site can set such demands, knowing that without its data the quality of dataset drops a lot.
15 comments
[ 4.8 ms ] story [ 45.0 ms ] threadAll this writing, mine and thousands of others', is what makes reddit's content valuable. Pay the value creators.
An index is a derivative-enough work that copyright doesn't apply.
The idea of ignoring robots.txt when it is abused for greed sounds okay, but it would be much better if the industry was in a spot where we respected each others' requests, and peers don't make requests that put legitimate, honest actors at a disadvantage compared to rouge crawlers
There are 195 other countries you could scrape from though.
As for whether an index constitutes copyright infringement, that depends on its construction, but in the US generally not.
The terms of service bind the legal entity that is accessing Reddit’s servers, whereas reproduction of the data is governed by copyright law. Microsoft would still have to abide by US copyright law but there’s some wiggle room in terms of who is doing the scraping.
> He specifically named Microsoft, Anthropic, and Perplexity for refusing to negotiate, saying it has been “a real pain in the ass to block these companies.”
Surely updating robots.txt isn't the pain in the ass he is referring to?