I hope that the next big shift will be to undo the asymmetry that crept into the internet quickly after it first became popular. Let us host stuff at home. Let us run odd and strange and great systems wherever we are. Undo the cloud, undo the capture of the net – I'm old enough to remember when we just had a bunch of boxen under a desk somewhere, and it was pretty great.
I guess this is the 'meat' of the article. Interesting that they regard reddit as a 'niche' quirky corner of the internet. To me they are well over the peak of their enshittification journey with mainly bots and overzealous mods and pay-to-play accounts. Just like digg and others before something else will take their place soon enough as the place where enthusiasts share valuable content before businesses poke their nose in and we repeat the cycle again.
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old. For those of you old enough, think back to the Internet not of the last 15 years but of the last 35. We’ve lost some of what made that early Internet great, but there are indications that we might finally have the incentives to bring more of it back.
> It seems increasingly likely that in our future, AI-driven Internet — assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them — it’s the creative, local, unique, original content that’ll be worth the most. And, if you’re like us, the thing you as an Internet consumer are craving more of is creative, local, unique, original content. And, it turns out, having talked with many of them, that’s the content that content creators are most excited to create.
Colour me cynical mauve, but you would have to pry the current business model out of advertisers' cold dead hands.
What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
The Open Letter reads naive at best, asking us to imagine an Internet where creators are rewarded for "filling the holes in human knowledge". We all know that is not what sells, and that the opposite of this will continue to inundate the infosphere.
Everyone seems to be dismissing this without reading it. As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers. That would indeed be a new internet business model.
This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.
In short: we'll have the machines tell the creators what to create.
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.
> As we think about our role at Cloudflare in this developing market, it's not about protecting the status quo but instead helping catalyze a better business model for the future of Internet content creation. That means creating a level playing field. Ideally there should be lots of AI companies, large and small, and lots of content creators, large and small.
Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
The "level playing field" rhetoric reminds me so much of Apple talking about the App Store. This new internet business model is just the App Store, substituting websites for apps and Cloudflare for Apple. The system only works with some middleman between the AI companies and the content creators.
Something that stuck out on this was the notion of the LLMs as models of human knowledge, and the notion that we could "see the gaps". There's been an interesting debate* in journalism over the concept of neutrality - what it means to "just present the facts," and whether that's actually a useful service, and what are actual facts, and how do you define a fact (it's gotten rather epistemological), and that's just for things that actually happened in the real world. I think what's not taken into account here is how much of the world is not able to be summarized into One Universal Truth - how much of what we look for is actually the preference of the author ("what are the best new albums this year?" "how do I cook chicken?"), or an intellectual synthesis of a subject (basically anything written by that acoup guy), or just some entertainment or random digression on a topic. I think this is part of the crux of the argument behind the artists vs LLMs debate (you know, aside from all the economic exploitation) - even the act of writing a summary is a creative act when performed by a human that generates something new, whereas an LLM is, as far as we can tell from both the mechanics and the actual output, a remix of existing content. I think our appetite for the latter is less than we think it is.
I don't fundamentally think that Cloudflare's view here is _wrong_ - by and large, when I google what internal temperature to cook pork to so I don't die, I'm not looking for an opinion or someone's life story - but I think I'm much more interested in how we create the weird niches that create the "knowledge" for the blender. Cloudflare alludes to it with the talk about Reddit and whatnot, but I'd love to know what the plan is to actually create and nurture those communities where people can really get weird into whatever topic they're interested in. Right now we're all sort of just ghettoized into various Facebook communities or whatever, but recreating an actual vibrant communal internet where people can find their weirdo subcultures and actually negotiate on some kind of reasonable footing with the LLM scrapers (who were the social network people, who were the search engine people, who have always been the ad people) would be a genuine improvement to the internet.
> Make no mistake, the Internet has never been free. There's always been a reward system that transferred value from consumers to creators and, in doing so, filled the Internet with content. Had the Internet not had that reward system it wouldn't be nearly as vibrant as it is today.
This is an attempt to rewrite history. Back in the day, we stood up servers at our own expense and filled them with content for free, for nothing other than the fun of it. In fact, the "vibrancy" of the internet appears to be inversely correlated to the number of people using it to generate a profit.
”What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old.”
I don’t know if I see a company extracting rent from other people as a win…
My concern is Cloudflare will implement this plan in a way that makes it very profitable for large players and absolutely kills any new entrants to the market in the future.
A whole new SEO style race to the bottom, as an arms race between quality and exploiting monetization begins, with cloudflare arbitraging every step of every interaction. Sounds great! Let's enshittify everything, and give total control to a private company who's more than happy to grant undue power to governments and bureaucrats to oppress and harass and exploit citizens, as long as they make a buck from every click and harvest every bit of data traversing every network on the planet.
I think providers of so-called "answer engines" will read the writing on the wall and find new ways to support content creators in order to keep their databases fresh, relevant, and centered on human perspectives. I also think that to see how the economics will play out, you only need to look at similarly centralized systems like Apple's App Store and Google's AdSense.
Because the providers will act as a single tunnel that all content passes through before reaching the end user, the tolls they collect will be large. So, I don't doubt that there will still be opportunities for content creators to earn money as answer engines siphon off more and more of the web's traffic, but expect those opportunities to be broadly low-paying, falling decidedly in the "side hustle" category.
AI providers will want to incentivize content creation. There will still be a glut of ready providers, and little reason for providers to make anything but small, nominal payments.
Killing the open internet is generally a good thing. Large companies and hostile nation states benefit from the open internet massively, while providing none of it back. The Chinese intranet is not accessible to EU/NA scrapers, but they can read all of our scientific journals. Facebook posts aren’t freely available for you to scrape, but llama is trained on obscure usenet posts and the entire comment history of reddit and hackernews. North Korea has their own linux distribution. Etc.
If the open internet is already dead (and it is already dead), it’s better to accept that reality and silo off the good parts behind paywalls so that people can get paid, rather than to let bad people benefit massively from it while they build their walled gardens. This has been a long time coming.
68 comments
[ 3.5 ms ] story [ 55.2 ms ] thread(https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=tru...)
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old. For those of you old enough, think back to the Internet not of the last 15 years but of the last 35. We’ve lost some of what made that early Internet great, but there are indications that we might finally have the incentives to bring more of it back.
> It seems increasingly likely that in our future, AI-driven Internet — assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them — it’s the creative, local, unique, original content that’ll be worth the most. And, if you’re like us, the thing you as an Internet consumer are craving more of is creative, local, unique, original content. And, it turns out, having talked with many of them, that’s the content that content creators are most excited to create.
What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
The Open Letter reads naive at best, asking us to imagine an Internet where creators are rewarded for "filling the holes in human knowledge". We all know that is not what sells, and that the opposite of this will continue to inundate the infosphere.
Fantastic! I wish I could undermine their clickbait business model even more..
> But there’s reason for optimism
You mean Cloudflare being investigated for antitrust?
This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.
Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
The "level playing field" rhetoric reminds me so much of Apple talking about the App Store. This new internet business model is just the App Store, substituting websites for apps and Cloudflare for Apple. The system only works with some middleman between the AI companies and the content creators.
I don't fundamentally think that Cloudflare's view here is _wrong_ - by and large, when I google what internal temperature to cook pork to so I don't die, I'm not looking for an opinion or someone's life story - but I think I'm much more interested in how we create the weird niches that create the "knowledge" for the blender. Cloudflare alludes to it with the talk about Reddit and whatnot, but I'd love to know what the plan is to actually create and nurture those communities where people can really get weird into whatever topic they're interested in. Right now we're all sort of just ghettoized into various Facebook communities or whatever, but recreating an actual vibrant communal internet where people can find their weirdo subcultures and actually negotiate on some kind of reasonable footing with the LLM scrapers (who were the social network people, who were the search engine people, who have always been the ad people) would be a genuine improvement to the internet.
* read "hellish swamp diving affair"
This is an attempt to rewrite history. Back in the day, we stood up servers at our own expense and filled them with content for free, for nothing other than the fun of it. In fact, the "vibrancy" of the internet appears to be inversely correlated to the number of people using it to generate a profit.
I don’t know if I see a company extracting rent from other people as a win…
What could possibly go wrong?
Because the providers will act as a single tunnel that all content passes through before reaching the end user, the tolls they collect will be large. So, I don't doubt that there will still be opportunities for content creators to earn money as answer engines siphon off more and more of the web's traffic, but expect those opportunities to be broadly low-paying, falling decidedly in the "side hustle" category.
AI providers will want to incentivize content creation. There will still be a glut of ready providers, and little reason for providers to make anything but small, nominal payments.
If the open internet is already dead (and it is already dead), it’s better to accept that reality and silo off the good parts behind paywalls so that people can get paid, rather than to let bad people benefit massively from it while they build their walled gardens. This has been a long time coming.