RSS is still alive and kicking. I came here via RSS.
But the masses browse a locked down web. That is true.
We might have another shot at an open web for the masses via the whole crypto movement.
If content goes on decentralized blockchains in the future, nobody can lock the content down like big tech did for Twitter, Facebook, Instagram, TikTok etc.
Projects like Arweave show how it is possible to store huge amounts of data on a blockchain without using much resources.
If we get there, it will be trivial to build blockchain-to-rss bridges. I am ready to build some to get my favorite content into my RSS reader.
My feeling is that there might be a way towards a decentralized web via financial incentives. In a decentralized web, crypto currencies can float around freely. Making it easy for the actors to earn. Old platforms will probably be slow to open up to crypto currencies.
Consider a “decentralised” blockchain system that does the work of The Pirate Bay – providing hashes of particular files, along with their names, so that people can select and request data from the P2P network.
How, exactly, does this system help with RSS in any way?
Additionally, how does this system help with the “censorship” issues that The Pirate Bay has? Even when The Pirate Bay remains up, that doesn't mean anybody's seeding.
And a blockchain-based system has to have this limitation. If all the data is directly on the blockchain, nobody can actually download it all, so the blockchain becomes unverifiable and ceases to have any cryptographic properties at all. (In short, there's no benefit to a blockchain at all.)
How, exactly, does this system
help with RSS in any way?
You cannot make a Twitter-2-RSS bridge because Twitter block access to the data. In a decentralized system, access to the data is inherent.
Even when The Pirate Bay remains up,
that doesn't mean anybody's seeding
That is where the financial incentives come into play. Seeding will be economically incentivised.
If all the data is directly on the
blockchain, nobody can actually
download it all
Blockchains have come a long way since Satoshis version in 2009. We have approaches now where participants only need to download arbitrarely small parts of the chain. Yet the network as a whole keeps the chain intact.
> In a decentralized system, access to the data is inherent.
Well, yeah – but we already have that. It's called the Fediverse. No blockchain in sight. (In fact, it predates Bitcoin.)
> That is where the financial incentives come into play. Seeding will be economically incentivised.
The margin for any given piece of content will be low, so there's an economic incentive to honour takedown requests, no matter how fraudulent or unenforceable, if the effort to check whether they're valid is lower than the expected average reward minus the expected average legal cost.
> We have approaches now where participants only need to download arbitrarely small parts of the chain. Yet the network as a whole keeps the chain intact.
We had those back in Satoshi's time, too. They work by having n centralised trusted nodes, then asking all your peers to double-check what you got from those trusted nodes. That approach is, surprisingly enough, also vulnerable to censorship!
(Also, I don't really think your proposed scheme is good enough to have Satoshi's name associated with it. The innovation of Bitcoin was the coin melting pot thing, not the blockchain; Satoshi's original blockchain, which Bitcoin still mostly uses, was just a worse version of Git repositories.)
The margin for any given piece of content will
be low
Not when the financial incentive rises the rarer (less shared) a piece of content gets. This will be baked into the protocols. Arweave already has this.
I think you fundamentally misunderstand Bitcoin. People don't pay to store data on the blockchain; they pay to get their transactions mined. There's no guarantee that all the blocks of the Bitcoin blockchain will stick around forever, only that there'll be a consensus as to what the current block is, and what the states of all the wallets are (since that's the only information you need to validate a new block).
Keeping historical blocks around is not a property of Bitcoin. No up-front blockchain payment system can keep data around for any length of time. (There is no mechanism enforcing the storage of the data; such a mechanism cannot exist within a computer system.) “$10/GB for 200 years” are the words of a snake-oil salesperson – or, more charitably, somebody who hasn't properly read the marketing copy.
Spam cannot be dealt with. Illicit material sticking around is a feature. There is an economic incentive for material that creates a liability for the “seeders” to not be kept around.
To give it as much credit as I can: individual peers could do their own content moderation, deciding what they do and don't want to be around, and if enough people decide something shouldn't be there, it'll basically drop out of availability. But that goes against the whole “market forces” philosophy that it's supposedly based on, and also goes against some of the claims that the proponents of these systems keep making.
It's just a bad system, not even taking into account the pointless CPU cycle wastage. Blockchains sort of make sense for something where each block is based on the previous, but for arbitrary pieces of data? Chaining them like that is faddish unless it brings real concrete benefits over just doing the same thing without blockchain.
Mirror.xyz currently renders HTML from its Arweave backend, but not RSS, which is a bit disappointing, as there are a few Mirror blogs I'd love to have in my news reader. If it's capable of rendering HTML, RSS should be just a few lines of code away, but I can't find any repository for the frontend to try and implement it myself. More disappointing is the overall reactions to this comment including totally unhelpful comments that are basically just "OMG GTFO" and likely violating HN guidelines.
Thank you I hadn't looked into a replacement, just thought RSS feeds infringed on some techno-gatekeepers fiefdom and had been deemed too dangerous to live.
I'm still consuming plenty of stuff via RSS using the Mac app Vienna. I honestly don't spend as much time in Vienna as I did in Google Reader back in the day though - primarily because over the years more and more feeds went dead.
It seems like different niches treat RSS differently - many of the webcomics I've subscribed to still publish via RSS. Even if the actual comic image isn't included in the RSS, there's at least a feed entry with a link to go to the site to view the comic. But my "Cars" category in Vienna is basically empty - I guess the various automotive blogs and magazines I subscribed to were run by folks who didn't care about RSS as they migrated platforms, etc. Or maybe they actively shut their feeds off to try to drive traffic to their homepages.
I still use RSS. I use Feedly to curate various RSS feeds and many website still have them, or at least the tech focussed ones.
I used to be subscribed to more but some RSS feeds have such high traffic it's difficult to follow them, such as the BBC news feed.
Feedly has a nice new feature [0] that makes an RSS feed out of a standard website, so that might be worth considering if you have some news sources that don't publish a RSS feed. Unfortunately this wouldn't work with anything that requires authentication like private Twitter feeds for example.
When Google reader shut down, I moved to feedly for like a week. Then I started getting anxiety that feedly would shut down too. Because if google could kill off a product i relied on, anyone could. And that got me into self hosting and running my own services.
ttrss was basically my first self hosted thing almost a decade ago. It was great. And then it turned out the guy running it was a bit not pleasant, and I didn't want to support that. So I recently migrated to freshRSS. Both work great, are open source and self hostable.
And feedly hasn't shut down yet (probably just to spite me). So yeah, tons of options for RSS.
I'm aware. but also, they aren't the only company to shut down services. They do it more often and more flippantly than other companies, but other companies still do it from time to time.
My response to the situation was drastic overkill, but now I get to rely on myself and I have a hobby I enjoy, rather than having to worry about what other companies decide to do with my data.
Having self-hosted for a long time, I find it's getting easier in a lot of ways what with Docker and all.
I'm starting to wonder where "Sandstorm 2.0" is. Sandstorm, for those who don't know, was basically an attempt to make self hosting really viable, but was tragically ahead of its time because it predated Docker. So they burned tons of effort on sandboxing, and wrapping existing applications into their sandbox, and it was just too hard to port things into their world to get very many applications running.
It seems like a project that would do that in terms of docker-compose files could be created for much less effort, and maybe not quite all the pretty-shiny they had. But as I'm struggling right now a bit to bring up a Bitwarden server, there's still pain around setting up the forwards properly and getting the Let's Encrypt certificate. Something that managed all this better wouldn't be too hard, and could just be slammed up on a small AWS instance or something would be easy. (Branching out to other services over time or something.) Plus setting up proper backups would be nice. We're so, so much closer to being able to do this nowadays than we used to be... for instance, S3 has also become a near universal API, so backups using it have gotten to be easy but they can still be done without vendor lockin.
Then it would be really easy to self-host an RSS reader or something.
I'm hoping this will either prompt someone to consider this project, or prompt someone to tell me "It already exists, go here and here and run this docker command to install it."
I've taken the self hosting journey to learn tons about docker, and kubernetes. (Disclaimer: Kubernetes is overkill for self hosting). Not because it was the easiest, but because it was a great way to learn about a technology that has benefited me in my job.
Nowadays, for me at least, it IS really easy to self-host anything. Get the docker image, add a deployment to my k8s cluster. Ship it. Getting to this point was less simple.
I use Dokku for that (I can share my Bitwarden repo if you want, the entire thing is four lines or something). I also made https://gitlab.com/stavros/harbormaster for things that weren't so "web server -> app -> database" and love it.
I’m using NetNewsWire which works great but is iOS and Mac only. I’ve considered Feedly, but I would have to do the pro plan because of the cap on number of feeds. I haven’t switched because I have a hard time justifying the cost. I would rather that money go to the people creating the content.
I started using feedly again this week for monitoring some listservs / mailing list archives, and also some craigslist searches for things I want to buy, all in one place.
The sensible websites still offer it, and the ones that don't are a toxic dumpster fire anyway. Yes, I too remember the good old days when you could monitor facebook, twitter, etc all in an RSS reader, and then they stopped their RSS feeds. The thing to realize is that they also stopped being useful, user-oriented sites at the same time.
This topic comes up semi-frequently. I'm personally a fan of RSS but it certainly has issues:
(For the purposes of this comment I'm mostly referring to Atom/
1. RSS is, at the end of the day, a syndication format, _not_ a feedback or posting format.
2. RSS feeds have no real concept of portability between devices or user agents. Kludges like OPML can be used to aggregate feeds for sharing across user agents, but they still don't actually store device-specific metadata that a user would like.
3. If RSS feeds contain both the article and the content, then there's both little incentive for the user to actually visit the site and no analytics for the author to understand which articles are popular. When authors put summaries in their feeds with links to the original article, they degrade the experience for the reader.
Services like Wallabag which scrape and store article contents for later viewing seems, IMO, more Lin line with what users want while being fairly seamless for the author. There's also technologies like WebMentions for server owners to comment on other blogs or things like Usenet to freely discuss articles.
>If RSS feeds contain both the article and the content, then there's both little incentive for the user to actually visit the site and no analytics for the author to understand which articles are popular. When authors put summaries in their feeds with links to the original article, they degrade the experience for the reader.
Yeah, this is definitely a problem. FWIW, Ars Technica solved that by giving you full-text fees only if you pay for a subscription, which seems like a nice approach that more places could hypothetically use.
I've been working on a self-hosted (not federated) model for social media called Haven[1]. RSS (with HTTP basic auth) was the obvious answer for "how do I track all of my friends' different sites?" It has the advantage that building an RSS reader into the system means you can follow other Havens and other websites in one place.
It's interesting to me that you say "Not federated", because RSS has always seemed to me to be one of the best-adopted federated protocols, after SMTP and HTTP. Indeed, the first generation of federated social media protocols were originally based on souped-up RSS feeds (Atom) with an optional PubSub enhancement (Pubsubhubbub, later renamed to WebSub).
So when you say "not federated", what exactly do you mean? If you're publishing to your own site, and other people are reading it on their own compatible sites, isn't that fundamentally a federated model?
I'm trying to distinguish between Haven and Mastodon-type systems. "Federated" to me generally means you have many multi-user servers. With Haven, the server only hosts a single identity--yours, since you're the administrator/operator. I wouldn't describe a Wordpress blog with RSS as federated either.
Mastodon runs just fine in single user mode—in fact, it's was one of our most popular options back in the beginning. I personally don't think of single user and multi-user mode as particularly different, and neither is "core" to the functionality of Mastodon—the core functionality is the ability to interface with other users, whether they're hosted on your server or elsewhere. In that sense, whether a server is single- or multi-user is really isomorphic to the way I think about Mastodon's user experience, most of the time.
I use RSS daily with Miniflux. Tiny Tiny RSS is another popular self hosted option and there are a number of freemium platforms like Feedly & Inoreader. I prefer to self host primarily for cost/no ads or tracking/private data. It's true that social media platforms don't support it but that's never been my personal use case - every news site or discussion group I like to keep up with has a feed and it's worked great for me.
Some days ago there was a ShowHN (if I'm not misremembering) of a tool to generate RSS for sites that do not support them. I haven't tried it but maybe it can be useful to you
If you're familiar with CSS selectors, you could try this that I work on to generate a feed from the webcomic site: https://createfeed.fivefilters.org/
WebToons does have feeds, but they only show the first frame. Then you need to wait 10s for their shitty JS to load before it starts to download the comic.
I really want to make a scraper that will pull the full comic into the feed but don't want to play the cat-and-mouse game.
how do you have a RSS feed of HN? what links does it include out of the thousands that are posted here daily? also, can I ask you what service are you currently using for you RSS feed?
thanks.
There's probably more of them but if you look at the html (browsers used to have an icon for this...) there's a <link rel="alternate" type="application/rss+xml" title="RSS" href="rss"> tag in there on the home page, so https://news.ycombinator.com/rss should be the feed
Unfortunately, it only includes the title and the submission’s link, no content, which makes it a bit useless if your intension is to read the articles and not just the titles, which you can already do by visiting the home page.
Some people have created complementary RSS feeds like this → https://github.com/cixtor/rssfeed#readme which basically take the submission’s URL, download the web page, and removes the irrelevant HTML tags using Mozilla’s Readability.js library. Although, this project uses a Go (golang) port: https://github.com/go-shiori/go-readability#readme . It seems to work quite well, with minor bugs here and there due to inconsistencies of modern web development.
Me too where I use inoreader for my feeds. I really can’t imagine consuming content any other way. Agree when Google reader died then a lot of webpages cut off support.
I get the RSS feed of Hacker News via Feedbin, but only as a backup; I find it much more pleasant to log on to the website once or twice a day and browse.
Haha, I literally just set up Newsboat[1] and this is my second article I read from the comfort of my terminal :) So far Newsboat is great: Everything configurable with text, fast and effortless. The key-binds took a little bit, but now I can even do stuff with hjkl!
I will take the opportunity to ask about two issues I am currently facing.
- Is there a feed for Github discussions?
- Does anyone have some ideas/sources for bookmark scripts? I found [2], but I am not really sure what it does.
E: To add, in my opinion the killer feature of Newsboat is the fact that all links get appended to the article, similar to how I added the two links below my post. In Newsboat I can just press the corresponding number and open the link!
GitHub has per-repo feeds for commits and a private feed for all your events[1], but not for issues or discussions, apparently[2]. Of course, you can always roll your own using the API, a web server, and a cron job, even if it is a bit inelegant.
To be honest, I don't. Websites like Reddit or Hacker News are better because they provide popularity scores into the links, helping me filtered out junk links.
I thought like this until I started missing posts from blogs I liked. I remembered why RSS feeds are useful and got back into it after a friend recommended a reader he uses. I also picked up a good habit from him: subscribing to releases/commits of projects I want to stay up-to-date with.
Feedly and Innoreader are both good and use various apps and or extensions that hook into those two. It’s one of the best ways to quickly scan content from multiple upsteam sources and a decent reader makes for a pleasant reading experience too.
I have a private Discord server that acts as an RSS reader via MonitoRSS[1]. It works great because Discord has really good notification configuration. Some feeds I want to know about immediately, others I only want to see when I check the news, and because I can mute channels I can let my friends use it as well. They set up their own feeds and I don't have to be alerted to it but I can go look if I'm interested in what they're reading lately.
All that is to say that RSS still works. What's missing is the original content creators of the early RSS world. Nowadays, most people create small, easy-to-write, easy-to-consume content in one of the walled gardens since the notifications, interactions, and network are all included for them.
The author paints it as some kind of conspiracy, but the simple truth is that RSS waned in popularity once push notifications became popular and widespread. Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today (especially on mobile). Pub/sub is the clear way forward, but it's just sad that there is no accepted standard for it.
>The author paints it as some kind of conspiracy, but the simple truth is that RSS waned in popularity once push notifications became popular and widespread. Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today. Pub/sub is the clear way forward, but it's just sad that there is no accepted standard for it.
Polling isn't a requirement. If your reader is polling it's strictly because the author wanted to implement such a feature; that's it.
There's also ETag and last-modified which are used when checking and have very little overhead (most especially in the case when the content has not changed).
>Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today.
1) It's not that inefficient, especially if you recalculate your feed only when it changes and use HTTP statuses (particularly 304 Not Modified) correctly.
2) Most people then and now would go through an aggregator. So they'd really be talking to that instead of the source feed.
3) There were push notification standards out well over a decade ago, including RSSCloud and PubSubHubBub.
> 1) It's not that inefficient, especially if you recalculate your feed only when it changes and use HTTP statuses (particularly 304 Not Modified) correctly.
Besides, assuming you're not hideously overcomplicating things, you'll just be serving a static file off disk. If your server is a raspberry pi, you ought to still be able to serve some 1000 requests/second if that is what you are doing.
Assuming most RSS readers poll at most once every 15 minutes (which is a really high rate), you'll need at least a million subscribers before you need to get a second raspberry pi.
Also worth mentioning the obvious, feedly etc is updating all the users on their platrom with a single call to your site (every 7 minutes in your case). No doubt they scale the update frequency by how popular it is on their platform / frequently updated / however often the ai decides it is needed.
Should also depend on whether a push extension is used, then the reader can fetch the feed less often. The feed of that blog seems to be a plain RSS feed without that.
Everyone used to "ping" sites whose only purpose was to provide feeds of when blogs changed, so all the aggregators would use that as a starting point anyway. But really it isn't that necessary - there are few feeds where getting things to your subscribers that urgently matters enough that you necessarily want them to hammer your site the moment you publish.
I'm not aware of any push functionality in RSS -- something needs to poll the feed, no? Or is there an RSS extension that uses long polling or something?
PubSubHubbub/WebSub (the latter is effectively the slightly modernized W3C standardization of the former) is the standard solution for that. (Feed declares where to register for updates (hubs), readers register their callback URLs with hubs, publishers tell hubs when something changed, hubs run through the registered callbacks and ping them with the update. Thus needs a server-component for the reader side.)
Yeah, I ran a niche aggregator for a few years and we polled ca. 3m feeds a day. Every time a feed had an update since the last time we'd polled it we'd subtract a percentage of the polling interval with a floor of about 5m. Every time there wasn't an update we'd reduce the rate until a ceiling of about 24h. Very simple and no doubt you could do better but better wasn't necessary.
That's not true, but might be a common misconception. RSS rode the wave of the realtime hype at that time and has multiple standards for push notifications. RSSCloud und Pubsubhubbub are the two immediately in my mind, note that PuSH got turned into WebSub.
Wordpress does support both. You can make a blog on wordpress.com right now and you get both standards activated by default in the RSS feed. That takes care of half the web ;) (well, it would if it's also true for standalone Wordpress installations, which I don't know). And all relevant readers support at least PuSH. This is really not a standard problem, and I doubt the concept itself is part of the adoption problem.
No, not having a way to subscribe when stumbling over an RSS feed was the main problem in my eyes, and the existence of a feed not being highlighted in browsers anymore. Firefox dropping support was not only an act of treason against the free web, it was also very effective in making it very hard for new users to understand how to use RSS. Not that chrome is any better. Here lies the problem, and that's where it could be solved.
Add into browsers an icon when a site has an RSS feed, and let that point to a foundation managed site that points to RSS readers, where users can select their favorite. I'm 100% certain that would have a huge effect on adoption numbers.
As this comment thread shows very well RSS is not dead at all, but it's not in the mass market right now and that could be changed easily enough.
I think it was the sunsetting of Google Reader that triggered the downfall. I don’t think I ever cared about push notifications for feeds. When you have your morning coffee, open up Reeder, refresh the feeds and read through a few articles.
I don't think it was even just a technical issue. Linear feeds of all content just died out to be replaced with sites like Hacker News / Reddit where you have ranked ordering and comments.
As much as HN hates the out of order feeds, that's what the average user wants because most content is boring.
I'd assume that somebody who misses RSS will also be happy with pulling only few times a day and not every few minutes. That's why push became so popular but with a different subset of users. Everybody goes through a push-positive phase but some (like me) come to a conclusion that this is unhealthy and distracting in the long run - hence the fond memories of technology like RSS.
Switching back to Linux after ten years on Mac, I had completely forgotten how much I love it when my computer does _not_ notify me about anything. I have no notification system installed. It’s been a few months and I haven’t missed anything, but I feel more relaxed and focused while using the computer. Notification are mind pollution.
I like notifications on linux. It allows me to know when a livestream starts or when a person I like tweets immediately instead of me only realizing hours later.
Haha, I suppose this illustrates how well Linux can work for every different preference, but both of those sound like exactly what I don’t want at all. Being called to pay attention to something, immediately, because some other person did something, not directly interacting with me, is the antithesis of how I want to spend my time.
> Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient
I think you're greatly overestimating this inefficiency. Especially for websites properly implementing the HTTP protocol. 'Push' requires a lot more complexity and it's also outside of the user's control. For fuck's sake, you can just serve a static file. We've been doing this for decades.
Talking about control, my hypothesis is that it was the main driver of the downfall of RSS. It started when Google killed Reader and Facebook took control of what you see in the timeline. Users picking and choosing what/when to read is not compatible with the way the major gatekeepers of information drive engagement.
FWIW my RSS reader is set up to only refresh manually when I open it and pull to refresh, and I like it that way. It's a much nicer way of reading than having everything yelling for my attention all the time.
Sites like Movim (https://movim.eu/) offer an user friendly front-end for this protocol (building upon the ATOM format). You can login using any XMPP account and also self-host Movim instances.
Push notifications have a cost too, now you need the server to maintain state of all clients and notify them on changes. This has huge implications at scale, especially if it's extremely cheap or easy for new clients to spin up. Think about every single browser tab now requiring a server to maintain some state on the backend--that's a nightmare with billions and billions of users and tabs. Pull-based architectures have the nice effect that less popular content has almost no cost to continue serving and just falls into the background forever.
Push notifications are a replacement for RSS in the same way that cars are a replacement for public transit.
This is a nonsense comparison. RSS is about presentation, not about real time interaction. And it would not be hard to have RSS feeds support push notifications, or other modernization.
RSS is a universal and accessible way of presenting content, that is what is so inconvenient about it to publishers.
The business model of most platforms today has RSS as competition. It completely undermines analytics, and their ability to fine-tune presentation, for the benefit of the end-user.
So of course GM will say that their gas guzzlers make electric trains obsolete.
I don't agree with your premise or conclusion. Its far easier for a website to just publish a dead XML file that any client can read at their own pace, as frequently as they would like. Also, anyone can create a pub/sub 'middle man' that can implement a push system for clients who want that. You can have the best of both.
I've noticed a healthy amount of websites with pretty granular RSS, for example the auto parts outlet rockauto.com has RSS feed for every make, model, and production year that they service.
I'd wager this saves them some unpaid overtime. It seems that a service which provides correct, useful RSS nearly obviates the entire hassle of dealing with bot scrapers and the resource drain and health insurance premiums that come along. You can easily serve any volume of RSS with cloudflare or nginx.
It is quite true that mobile clients scraping a url is inefficient vs pub/sub. Maybe you could just uses mqtt to trigger a GET or something.
I spent about five minutes considering what a world with facebook, instagram, ebay, amazon RSS feeds might look like. The experience felt like goat staring.
I was part of a startup that aggregated items for sale years and years ago (we were sold to LookSmart for a pittance in the end) and we did RSS feeds for everything. Every page spat out a list of items, in fact and the website and the RSS and atom feeds were all generated from the same middle layer. A huge part of our inventory came from RSS feeds too.
I still use RSS on a daily basis, and it's great. I like Newsblur.com but there are a fair number of options based on your desire to self-host and/or use social features.
The primary thing I wish was better is a way to handle social commentary. Unfortunately this is an extremely hard problem to solve without allowing spamming and brigading.
312 comments
[ 4.1 ms ] story [ 313 ms ] threadBut the masses browse a locked down web. That is true.
We might have another shot at an open web for the masses via the whole crypto movement.
If content goes on decentralized blockchains in the future, nobody can lock the content down like big tech did for Twitter, Facebook, Instagram, TikTok etc.
Projects like Arweave show how it is possible to store huge amounts of data on a blockchain without using much resources.
If we get there, it will be trivial to build blockchain-to-rss bridges. I am ready to build some to get my favorite content into my RSS reader.
My feeling is that there might be a way towards a decentralized web via financial incentives. In a decentralized web, crypto currencies can float around freely. Making it easy for the actors to earn. Old platforms will probably be slow to open up to crypto currencies.
https://hnrss.github.io
Consider a “decentralised” blockchain system that does the work of The Pirate Bay – providing hashes of particular files, along with their names, so that people can select and request data from the P2P network.
How, exactly, does this system help with RSS in any way?
Additionally, how does this system help with the “censorship” issues that The Pirate Bay has? Even when The Pirate Bay remains up, that doesn't mean anybody's seeding.
And a blockchain-based system has to have this limitation. If all the data is directly on the blockchain, nobody can actually download it all, so the blockchain becomes unverifiable and ceases to have any cryptographic properties at all. (In short, there's no benefit to a blockchain at all.)
Well, yeah – but we already have that. It's called the Fediverse. No blockchain in sight. (In fact, it predates Bitcoin.)
> That is where the financial incentives come into play. Seeding will be economically incentivised.
The margin for any given piece of content will be low, so there's an economic incentive to honour takedown requests, no matter how fraudulent or unenforceable, if the effort to check whether they're valid is lower than the expected average reward minus the expected average legal cost.
> We have approaches now where participants only need to download arbitrarely small parts of the chain. Yet the network as a whole keeps the chain intact.
We had those back in Satoshi's time, too. They work by having n centralised trusted nodes, then asking all your peers to double-check what you got from those trusted nodes. That approach is, surprisingly enough, also vulnerable to censorship!
(Also, I don't really think your proposed scheme is good enough to have Satoshi's name associated with it. The innovation of Bitcoin was the coin melting pot thing, not the blockchain; Satoshi's original blockchain, which Bitcoin still mostly uses, was just a worse version of Git repositories.)
Same with the blockchains that support larger amounts of data. Whoever wants to store data pays for it.
The specifics of how the storage of rare content is incentivized depends on the protocol.
On Filecoin the publisher pays an ongoing fee.
On Arweave it is paid upfront. The current estimate is that $10 will store a GB for 200 years.
So doing a tweet on an Arweave powered social network would cost you something like $0.001 if you want your tweet to survive for 200 years.
Keeping historical blocks around is not a property of Bitcoin. No up-front blockchain payment system can keep data around for any length of time. (There is no mechanism enforcing the storage of the data; such a mechanism cannot exist within a computer system.) “$10/GB for 200 years” are the words of a snake-oil salesperson – or, more charitably, somebody who hasn't properly read the marketing copy.
You are correct that in Bitcoin there is less incentive to store old blocks. But in chains crafted to store data for a long time there is.
I am stopping this discussion here because you are too negative (almost aggressive) for my liking.
But it will be an interesting thread to read again in 20 years when all this has played out.
To give it as much credit as I can: individual peers could do their own content moderation, deciding what they do and don't want to be around, and if enough people decide something shouldn't be there, it'll basically drop out of availability. But that goes against the whole “market forces” philosophy that it's supposedly based on, and also goes against some of the claims that the proponents of these systems keep making.
It's just a bad system, not even taking into account the pointless CPU cycle wastage. Blockchains sort of make sense for something where each block is based on the previous, but for arbitrary pieces of data? Chaining them like that is faddish unless it brings real concrete benefits over just doing the same thing without blockchain.
https://nodetics.com/feedbro/
https://github.com/nt1m/livemarks/
It seems like different niches treat RSS differently - many of the webcomics I've subscribed to still publish via RSS. Even if the actual comic image isn't included in the RSS, there's at least a feed entry with a link to go to the site to view the comic. But my "Cars" category in Vienna is basically empty - I guess the various automotive blogs and magazines I subscribed to were run by folks who didn't care about RSS as they migrated platforms, etc. Or maybe they actively shut their feeds off to try to drive traffic to their homepages.
I used to be subscribed to more but some RSS feeds have such high traffic it's difficult to follow them, such as the BBC news feed.
Feedly has a nice new feature [0] that makes an RSS feed out of a standard website, so that might be worth considering if you have some news sources that don't publish a RSS feed. Unfortunately this wouldn't work with anything that requires authentication like private Twitter feeds for example.
[0] https://blog.feedly.com/easily-follow-websites-that-dont-hav...
ttrss was basically my first self hosted thing almost a decade ago. It was great. And then it turned out the guy running it was a bit not pleasant, and I didn't want to support that. So I recently migrated to freshRSS. Both work great, are open source and self hostable.
And feedly hasn't shut down yet (probably just to spite me). So yeah, tons of options for RSS.
I wouldn't use them as a standard for keeping services up.
I'm starting to wonder where "Sandstorm 2.0" is. Sandstorm, for those who don't know, was basically an attempt to make self hosting really viable, but was tragically ahead of its time because it predated Docker. So they burned tons of effort on sandboxing, and wrapping existing applications into their sandbox, and it was just too hard to port things into their world to get very many applications running.
It seems like a project that would do that in terms of docker-compose files could be created for much less effort, and maybe not quite all the pretty-shiny they had. But as I'm struggling right now a bit to bring up a Bitwarden server, there's still pain around setting up the forwards properly and getting the Let's Encrypt certificate. Something that managed all this better wouldn't be too hard, and could just be slammed up on a small AWS instance or something would be easy. (Branching out to other services over time or something.) Plus setting up proper backups would be nice. We're so, so much closer to being able to do this nowadays than we used to be... for instance, S3 has also become a near universal API, so backups using it have gotten to be easy but they can still be done without vendor lockin.
Then it would be really easy to self-host an RSS reader or something.
I'm hoping this will either prompt someone to consider this project, or prompt someone to tell me "It already exists, go here and here and run this docker command to install it."
I think that's the most similar think you will find to a Sandstorm 2.0 running on docker
Nowadays, for me at least, it IS really easy to self-host anything. Get the docker image, add a deployment to my k8s cluster. Ship it. Getting to this point was less simple.
In terms of automating things like handling certs for you, I'd say Caddy or even my own boringproxy.
I'm unaware of other ways to get consistent listings of new content at various sites in a machine readable format.
The sensible websites still offer it, and the ones that don't are a toxic dumpster fire anyway. Yes, I too remember the good old days when you could monitor facebook, twitter, etc all in an RSS reader, and then they stopped their RSS feeds. The thing to realize is that they also stopped being useful, user-oriented sites at the same time.
(For the purposes of this comment I'm mostly referring to Atom/
1. RSS is, at the end of the day, a syndication format, _not_ a feedback or posting format.
2. RSS feeds have no real concept of portability between devices or user agents. Kludges like OPML can be used to aggregate feeds for sharing across user agents, but they still don't actually store device-specific metadata that a user would like.
3. If RSS feeds contain both the article and the content, then there's both little incentive for the user to actually visit the site and no analytics for the author to understand which articles are popular. When authors put summaries in their feeds with links to the original article, they degrade the experience for the reader.
Services like Wallabag which scrape and store article contents for later viewing seems, IMO, more Lin line with what users want while being fairly seamless for the author. There's also technologies like WebMentions for server owners to comment on other blogs or things like Usenet to freely discuss articles.
You can encode html in feeds. You just have to mark them as cdata.
Yeah, this is definitely a problem. FWIW, Ars Technica solved that by giving you full-text fees only if you pay for a subscription, which seems like a nice approach that more places could hypothetically use.
Anyway, the people reading your RSS will load every article. They aren't the ones setting the difference in popularity between the articles.
[1]: https://havenweb.org/
So when you say "not federated", what exactly do you mean? If you're publishing to your own site, and other people are reading it on their own compatible sites, isn't that fundamentally a federated model?
These days most things are abstracted away by a web interface or app hosted in the cloud.
Sure this makes a nicer UX, but it has the side-effect of creating walled gardens.
There is one webcomic I enjoy, but can't get an RSS feed, because it's on a service similar to WebToons.
Edit: it wasn't a showHN: https://news.ycombinator.com/item?id=29772540
I mostly follow Dilbert and Questionable Content. I used to follow Wizard of Id and a few more, but they also lost support for RSS.
I really want to make a scraper that will pull the full comic into the feed but don't want to play the cat-and-mouse game.
<shrug>
Yes, google basically killed RSS through embrace, extend, extinguish.
There’s no money in democratizing self-publishing.
[1]: https://hnrss.github.io/
[2]: https://hnrss.org/frontpage
Unfortunately, it only includes the title and the submission’s link, no content, which makes it a bit useless if your intension is to read the articles and not just the titles, which you can already do by visiting the home page.
Some people have created complementary RSS feeds like this → https://github.com/cixtor/rssfeed#readme which basically take the submission’s URL, download the web page, and removes the irrelevant HTML tags using Mozilla’s Readability.js library. Although, this project uses a Go (golang) port: https://github.com/go-shiori/go-readability#readme . It seems to work quite well, with minor bugs here and there due to inconsistencies of modern web development.
I will take the opportunity to ask about two issues I am currently facing.
- Is there a feed for Github discussions?
- Does anyone have some ideas/sources for bookmark scripts? I found [2], but I am not really sure what it does.
[1]:https://newsboat.org/
[2]:https://github.com/gpakosz/.newsboat/blob/master/bookmark.sh
---
E: To add, in my opinion the killer feature of Newsboat is the fact that all links get appended to the article, similar to how I added the two links below my post. In Newsboat I can just press the corresponding number and open the link!
[1] https://webapps.stackexchange.com/q/20535
[2] https://github.com/github/feedback/discussions/31
Which you can also do—and better—with RSS.
hnrss.org allows you to get a feed with posts over N points. Example: https://hnrss.org/newest?points=300
Adding `top.rss` to a subreddit’s URL gives you a feed you can further manipulate with `t` and `limit`. Example: https://www.reddit.com/r/wallpapers/top.rss?t=month&limit=20 lists the top 20 posts from the past month in `r/wallpapers`.
Feedly and Innoreader are both good and use various apps and or extensions that hook into those two. It’s one of the best ways to quickly scan content from multiple upsteam sources and a decent reader makes for a pleasant reading experience too.
All that is to say that RSS still works. What's missing is the original content creators of the early RSS world. Nowadays, most people create small, easy-to-write, easy-to-consume content in one of the walled gardens since the notifications, interactions, and network are all included for them.
[1]: https://monitorss.xyz/
Polling isn't a requirement. If your reader is polling it's strictly because the author wanted to implement such a feature; that's it.
There's also ETag and last-modified which are used when checking and have very little overhead (most especially in the case when the content has not changed).
1) It's not that inefficient, especially if you recalculate your feed only when it changes and use HTTP statuses (particularly 304 Not Modified) correctly.
2) Most people then and now would go through an aggregator. So they'd really be talking to that instead of the source feed.
3) There were push notification standards out well over a decade ago, including RSSCloud and PubSubHubBub.
Besides, assuming you're not hideously overcomplicating things, you'll just be serving a static file off disk. If your server is a raspberry pi, you ought to still be able to serve some 1000 requests/second if that is what you are doing.
Assuming most RSS readers poll at most once every 15 minutes (which is a really high rate), you'll need at least a million subscribers before you need to get a second raspberry pi.
No, not having a way to subscribe when stumbling over an RSS feed was the main problem in my eyes, and the existence of a feed not being highlighted in browsers anymore. Firefox dropping support was not only an act of treason against the free web, it was also very effective in making it very hard for new users to understand how to use RSS. Not that chrome is any better. Here lies the problem, and that's where it could be solved.
Add into browsers an icon when a site has an RSS feed, and let that point to a foundation managed site that points to RSS readers, where users can select their favorite. I'm 100% certain that would have a huge effect on adoption numbers.
As this comment thread shows very well RSS is not dead at all, but it's not in the mass market right now and that could be changed easily enough.
As much as HN hates the out of order feeds, that's what the average user wants because most content is boring.
I think you're greatly overestimating this inefficiency. Especially for websites properly implementing the HTTP protocol. 'Push' requires a lot more complexity and it's also outside of the user's control. For fuck's sake, you can just serve a static file. We've been doing this for decades.
Talking about control, my hypothesis is that it was the main driver of the downfall of RSS. It started when Google killed Reader and Facebook took control of what you see in the timeline. Users picking and choosing what/when to read is not compatible with the way the major gatekeepers of information drive engagement.
Sites like Movim (https://movim.eu/) offer an user friendly front-end for this protocol (building upon the ATOM format). You can login using any XMPP account and also self-host Movim instances.
This is a nonsense comparison. RSS is about presentation, not about real time interaction. And it would not be hard to have RSS feeds support push notifications, or other modernization.
RSS is a universal and accessible way of presenting content, that is what is so inconvenient about it to publishers.
The business model of most platforms today has RSS as competition. It completely undermines analytics, and their ability to fine-tune presentation, for the benefit of the end-user.
So of course GM will say that their gas guzzlers make electric trains obsolete.
I'd wager this saves them some unpaid overtime. It seems that a service which provides correct, useful RSS nearly obviates the entire hassle of dealing with bot scrapers and the resource drain and health insurance premiums that come along. You can easily serve any volume of RSS with cloudflare or nginx.
It is quite true that mobile clients scraping a url is inefficient vs pub/sub. Maybe you could just uses mqtt to trigger a GET or something.
I spent about five minutes considering what a world with facebook, instagram, ebay, amazon RSS feeds might look like. The experience felt like goat staring.
The primary thing I wish was better is a way to handle social commentary. Unfortunately this is an extremely hard problem to solve without allowing spamming and brigading.