126 comments

[ 3.7 ms ] story [ 183 ms ] thread
> stop dragging books into libraries that are known to be burned down in a couple of years for sure.

Spot on. We should be encouraging users to redecentralize and use distributed open systems. Not walled gardens.

As much as I’m with you in spirit, guaranteeing long-term data availability in a distributed system that would replace Reddit is a hard one.

Users are not enough, we need concerted engineering efforts.

this! Having a distributed architecture or business model does almost nothing in guaranteeing the sustainability of your content (and let's be honest, how often do we go back and look at something we wrote 3+ years ago?)

it takes engineering effort, time, money, and reason to keep any system running for a prolonged period of time, centralized or other.

Call me a hoarder, but personally I'd like to see some of those old posts just out of interest.

My posts on every platform over my whole lifetime can't be more than what can fit on a flashdrive. It shouldn't be an insurmountable goal to store them, it's irritating that it's so difficult to get data out of all these fragmented systems.

Well, post directly to archive.org.

I have seen so many large and important web sites from early 2000s to have been gone by now, to say nothing of personal blogs.

Archive can also be gone... we need a decentralized database that is "append only and that is resistant to efforts to delete it by any entity
Agreed, though I would also add "does not require electricity proportional to the nominal stored value" to the requirements. As of 2019, bitcoin alone is using more electricity than Switzerland[0], which is unconscionably high.

[0] https://www.bbc.com/news/technology-48853230

On the other hand, this website, "karl-voit.at," isn't loading right now.
Walled gardens like Hacker News? ;)
(comment deleted)
I agree. But how would you do that?

Facebook, Reddit, Imgur, Youtube, HN, SO, and others are where all the people go because it's easy and, well, all the people are there.

Those are your potential readers and customers. So you go there, as well.

I remember having to visit multiple forums and buying stuff directly on small websites. I needed to have credentials for all of them, but that was the hardest part, really. I liked the experience more.

Reddit et all are the same, they're technically a collection of smaller forums, with the important difference being that they're controlled by a single, big company.

I think it's a dangerous thing for the future of the Internet and information in general. As Google and Facebook demonstrate, one ban and you lose potentially everything digital that you "own" (don't get me started on that).

Anyone would be insane to take that risk, but most people don't even realize it exists.

People don't really contribute, they share content and opinions, they discuss and interact.

The "contribution" is retrospective, it is only a thing that exist when you look back.

The centralisation works because brings people together. Even with torrents and bitcoin, the two decentralised networks that actually work, are valuable only when there's The Pirate Bay and Binance. Without the centralisation tools, these are simply cool technologies in the lab.

Then prior to that you'll have to encourage users to subscribe to non-cable internet connections that have some amount of upstream bandwidth so that they can publish via machines they control.
tldr: usenet???
Wikipedia, OpenStreetMap, the Internet Archive, Mastadon/ActivityPub, Peertube, git and company, and something like but more open than StackOverflow (which feels like it should be a part of Wikipedia in some way, shape, or form). NNTP is suggested, but feels like it's been superseded by superior mechanisms (although there's much work to be done as part of the Distributed Web movement [1] [2] [3], which would also be applicable).

TLDR Open abstractions on top of open data trivially and rapidly distributed.

[1] https://hacks.mozilla.org/2018/07/introducing-the-d-web/

[2] https://blog.archive.org/2018/07/21/decentralized-web-faq/

[3] https://github.com/gdamdam/awesome-decentralized-web

> which feels like it should be a part of Wikipedia in some way, shape, or form).

Maybe a separate project of the Wikimedia foundation, sure.

I keep dreaming of someday seeing a resurrected Usenet, but I know it will never be anything but a dream.
When humans finally make settlements on Mars and venture into space, decentralized web will be resurrected, as anything else would be utterly unusable with the interplanetary latencies.
> I keep dreaming of someday seeing a resurrected Usenet, but I know it will never be anything but a dream.

Maybe not. You're definitely not the only one who wants something like that.

There are definitely still some active text groups on usenet. Either people that never left or folks returning/visiting for the first time, the potential is there.

Years ago some folks broke off from Slashdot and started posting to comp.misc. They're still going today

There seem to be quite some people in here feeling the same about USENET.

I had a quick look around, and the tools aren't trivial to set up. If someone who has experience with NNTP servers could bundle things as nice Debian packages with easy setup then that may help spread it again.

Spam can be reduced by using moderation, I wonder if there are new ideas around for countering spam in non-moderated groups?

Turns out old Usenet content is particularly hard to find nowadays...
Write a blogpost and share a link.
That's what I do. I come up with the solution, I deserve the credit. Fair, right?
Does... uh... does HN count?

edit Upon reading the content, yes it does. Please ignore this thread.

(comment deleted)
I'm surprised that Stack Overflow/Stack Exchange wasn't in the list of platforms; their policies against eg. duplicate questions are frustrating to users expecting a forum but are explicitly designed to generate content that's useful in the long term. I have no idea how many man-hours (man-centuries?) of effort have been spent by programmers generating post-hoc documentation within the SO platform.

It's unquestionably useful today, it's somewhat distributed by mirrors and the database export, it has an API, but there's no question that if SO folded tomorrow that a lot of utility and a lot of effort would be lost. However, I am confident that if SO did not exist and those contributors did not become addicted to the dopamine boost from "+10 Karma!" notifications that they would not have instead created permanent, Google-able, individual blogs with an equivalent amount of utility as what SO provides.

The criticisms are valid, but this article does not address the issue of audience and community, which is everything.

I post on Reddit and HN rather than a private blog because that is where my thoughts will be seen and responded to.

I cannot for the life of me get that link to open. Here’s a web archive version for those with the same issue: https://web.archive.org/web/20201029194008/https://karl-voit...

edit: fwiw the author has web archive links at the bottom of all their posts. Since the page isn't loading, I couldn't see it.

i guess it's getting a lot of load, it took a very long time for me to load up, like run a terraform init, plan and apply long ...
Probably wouldn’t have crashed if it was contributed to a web forum instead of a personal site...
The irony here is off the charts, of course.

A post complaining about how you shouldn't rely on major sites because their content doesn't get backed up... is failing to load... and being saved by the same centralized, private third-party backup service that also backs up major sites.

You can't make this stuff up. ;)

> Typically, those services don't provide any possibility to extract or synchronize content. They don't offer open APIs that allow users to choose among different and open user interfaces.

None of this is true. All these services have pretty fully-featured APIs and several independent clients.

Even otherwise, the fact that some of them may not exist at some point in the future is a terrible reason to not use them at all. I still find tons of relevant content in hosted forums from the 90s and 00s, and more recent ones like Stack Overflow and Github.

I'm pretty confident that most of the services he is complaining about will last longer than his own blog post.

Not to mention Reddit doesn't crash when HN links to it, whereas it seems like his blog does.
Meta point: Oh so that's what happened. It occurs so frequently that I'm sworried about the lack of robustness against this phenomenon.
That’s easily fixed though, e.g. by using Cloudfront or something similar.
If it is that easy to fix, then why does it happen over and over and over with posts on HN? There is obviously enough of a barrier (either in time or cost) that keeps everyone from doing it.
Because it's easy to fix, but the effort is not _zero_

Also, people may not realize this is a problem until it actually becomes a problem. In the mean time they will be happily rendering stuff on every page load, after fetching (static!) data from mysql on every single request.

It also takes time and money to test.
> I'm pretty confident that most of the services he is complaining about will last longer than his own blog post.

Ironically, I can’t actually read this blog post because it has become unavailable. I suspect that undermines his entire point.

> Ironically, I can’t actually read this blog post because it has become unavailable. I suspect that undermines his entire point.

I know people are really enjoying dunking on this guy for his web service outage, but I really don't think this is the right takeaway.

On this topic specifically he notes:

> All major commercial services such as reddit, Facebook and so forth keep everything a secret that is not ultimately necessary to use their services. Their software is a secret, they don't offer open APIs or only very crippled ones, you don't have the possibility to get to the raw data. So no luck there. You do have a lock-in situation.

This is an issue, not of availability, but of control.

Yeah, okay, his blog might be experiencing an outage right now. But he can move it. He can shift the content to another medium. He can back it up and retain ownership over it. As he very rightly points out:

> without very good support for data export, service duplication, open standards, any content you provide in closed web-based services will be lost just as MySpace already lost twelve years of content just so, just to mention one big example.

But, hey, let's ignore that and joke about his blog being down. After all, who doesn't enjoy a little schadenfreude from time to time, right?

I’ve seen errors on multiple occasions on hn, reddit, twitter, ...

I suspect that undermines your entire point.

> Even otherwise, the fact that some of them may not exist at some point in the future is a terrible reason to not use them at all ... I'm pretty confident that most of the services he is complaining about will last longer than his own blog post.

I can't say I share your confidence.

Take Google Groups as an example.

Were it not for archival work being done by various volunteers, the content on Usenet was at real threat of vanishing from the internet.

This is hardly unique. We've seen the same thing happen with Geocities and Myspace, for example. Another poster also mentioned G+, which is an excellent recent case.

Is this a reason to not use these platforms at all? I don't think so.

But it is a reason to retain your own copies of the things you care about, hence why, for my own blog, I follow the POSSE (Publish Own Site, Syndicate Elsewhere) pattern:

https://indieweb.org/POSSE

> Take Google Groups as an example.

And speaking of that, because of purported spam problems a number of Usenet groups (and especially their archives) currently aren't available at all via Google Groups.

Solving the groups problem would be a game changer. I can't recreate the same kind of group value that Facebook Groups provides or Google Groups provided anywhere else.

Maybe it's impossible, but hopefully some day we can get to a point where social media silos are not necessary, with easy to set up and use personal sites/apps that use Indieweb principles. The Indieweb movement is excellent for this but needs a lot more developer contributions to make this dent a reality.

> I'm pretty confident that most of the services he is complaining about will last longer than his own blog post.

And yet years of fantastic RPG community contributions on g+ are absolutely fucking gone. At their best they got taken out into barely-accessible formats, so they technically "exist", even if you can't find any of their content anymore.

> fully-featured APIs and several independent clients

As platforms mature, their APIs become more and more restrictive. E.g. Twitter's current API allows only a fraction of what it used to allow in its early days.

For me the point is that if you use an external platform then you no longer have control over your content; I think it still stands.

Stack Overflow is a nice counter example. Then again, all works fine as long as the goals of content authors and the platform are aligned.

> that some of them may not exist at some point in the future is a terrible reason to not use them at all

the reason is that the longer I use them the more I pump data into it and the more I am getting suckered for when they shut them down. this is exactly the justification for why standards should be open.

On the other side of this point: getting paid for work is a lot better than not getting paid to work.
Actually there is stuff I posted to Reddit just a couple of years ago that neither I, search engines, or any of the clients can even find anymore. Same for a couple of public Slack groups. I consider these services as wholly ephemeral.

Same goes for any product from Google, since they are notorious for service termination. In doing so, they abandoned their core promise of indexing the world's information. G+ is gone, and the Groups archive is ever-thinning: Usenet posts I made a couple of decades back were once accessible thus, since Google bought the Dejanews archives, but are no longer discoverable.

Fortunately all my most interesting snippets and contributions these days start life in my own private repositories, so I have my own record. Everyone should do the same.

Reddit's automatic locking of threads after a few months is extremely annoying.
Sure, it's like storing data in an undocumented proprietary format.

I think I've contributed one ‘significant’ fact on Reddit and one on HN.

Why should I trust that my own private blog would outlast commercial platforms?
It is not the web's fault, and let's not throw out the baby out with the bathwater. There are Web forums out there which are archivable, backupable, and distributable. And many pages can be saved locally. There is also Internet Archive, archive.today, and much other infrastructure to address distribution, backup, and all-around resilience.

All the big platforms suffer from not giving you much choice to export your own writing once you've sent it. I think it is as much for reasons of attempted lock-in as it is due to incompetence and scaling pains.

Facebook, Twitter, Reddit, and other platforms will all present a challenge when you try to "find that thing I wrote two weeks ago".

The only solution I can see is to build services which allow content export in usable format and to continuously audit these backups yourself.

I think it is a solution which only seems difficult at first, but is possible already with tooling available today, and can be made accessible to a non-programmer with some effort.

For my own project, I consciously chose .txt as the base format for both text and metadata, and have tried to make even the metadata as readable as possible.

I regularly do a clone test of my blog and message board, meaning I download the data export, unpack it on another (disconnected) instance, and ensure all content has made it across. (Thanks to using PKI for accounts, full continuity and sync can be achieved across multiple instances.)

I think the key to maintaining a trustworthy community is keeping the size small, to where all the activity can be monitored and reviewed by a human. Once you're facing a firehose, you're at the mercy of all the forces out there.

At least Reddit is usually up so you can read its content. (The site hosting this post is currently down.)
Reddit is down fairly often, to be honest. It does the thing where it pretends like it's up by showing you its front page, but if you click on anything it'll error out on you.
Yeah I've been using reddit since 2008 and daily downtime was not uncommon at some points.
That one's on the author, not the medium.
OP's point is that there are extra complexities which can lead to you shooting yourself in the foot in an attempt at further visibility. These trade-offs must be considered and taken seriously, and now the author will have to reevaluate his position. Maybe we'll get a post-mortem reflection from them.
I think the article misses the point of forums.

The point is to communicate with other people here and now, to help each other on short notice, to spread knowledge which the readers can apply immediately.

It's like mistaking talking for books. Talking is fleeting, but also indispensable for doing anything of consequence (including writing a book).

FWIW this seems to be the general challenge of Stack Overflow and Quora which are stuck halfway between "ask questions and get answers live-ish" and "come and look up information that was discussed in the past"

I think the curation piece that separates chatter now vs. published organized information is where forums lie in the middle of

Yes. So much of the Internet is driven by "Everyone should do what I do in this exact, specific situation which may or may not apply to your life ever."
Nah that’s more like messaging platforms. IRC, Telegram, matrix, etc.

I was thinking about this yesterday actually. I’ve been having a lot of questions about this new linux distro I’ve been using, and piling them on in the IRC channel, but if no one answers it right away it gets buried by the next message after me. So I kept re-asking it for multiple days before realizing, you know what if this is something that only even a few people might know how to help me with, I should probably leave it to “steep” on a proper Discourse topic, or reddit thread, or stackoverflow question or whatever. I hadn’t done that because I was just super impatient and knew it might be a good half a day or 2 or 3 days before I get a reply, but it’s much more likely to get one at all than in the IRC, where someone butting in with a meme or something right after me means my question might as well be nonexistent in the conversation stream.

Im working on a new topical discussion site just for cases like this, where each post has an irc-like chat built in, is open, and can be shared by url. https://sqwok.im
Forums are still closer to talking than they are to a book.

It's just that the internet allows them to be async conversations. A forum is fairly equivalent to email, just usually more public than email and the underlying infrastructure isn't federated.

Agreed. But like email, I expect forum threads/posts to stay indefinitely and be searchable, until I explicitly delete them.

Email is a persistent conversation which gets archived indefinitely, not an ephemeral one like talking with our mouths.

Your personal email may be retained indefinitely.

Corporate email (and IM, etc) gets purged pretty quickly, within 2-3 years, unless you save a particular message / thread explicitly.

Aren't they worried about losing most of the knowledge of the context of decisions made?
> The point is to communicate with other people here and now

Forums are -- or used to be, anyway -- a hybrid form of communication: it's both immediate but also useful for achival purposes. Many forums will tell you to read a specific thread when you want info about something in particular. In some forums, users use specific threads for archival/progress of a project.

Chat is ephemeral like you describe, but forums aren't necessarily.

I can’t count how many times forums saved me with obscure bugs/ failures. “Nvidia driver crashes when playing x at x resolution” good luck with tech support if they can’t replicate the failure.
The first sentence is

>If you're, for example, contributing to a reddit thread about something which is irrelevant or anything with only a short-term relevance, this article does not apply to you right now.

Forums often end up with some posts that have a more long term benefit, and those should be hosted somewhere else.

(comment deleted)
Yeah a paragraph or two with critical information is mere bytes and therefore readily accessible by even the slowest internet connection and processing device - picture & audio are more expensive to serve. This means text is highly visible - I treat posting with a certain long-term awareness.

I have parsed many old forums after a search engine sent me there - this has been my primary education tool for different things. Niche issues encountered by several strangers across the world can be resolved by someone who made a post one time so it can occupy some hardware storage, perhaps many times archived in several scattered drives depending on its contribution value, from where it can be served to anyone else who needs it in the future.

Web forum archive shuttering described by the author would limit information available for the search engine to parse. But the author's solution is to reduce contribution effort because forums may disappear one day? [1]

I agree with you - within a week of something posted is when the "work" gets done (and idea transmitted out of internet into physical group of people who can relay the memory). Seeing a snapshot of a helpful interaction between people skilled in different domains sharing observations/stories/explanations is often inspiring to me and so I would want the creators of this content to put their best foot forward instead of treating it as disposable communication space.

[1] https://en.wikipedia.org/wiki/Nihilism

This reminds me of early Google+ and the wealth of information there on several of Google's long bets such as Glass, Waymo (then Google Self Driving), constantly seeing insightful posts from the engineers working on these moonshots.

And just like it was all gone. There isn't a metric to quantify the novelty of a service, but I genuinely felt like we burnt one the last resources on product/engineering/building/hacking when Google decided to pull the plug on that.

This article highlights that fact too.

A blog post on forums, to avoid web forums and no mention to EZBoards..
I use ifttt to copy all of my reddit comments to Google sheets, it works fine.
Are out of context comments useful?
If the context is relevant, I can usually find an archive of the comments, if they have been deleted.
How is this different from blogs dying due to someone not paying for a server? If I died tomorrow, my personal site would be gone as soon as my AWS credits ran out. Or all the sites which will fail once the CDNs providing their JS libraries are deprecated?

My Hacker News comments will probably live for at least a decade or two. And I had no issues archiving this thread right now. https://archive.is/JrJEe

Who remembers when Photobucket went paid? All those how-to articles, for everything from installing performance parts on cars to repairing your toilet, became useless without photographs.

Imagine all the information that's been lost that was posted to private forums and they were killed off, databases crashed, etc.

At least with USENET your stuff was all over the world. And archived by Deja, eventually.

The web interface of Reddit seems mind-bendingly bad. Seems like you can't just view an entire thread, you have to look at it in a newsfeed type stream of other threads/stories, and when you want to drill down to a particular branch of discussion it has to reload the page and you lose visibility of the rest of it, all the while being blasted with obnoxiously large adverts for offensive stuff like Borat. The web just gets worse and worse... how did we get here?
The day old.reddit.com option dies is the day I stop visiting reddit. The modern interface is a nightmare of sluggish performance and confusion.
> Even with personal blogs, "fragile" as they are, you are able to use the Wayback Machine of the Internet Archive to back up your blog.

With that level of expectation, I'd expect you to encourage people to donate to the Wayback Machine / Archive

Reddit can ether help you achieve unbelievable career success, or you can get called names all-day.

As I rule I no longer discuss cultural , political, or socio issues on the internet. In my social circles , with some friends, I might talk about this stuff.

On the internet it's very easy to forget the human and start acting like a jerk.

I've also accepted I don't need everyone to agree with me. For example I don't use social media or online dating. These things weren't good for me mentally. I don't need to write 80 Reddit posts about how right I am.

That said, I have found Reddit to be much friendlier than Stack overflow for tech questions. On SO you get banned if you write too many subjective questions.

'What IDE should I use ' gets flagged and 2 or 3 other questions like that and your banned.

As long as you aren't straight up belligerent , you won't get banned on Reddit for that.

It's a careful balance. Once Corona ends I very much plan on deleting Reddit. Right now it's an interesting distraction , but like other social media it can't replace real human interaction

I love this luddism-light. Blogs are not necessarily well indexed or well optimized for search engines. Newsgroups are a mess and people abandoned them for many good reasons. Private blogs? I guess those would be fine if actually had any friends.

I will suffer through the "interface dictatorship" to reap the benefits of a huge audience and active moderation, both of which ensure that questions get answered quickly and correctly.

This feels like one massive false dichotomy. Discussing something on Reddit doesn't mean you can't post the content elsewhere. Write a blog post and post it on Reddit for exposure to both worlds.

Reddit, Twitter, and other fast moving websites are best for ephemeral discussions anyway. In most cases, that's a feature rather than a bug. Writing up a thoughtful blog post that's meant to be a lasting reference for many years takes a lot of effort, planning, preparation, editing, proofreading, and other work. The upper limit for this type of material is relatively low due to all of the effort.

On web forums Reddit and Twitter, people don't feel obligated to write perfect posts, so the conversation flows more freely.

There's a place for both long-lived and ephemeral content on the internet. I don't think it's productive to give blanket advice for people to avoid certain forums or deliberately withhold information to make a point.

hm this doesn't seem accurate about reddit

In a google search I often append "reddit" to the end of my search string and often do a custom recent time-range for the result, instead of the most SEO optimized answer which is usually outdated