Ask HN: How about a Reddit alternative where every community is a Git repo?
When I look at all the Reddit alternatives that come up, the problem seems to be that they all intermingle data structure, data storage, moderation, federation and frontend.
How about we build a Reddit alternative which just defines the data structure?
If each community is simply a directory, we could run such a thing as a git repo. Say we run the "aww" community with cute animal photos. Then the repo cold look like this:
2023-06-12/
Look at this funny cat/
content
comments/
very funny
i like her eyes
Our dog climbing stairs for the first time/
content
comments/
go dog go!
2023-06-13
A parrot singing a poem/
content
comments/
but can he sing a book?
This already would:
- Let everybody browse the content
For example via github, gitlab, codeberg etc.
- Let developers write content
Fork the repo, add a file, send a pull request
What does everybody think?
9 comments
[ 54.8 ms ] story [ 377 ms ] threadI think this would need to be load tested and I suspect it would not scale as well as using a some form of a database for the metadata that just links to file storage.
The first step would be to just write 'censored' into the 'content' file and commit. That would remove it from the current view.
Does it have to be removed from history?
If the history is not public, probably not.
Otherwise.. hmm.. the normal way to rewrite history would probably be too expensive. And would not play well with users who already have a fork of the repo.
How about locating the file in the .git/objects/ directory and truncate it, or write 'censored' into the file? That would be a very cheap operation.
Would that mess up anything?
I suspect this could be risky. If someone reports that illegal content was uploaded and it is known the site operates from git then it would be known that the contents would still be in git. The reason I mention this is that people who would view this site as their competitor game the legal system by uploading illegal material via proxies and compromised machines and automatically reporting it to authorities, server providers and dns registrars to get the nodes, domains and admins cancelled. This is how many small sites have been shut down and why many small forums have gone private. The competitors are anyone with a vested financial interest in the existing platforms. I think I would want all traces of the content purged in a timely manor to mitigate this risk if I were running the site.
Another tricky part of this is how git deals with binary blobs. The repos are going to get massive rather fast. I suspect truncating the binaries without making git aware may lead to errors. Even if they can be ignored it may get harder with time to troubleshoot the repo. I suppose you could make a prototype and load test it to see what happens when you upload a few TB of files and start truncating some of them.
I don't think binary blobs should be included at the beginning. The "content" file should be just a text file.
phpBB can use Postgres or MySQL for the metadata and it links to file storage for attachments. There is also a very advanced permissions system with limits per file type. If the site grew big enough the file storage could be scaled with Ceph.
The only thing they leave out is some tuning/hardening of php.ini disabling unused classes and functions [2] and implementing good outbound firewall rules to mitigate most 0-day's. Those are fairly straight forward to implement.
[Edit] FWIW I tried vouching your submission that was flagged but then it was re-flagged.
[1] - https://www.phpbb.com/
[2] - https://www.cyberciti.biz/faq/linux-unix-apache-lighttpd-php...
Personally these questions came off as a rude routine to me, one that prefigure whenever anyone suggests networked social systems. Healthy skepticism is good, just but people who spread disbelief dont rank high in my estimation.
As for rewriting history, maybe use a month by month git repo, so if old content needs to be rebased the blast zone is contained. Job done. Then git submodules to collect each month.
Used as a social platform vs code repository? Surely Github have enterprise class storage and networking so if the question is around hosting the platform on Github then maybe that works at a technical level assuming one is willing to take the risk of Microsoft cancelling their social platform when someone complains about a comment and Microsoft don't want whatever divisive content is on their platform.
If self hosting I would be surprised if a single server or single VM could scale git to millions of people making comments, tens of thousands of comments being edited and thousands of comments being purged. My experience with managing large git servers is that it gets exponentially slower with mass manipulation of the repositories and that was on fairly beefy servers and that was just Jira, Fisheye, Crucible and Confluence and less than 10k developers, not millions of individuals. That is unless by self hosting we are talking about using automation to spin up many instances and even then it is not clear to me that git can scale horizontally like that. Even then that is not really self hosting, that is getting VC funding and starting a new company which draws even more attention and requires even more staffing to manage the content.
I think there would need to be some simulated load testing that attempts to replicate real world social platform behavior to see where git starts to fall over otherwise we would just be speculating in circles. I suspect if this were attempted there would be multiple layers of caching nodes not doing git and trying to minimize or optimize edits and purges to the storage and that may not even be enough.
There's a constellation of ideas around returning to the file system & destructing data, using flatter data & the file system to file data that have so much promise and hope. Using git as a next layer to syndicate.
9p is a well known pointer into this universe. There's less discussion/clarity around how an app or how a network ought become file system oriented, about what flat data architectures would be, but these humble ideas you have here I think show exceeding promise, are clear & reasonable a start that any dev of any language can see & grok (which is huge) & follow a long with. The sense is self evident.
One other humble pointer I might drop might be Hull: the alternative to shell that I'll never have time to implement. https://250bpm.com/blog:153/
I love how such a community as these could form & if things take a turn badly, folks can fork. They can abandon one host & shift to another instance to bave discussions on.