I just reposted this article to comp.misc, where we've been keeping up a parallel discussion for years. Usenet doesn't have the sheer numbers it used to, but anyone still there is there because we love it. Usenet's flaws remain unmended, but its strengths remain unfettered. :)
I'm on the hunt for a good dedicated Windows NNTP reader, failing that I might have a go at writing my own (that supports text posts only - none of that dodgy binaries rubbish!).
Because Thunderbird sucks, as do other email clients that claim to also support news. As an email app it's below average, but as a newsreader it's downright horrible.
A good newsreader needs to be able to thread and block and filter and preview binaries. Few apps ever did this well, and many that did have since become obsolete (Pan2, Unison, etc).
With the rise of Windows and the decline of Usenet, the demand for newsreaders faded fast, leaving few or no supported freeware newsreaders alive today. Even commercial newsreader apps are increasingly rare.
Uuh, despite the enormous attention, the article was written in just about an hour, and I'm not a native English speaker; that error slipped through the cracks. Thanks for pointing it out.
The same thing happens to every subreddit. It's good, people like it, it grows very large, the quality of submissions goes down (or the same content gets reposted constantly). People are incentivized by their visible "karma" score to post mass amounts of low-effort content.
I feel like Hacker News is one of the only online communities I've participated in where the score is handled appropriately. It's a good personal measure of how well my posts and comments were received by the community, but it's not bragging rights. No one else can see my vote tallies here, so I have no real incentive to increase it.
I've seen this affect sites like Stack Overflow as well. At the bottom of any popular question are usually very low-effort answers from posters who really shouldn't have answered at all. I wonder if it's an example of their otherwise well-executed reputation system creating noise that might have been avoided.
This can be solved by moderation. Not voting, old school approach of moderators removing things. https://www.reddit.com/r/AskHistorians/ still has quite a high standard, and that's because they have strict moderation.
You can't have "democracy" and "voting" and "algorithms" and then have high content.
With all due respect, what changed since the 56k days (which I remember fondly as well) is that people showed up. Before that, the only users on the internet were early adopters. It's unreasonable to expect 'those people' to go away.
The Internet is decentralized. It doesn't matter that all the 13 year olds are congregating on Club Penguin or whatever; we can set up our own communities. That's why HN is great (and other small communities). It's intrinsically hostile to the wider internet simply by virtue of the subject and the format of its content.
The issue is when the unwanted folks follow the cool kids around. This happens a lot on Reddit. People will make a subreddit with the type of community they want. Others will join. Eventually people will complain that the quality has deteriorated. Then the cool kids will start a new one and migrate there. Rinse. Repeat... ad infinitum.
Edit: I don't know if there's a general name for this phenomenon, but I believe it is related to network decay. [0]
Yeah, that was posted elsewhere in the thread. I thought of it as a name specific to Usenet, as Network Decay is specific to television networks. I was hoping for a general term.
I don't think the term is specific to Usenet at all. It's genesis is in Usenet, but I've heard it used many, many times to refer to the general phenomenon that the original Eternal September was a specific instance of.
It's fine to say that, but this phenomenon is even older than the Internet. Think of the hip bar that gains a sudden influx of customers following a New York Times story. Yogi Berra remarked on this phenomenon with one of his famous quotes "Nobody goes there anymore. It's too crowded!"
Another idea related to the general phenomenon is gentrification: the cool artist colony suddenly overwhelmed with yuppies who drive up the rent, forcing the artists to move out.
Is there not a general term for this whole phenomenon?
Reddit really 'dug' (see what I did there) its own grave by aggressively catering to casual users, in order to drive signups and grow the user base. I can't remember exactly what happened at which point, but it all follows the general pattern of appealing to the mass market. E.g. the curation of the frontpage via default subs such as /r/AdviceAnimals that don't really contain meaningful discussion. There's no real need to do this if one is concerned with maintaining an existing community, or the standards of it. As a counterexample, private trackers have been rather good at this, generally by having interviews/invitations for new memberships, or even disabling signups entire. Though these are really the most extreme options.
The other main problem is that like Digg, that there is a powerful cabal of moderators in control of most mainstream subreddits, including those concerned with serious topics like news and politics and they do a lot of "curation" of content by removing stuff, rather than letting the users decide on which links and topics are relevant to discussion (by, ya know... upvoting/downvoting) with chilling effects on those respective subs. That this was allowed to continue, without overhauling and maybe even democratising the moderation system of default subs, was Reddit's choice. I can only suspect that this either (a) plays into the hands of advertisers directly via corporate censorship or (b) indirectly, by relegating content from users who fall outside of the "mass-market" demographic to niche subreddits or other websites.
Curation of shitty content is also built right into the algorithm.
They'll hide behind "But people are voting for this mindless content" but that's not the full picture. A vote after 5 seconds is actually weighted higher than a vote after 5 minutes.
It's specifically because Reddit the company is against too much engaging content. They want people to smirk, chuckle, acknowledge, and move on. Engaging content means fewer pages viewed, which means fewer ad views.
A vote after 5 seconds is weighted, or acts as if weighted?
If that's built into the algorithm, that's one thing. If the social effect of early moderation is amplified, well, that's still a problem (and a major one), but ... of a different sort.
The first graph linked below shows "hotness" as a function of time with upvotes held constant. The equations are above the linked section. Compounding this effect, there is a feedback loop where hotter elements are more likely to be seen and voted, which increases hotness logarithmically.
So no, the algorithm doesn't intentionally punish dense material. But it does happen incidentally.
It happened a lot before Reddit too. Forums always used to have this cycle of starting out appealing to more mature audiences before getting flooded with casual users and spammers and many people then moving elsewhere.
Heck, in the world of webmaster forums, the ease in which they got filled with spammers and low quality content meant that the same people usually ended up being members of dozens of different forums and moving to a new site every month or two.
Yes, but there have been an increasing trend in "reddit style" comments on news.ycombinator.com . Myself and other users have noticed, called people out, downvoted etc for the lousy comments.
But the realities of the threading style & upvote democracy is that lowest common denominator spreads easily.
It always sucks to have your insightful (in your own eyes at least) comments downvoted, just because there's a group of people who disagree with it. It's a downvote democracy, but not a true democracy because it only takes a minority to vote that way. I don't even have access to this downvote feature, I guess I don't have enough karma.
You do have access to the upvote button,so your vote can cancel out an unfair downvote.
Comments that get downvotes also don't get upvotes, so it's not just the small group who can downvotes who are causing the comments to be gray, it's also the large group who could upvotes but who chose not to do so.
If something is unfairly downvoted, wait a while and someone else will usually upvote it again.
But it's also worth remembering they're just internet points, there's no real value. Keep contributing good stuff & you'll eventually get past the 500 karma mark. After that it's just a vanity number.
Out of curiosity, could you explain what you mean by more Reddit style comments?
Because I don't think I've seen too many low effort comments here recently, or at least not much more than there were in the past.
And while political threads get a bit out of control, my experience there is that the most volatile, disruptive posts tend to get downvoted to hell before they can cause too much of an issue. So even if a certain percentage of alt-right/SJW type content is a bit more common than it was before, it tends to get rightfully buried under hundreds more thoughtful ones.
Example of "Reddit style comments" would be low effort jokes. Some subreddits are addressing this by using a convention where you can mark submissions with a "[serious]" tag, forbidding such replies (if there are any, they will usually be removed by the subreddit's moderators).
On HN, quite a few of them seem to get heavily downvoted (sometimes enough to mark them as dead, meaning you won't see them unless you explicitly opt in to see them).
There's a lot more joke comments than there used to be even 2-3 years ago. I would date the shift to dang taking over and asking everyone to be nicer - before that HN was more caustic and the noise comments were shut down more quickly.
Comments that are only jokes, with nested replies that are word-plays are continuations of jokes. Good for a quick laugh occasionally but I prefer not to see that on HN (puns + groupthink meme jokes are not what I come to HN for).
Which, as it happens, predates the arrival of 56k modems by several years. The author may in fact be one of those who arrived afterwards.
Which all goes to show that people, in general, believe the best time was when they arrived/discovered/attended whatever. In other words, "pull the ladder up after I get to the top."
Indeed. I wasn't on the internet in September 1993. Actually my first modem was a 28.8k around 1994/1995, but I can remember one of my most productive internet eras when I had a 56k around 1997.
This is true to a degree (reminds me of the old MAD Magazine adage that "MAD was at its best whenever you first started reading it"), though I feel communities usually have a localised version of Eternal September as well. At least if they get popular.
Eternal September on Usenet would hence be different from Eternal September on Reddit, Facebook or Hacker News.
I was using a 1200 baud, via SLIRP[1]. In 1994 (I think?) remember cycling into University with a box of 10 1.44M floppy disks to copy Slackware 0.1 so I could install it at home.
The bandwidth of a box of floppy disks on a bicycle was higher.
Go read old IRC logs, it's pretty "meh". You can join mailing lists where there mostly "old schoolers", which is also pretty "meh". The people or the conversations where never that exciting. It was all about the context of being exposed to something emerging. You were chatting with people from the other side of the world while most people had barely read a foreign newspaper. I'm sure one could have similar experiences today, but it's unlikely to be in a text interface among adults with $3000 laptops.
I'd argue it was only "meh" in the same way the average verbal conversation is to a dispassionate listener. For those now able to exchange ideas on a favorite topic with people who were previously inaccessible, it was great. And not because it was emerging, but for the experience and capability itself.
I'd still argue that those experiences and that capability had 'merit' because it was emerging. If you go and try and manufacture hardware in China you can have those same meaningful mundane conversations. Because that's something emerging so there's scarcity of information, everyone is to some extent a beginner and there's therefor incentive to share information. So I don't think it's the people or the conversations, but the environment.
Possibly. But at least there wasn't one million of them and they weren't out of context. Don't like a thread? Don't read it, mark it as mute. Won't see it again.
I don't think that's all of it. The Internet used to be these networks that we could, through tools, arrange to be around us. We'd have our e-mail network and our Usenet network and our IRC network, and we have ways to view all those things that aggregated among different e-mail servers and different USENET groups and different IRC channels, and gave us a view around How We We Engaging With The Content (e-mail for one-to-one or one-to-many longform discussion, USENET for many-to-many longform discussion, IRC for short-form discussion, both one-to-one and many-to-many). Now, instead of forming the center of our own Internet experience, there's these silos -- Twitter, Facebook, Slack -- and we form a network around the silo instead of our networks having us at the center.
Now, having large number of people makes many of the features of these silos more desirable, they're not unrelated. But if you could find a way to scale up the "me-centric" view of the web to modern needs, then everyone -- early adopters and the mass audience -- would both benefit, I think.
> But if you could find a way to scale up the "me-centric" view of the web to modern needs, then everyone -- early adopters and the mass audience -- would both benefit, I think.
Even assuming there were adequate mindshare to develop and maintain such an effort-- let's even just pick a modern day USENET-- how are you going to pay the bills?
It seems to me the way connections are built on the Internet is different now. It used to be for example that 90% of the people I knew online I didn't know in person. These days that percentage might be inversed.
Not saying it's the only factor, but when connections are built via real-life relationships instead of communities of interest and then sprinkled with automated personalized content aggregation perhaps you end up with an environment where everyone seems to live in their own bubble of information overload.
No, but we can advocate for tech world equivalent of user/dev apartheid. Equal people, seperated, segregated internets, so we can milk the cow, while still enjoying salad.
If only it where not for those meddling open devs, distributing all the nice things among the clueless.
Great point about Twitter and FB not having the "I've seen this" flag (although they probably have this info for their "engagement metrics"), it keeps people addicted and returning. I remember being glued to Twitter during the Mumbai terror attack, but the way it was designed, it was an endless stream of the same info, repeated. And spam, since bots add trending hashtags to their junk messages. I guess it's like cable news' rolling coverage, but instead of the same info repeated every hour, we can now read the same thing every second...
With Facebook I scroll down until I see something I've seen before, then I stop. Doesn't seem very hard, and I imagine its what most people do. If I want to revisit older things then I keep scrolling, but that's almost never the case.
Still possible with Twitter if you use a 3rd party app, like Tweetbot on Mac / iOS and Fenix on Android. I don't think it's possible on the website or official app (and if I was forced to use Twitter as currently designed, I think I'd give up on it).
That's what I do, but I also have to change the ordering from "Top Stories" to "Most Recent" fairly frequently. That usually puts out different stuff (Facebook seems incapable of remembering the setting. I assume it's deliberate design at this point, since such a blatant bug would have been fixed by now).
I suspect it's only a workable strategy if you either have few friends, or stop following all the ones with a low signal to noise ratio, or if you use Facebook somewhat frequently. Otherwise, there'd just be too many updates to finish scrolling through them in a reasonable amount of time.
"Most Recent" is most recently updated by comments, which is mostly useless for me. I just stopped using FB as much, checking in just to see how people are doing every few days.
How about implementing an RSS interface on top of the likes of Twitter and Facebook by screen scraping? Would have to take authentication of course. Then we could consume it via RSS readers just like the good ol' days. Does that already exist? I would pay for that.
Facebook used to support RSS feeds. So did Twitter. Now they're both gone, to force people to use their ad-laden interfaces. There are services to generate RSS feeds from both, but there's probably some catch to that.
The social media are pushing the social thing, which is fair enough from a business standpoint.
What isn't fine is that users are (intentionnaly, could be argued) led to believe that the media part is equivalent to traditional "RSS-like" media like AFP/Reuters/Bloomberg who carried the responsibility of being trustable.
There's a place and implementation for everything and social media is not an appropriate platform for diffusion of factchecked information. How we make use of the different information dispersion platforms is on us!
There is a tool that converts Twitter to RSS. I used to use it to get updates from a fantasy author who had neither a website nor a mailing list. It was self hosted but can't remember the name.
Most probably it would be illegal, against each website's TOS. If such tool catches up, it will be removed - and you still need to account for upstream changes that periodically kill the functionality.
Thinking of such a tool as a service is a symptom of the same disease that got us where we are in the first place. If such tool would, instead, be open-source and self-hosted (or a component of trivially self-hosted) application, websites could do exactly shit about it. Traffic will be coming from everywhere, and the tool will get updated by people to overcome upstream interface changes.
Ultimately, it's SaaS webapps that's the problem. The model has legitimate benefits (lack of installation, cross-platform), but it also is the reason the Web centralizes so much.
Forget Github. It might be just a regular git repo on someone's server. IANAL, but could companies really go against a person hosting the source code of a software that scraps a website?
Even if, that's probably solvable (by turning it into cat-and-mouse game, TPB/SciHub-style). Have a non-turing-complete set of conversion rules that do scrapping -> standardized-data-format via a general-purpose scrapper. Distribute those through P2P. There might be problems with malicious actors, but generally this should be possible to implement in a way that's mostly transparent for end users.
> And you still need to account for upstream changes.
Upstream changes will never be too extreme to handle manually, if we're talking scrapping solutions - because any such extreme changes would also piss off the regular browser users.
Unfortunately, that's precisely what the CFAA does, in the United States. EFF's interpretation:
The Computer Fraud and Abuse Act (“CFAA”), 18 U.S.C. § 1030, is an amendment made in 1986 to the Counterfeit Access Device and Abuse Act that was passed in 1984 and essentially states that, whoever intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains information from any protected computer if the conduct involved an interstate or foreign communication shall be punished under the Act. In 1996 the CFAA was, again, broadened by an amendment that replaced the term “federal interest computer” with the term “protected computer.”18 U.S.C. § 1030. While the CFAA is primarily a criminal law intended to reduce the instances of malicious interferences with computer systems and to address federal computer offenses, an amendment in 1994 allows civil actions to brought under the statute, as well.
I just don't get Facebook anymore, and I've never enjoyed Twitter. I understand they can still be useful tools for finding out some news quick, or sharing pictures of your kids with grandma. But is there anything online that is even close to what Myspace and early facebook were like? People seem to have forgotten what it was like when these were tools to augment your actual social life.
Yeah, I'm old as well. Things change. Not for the better maybe, but you can't turn back the clock.. the only way forward is to make something new and better.
I'm still using RSS without issue. Funny thing is, i didn't really use it back in the day, I've just gravitated toward the best UX available over the years.
This. Many stuff is still there for who want to use them. I use feeds, email and nntp everyday. Also paper books, fountain pens. And also WhatsApp. I don't get the need that the stuff I use to be justified by its popularity or its newness.
The problem is that it appears most people actually want noise, and it's becoming especially apparent as a lot of these content channels become more mainstream.
Anyone who's been on Reddit for the majority of the site's existence will probably have noticed monumental shifts in popular content, which I feel is essentially a movement away from serious diverse discussion towards entertainment and self validation.
I don't think I'd be going out on a limb to say a large amount of HN readers are interested in serious discourse, and I wouldn't be surprised to find there are many like me who feel completely alienated by modern Reddit. Previously it was enough to just ignore the default subs, but the momentum has become so strong that it has become pervasive even in niche subs.
I've been thinking about it a lot recently, probably mostly stemming from nostalgia of what Reddit used to be, but I feel something that is quite important is identifying that different people are interested in different content, an that nobody is really entitled to stop others enjoying the category of content they want to consume.
I don't care about entertainment on Reddit, and I particularly don't care about image macros, in-jokes and one liners. I absolutely love the high effort informational posts and discussions though, and some of them are truly insightful. I yearn for a mechanism to reliably filter out most of the entertainment things and promote the high effort content. I'd love for it to stay some form of voting mechanism though as it seems really effective when the community votes using congruent inclinations.
The Slashdot voting system worked using a classification system (interesting, funny, etc.) but that is hardly good supporting evidence for content classification as it didn't work particularly well, perhaps because it was so limited (votes themselves were limited in quantity.)
My current line of thinking is using emoji reactions might be an interesting option like you see on GitHub comments. If I want to avoid the entertainment posts I could filter out posts which predominantly have the laughing emoji. There is evidence that emoji reactions themselves are compelling enough to be used, and possibly more compelling than the style of categorisation that Slashdot used.
I feel that it's fundamentally important that a system like this needs to integrate into existing communities somehow, as there's no point having it if there's no content to apply it to. This is probably the hardest part but possibly also a very compelling part, having a content indexing / classification system might also be a way to centralise a lot of this content.
This post got a little long winded and ranty so apologies for that, but I've been wanting to get these thoughts in writing for a little while. If anyone has any ideas around this general topic I'd love to hear. I've gotten to the point where I'd like to invest energy into attempting to solve it.
Yeah it seems that people like the memes, image macros, and one liners. It's throwaway entertainment so people don't have to think (of course, there are people who do want to think and have intelligent discourse too). Like someone else wrote in this thread, what happened was that people came, people who didn't grow up using the Internet in the days when it was run by academics, government, and hobbyists.
I think the solution (as much as there can be one) is to just stick to your own communities where you find the discourse that you enjoy. For me, it's Hacker News, certain subreddits, and some forums.
My view has always been that after a day of work or school, the last thing most people want to do is engage in a deep discussion. Instead they want to scroll down funny content (I do this myself or just play video games, just in case I sound like a snob).
It's not that these points aren't valid; they are, and I know some people who've been rightfully hammering the exact same points for, well, decades, at this point.
But we arrived here neither out of pure accident or grand overarching malice, but simply out of the necessity of the now-commercial web and its lack of good business models, and perverse incentives.
Back when most of the traffic on the Internet was academic, institutional, government, or hobbyist, the incentives were to get the content out there and interoperate with others, because the dissemination of the information was seen as having intrinsic value, and the expenses were covered by out-of-band means (i.e. not monetized through the content or the consumers).
On today's particular flavor of a commercial web, especially when your next funding round depends on showing user numbers, easy out-migration is a liability, easy in-migration is a feature, so even the use of APIs frequently helps the product and the company more so than it helps the user.
When commercial entities started appearing on the Internet, particularly on the World Wide Web, some sites were pure billboards containing only an about page and contact information, e-commerce sites funded the operation through selling actual wares, but news/media/entertainment sites brought with them their previously trailblazed business model of giving away content and trying to recoup some of the cost with advertising. Later, VC-funded content silos took a page from early hobbyist web forums (that were mediocre reimagining of BBSes) and created login-walled playgrounds that funnel the content inward, making it easier to track, analyze, and monetize. It's no surprise that today's four largest display ad servers are Google, Facebook, Verizon/Yahoo/AOL, and Twitter.
Other business models only work for specialized players who can command some brand awareness and attract a discriminating customer willing to pay for quality (e.g. big-name or niche newspapers, streaming media sites, data brokers); and, as the HN meme goes, micropayments get much more interest from those who want to collect them than those who want to pay for them, so the everyman's market is full of me-too sites vying for limited attention, or captive content silos that re-create everything on the inside. The battle is largely lost, unless realistic progress is made in the web monetization space.
Luckily, there are no technical barriers to people banding together and making interoperable services like the way things used to be -- and keeping up the protocols that make that a reality. It's just that they will have to contend with the realities of playing in that space and competing with similar offerings that don't. Havens on the old web, or the old Internet for that matter still exist, just like amateur radio still exists along with public access television. It's just not where the mainstream attention-hours go.
The solution was in front of us all this time. It's the semantic web. What was missing is an accessible client, and it will be available soon.
Imagine one giant unified semantic decentralized database of everything. Add smart contracts, quantified self, intent inferring, and binary input (à la Tinder or Akinator). Organize this on a timeline, so that your OS becomes a task management system. That's it.
Unfortunately, my strategy is to target kids that haven't yet be corrupted by current communication paradigms. I think 5 to 12 year old is the sweet spot. Older people will be able to use it, but I don't know if they'll be able to grasp the full potential. Only time will tell.
query; results. So yeah. It's easy enough to scrape. All this stuff about structured data is trying to find something that isn't there to begin with. If you want structure from unstructured data then use machine learning, don't try to sell everyone a "semantic data format" they have no incentive to use.
Walled gardens and a "protectionist web" are the natural outcome of the legal structure we've chosen to put up around the internet. It's not a technical limitation, and technical solutions can not really affect it.
Just this week there was a "Show HN" that combined data from Twitter et al in interesting ways. It was offline less than 24 hours later as all the comments indicated that it was a clear violation of the policies of various content providers.
Why should we care about those policies? Well, because under the CFAA, we've made it a federal crime to send packets to a server in a way that displeases the server's owner.
There is no technical limitation stopping people from collecting and curating data from many sources and combining/filtering them according to their own interest. The internet is already open in principle (cue Schneier: "trying to make digital bits not copiable is like trying to make water not wet"). The issue is that we've given the Facebooks and Googles of the world the right to hold our data and our personal networks hostage.
Start a decentralized, standardized protocol like email or IRC and companies will cooperate insofar as they must to ride the wave, and then they will work aggressively to corner things off into their own little world.
Understand, all software companies want one thing: lock-in. They want to make it so that there is as much pressure as possible to remain on their platform. It's the age-old story of someone who can't move to Mac, even though they greatly envy it, because their greeting card program from 1997 won't work on it. Instead of importable programs, it's importable personal networks -- but now, with everything server-side, it's usually illegal to try to bridge that gap on the user's behalf (insofar as doing so involves contacting the server of a competitor).
The situation with potential copyright and patent violations was precarious enough when it was all occurring on the user's local machine (WINE is in a big legal grey area, for example; my instinct is WINE would lose if MS ever decided to seriously try to squash them), but once you cross the line into some company's IP space, all bets are off. The CFAA allows them to define "authorized access" to their servers on their own terms, including "people trying to access our server to provide data portability". This has already been litigated with specific regard to Facebook in Facebook v. Power Ventures.
As long as we give companies the legal tools to exert effective ownership over user-generated data, we are destined to see well-designed, decentralized protocols that maximize availability, resiliency, and portability get whittled away by the overriding corporate interest in establishing some element that can be used to keep users locked in.
The Halloween memos may have caused a stir in the late 90s, and they're all but forgotten now, but their sentiment is more alive than ever.
> Just this week there was a "Show HN" that combined data from Twitter et al in interesting ways. It was offline less than 24 hours later as all the comments indicated that it was a clear violation of the policies of various content providers.
I missed that. What's the HN URL?
> Why should we care about those policies? Well, because under the CFAA, we've made it a federal crime to send packets to a server in a way that displeases the server's owner.
There are ways to avoid such oversight. Consider, for example, Sci-Hub or TPB or third-generation dark markets. What's hard is doing that at scale. And that, I believe, is a huge niche that's aching to be filled.
> There are ways to avoid such oversight. Consider, for example, Sci-Hub or TPB or third-generation dark markets. What's hard is doing that at scale. And that, I believe, is a huge niche that's aching to be filled.
TPB and BitTorrent helps move files. Sci-Hub helps move a particular type of files (scientific papers). So maybe we need a TPB/BitTorrent for data streams? Some way of moving around generalized access to and results from APIs of server platforms?
The question is whether we're fetching this off of the same APIs i.e. Twitter themselves are using, or if we're scraping it from server-side rendered sites.
If we're attemtping to do it via their open APIs we're going to be hitting rate-limits, and probably also face liability.
What if there was some service that you could run on different computers, that each had their Twitter API tokens, and then we'd distribute the requests to different hosts, so that it could not be tracked.
>There are ways to avoid such oversight. Consider, for example, Sci-Hub or TPB or third-generation dark markets. What's hard is doing that at scale. And that, I believe, is a huge niche that's aching to be filled.
The TPB saga should scare all of us, because it shows how far corporate interests will go to remove a thorn in their side. TPB lives on because the swarm refuses to let it die, but its founders were rewarded with a witch hunt led by the top brass in the United States, exile, and imprisonment.
Anyone who wants to take this on: do not expect a more favorable outcome than TPB's founders got, especially if you live in the U.S., where they won't have to threaten sanctions against your homeland to get you brought down.
Perhaps some form of decentralized hosting to offload the costly aspect of hosting an online community to its members? The percentage of people with access to ample broadband connections at home is increasing constantly, and bittorrent proved that distributing large amounts of data amongst peers can be done without centralized hosting.
It would be pretty neat if everyone could simply buy or configure some low-power hardware (like a router in both size, cost, and energy consumption) and have that become their internet 'home'. When participating in a community that supports the protocol used, that device simply allocates an amount of bandwidth and does its share of distributing content to the swarm. All a community would need is a central place to host the data needed to get started. Anyone could set up a new community with a minimum of means.
If technique used is standardized (and open) and gained popularity, service providers could even create hosting packages that do the hosting of your internet 'home' for you for a small monthly fee, so you don't have to bother with setting up your own device (although you could, and should always be able to).
This kind of concept probably exists of course. The challenge lies in getting it both technologically feasible for mass-scale adoption, and simple enough for anyone to participate. Still, doesn't this make sense for a future were we won't be as depended on a handful of commercial silos?
This has been tried in various forms, and always fails for a few reasons:
- Spam. When the cost of bandwidth goes to zero, the most useless freeloaders are enabled with no negative consequences for their pollution.
- Media storage. There is a huge demand for storage of (pirated) movies, tv, music or video games, which sucks up the vast bulk of volume. Entertainment needs monopolize communication resources to the detriment of knowledge preservation and freedom of expression.
- Legal and other liability. Aside from the piracy, there's child porn, private information, politically sensitive topics, etc. Society wants an accountability mechanism and will destroy or discredit platforms that lack it.
Commercial silos tend to address all three. Social media in particular is effective at #1 by leveraging the social graph as a filtering and reputation system. Which means once people are invested in their profile or channel, you can wield it as leverage to ensure compliance with #2 or #3.
There are still huge pieces missing to do this in a decentralized and federated way. For starters, a reputation and identity system that is not fully public and transitive. And also, a return to the willingness to tolerate violations of #2 and #3 on a local scale for the sake of intellectual freedom, which the contemporary web sorely lacks.
Email isn't even an exception. Becoming a reputable sender is hard, the data volume is still limited, and the privacy protection is laughable in the wild.
I'm not saying we should get back to IRC or to NNTP.
I am. Not in the sense of go back to using them for everyting (they're still there if you want to), but int eh sense of building some new protocols on those existing foundations. NNTP was really good given its technical limitations. TBH I think part of the secret of Facebook's success is that it hews to the same ethos of standardization and simplicity, as opposed to myspace which quickly feel victim to its own customizability and ended up being as chaotic and hard to navigate as the web itself.
Did we get to this point of fragmentation due to smart phones and their walled garden philosophy? People stopped making applications for the sake of interoperability. ... if RealAudio™ was founded today it'd be king.
Or is facebook to blame? John Gilmore famously said "The Net interprets censorship as damage and routes around it." -- he was wrong, censorship just has to wait for the right monopoly to appear. And with the success of monopoly interoperability becomes its casualty
Amazon and Facebook, to me, are symptoms of a deeper problem: Technology is still hard. We're building complex large-scale products because the non-technical consumers can't use the simple small-scale products.
We're here because it's convenient, not because it's good.
I always felt that Google should have created simple tools for web publishing and creating personal walled-gardens - make the web personal for 7 billion non-technical humans, and the brands they consume. Surely there must be something in between Facebook and Jekyll/Hugo.
G+ copied Facebook when they should have studied why people were using Facebook instead of the open web. Don't make the problem worse, fixed those problems; take the role of guardian of the internet. Create or support standard APIs, release relevant IP/research, and bankroll SV startups that build the tools to publish or consume through the open web.
Google is still an ad/search company that relies on the open web. They need to protect it and drive further innovation. What happens after the coming adpocalypse - micropayments? Subscription bundles? Who knows? Or, perhaps we'll be chained to ads forever because we'll be locked into walled-garden apps... what a hell that would be.
Are there any companies working in this space? Closest I've seen is Squarespace, and even then... not really...
> Surely there must be something in between Facebook and Jekyll/Hugo.
I guess Wordpress is an example. It's got a hosted plan, which requires little more computer literacy than Facebook, and you can also leave at any time and self-host the software, which keeps the hosted offering honest. But it's just not as contagious as something like Facebook. That's the problem, Facebook isn't just more 'convenient' than self-hosted tools, it also has your friends tagging you in photos.
I'd argue that, in terms of features, Wordpress is little better than Jekyll/Hugo. Social integration is the aspect that's missing. Chat, access control, tagging, push/pull update, etc.
I envision this is a two part problem: the "back-end", one-click install or oauth register services to create your private space and identity (possibly a VPS or agglomeration of AWS services) for sending/receiving emails, direct messages, tags (push), and friend posts, group updates, news subscriptions (pull), plus APIs for search, access control, creating/managing your own content, so on; and the front-end, a glorified RSS reader for using the above services through standard APIs.
That's the challenge. The solution is way more complex then the existing offerings, like WP or other simple blogs, but all that complexity must be buried away so it's at least as easy as registering to Facebook.
It's possible, all of the constituent problems have been tackled separately... what's lacking is a unified standard, comprehensive tools, and stable (corporate) backing.
and I kinda wonder how much of this ultimately falls on ISPs for not building out high speed internet... it's not inconceivable to imagine an always on "social network" box, or more options for home servers
> Surely there must be something in between Facebook and Jekyll/Hugo.
There's an effort to try and build some standards for open web sharing at indieweb.org - cross-post comments, syndication to existing social networks. A few of those standards have now been given the W3C stamp (MicroPub, WebMention).
There's some services built on top of it like WithKnown, MicroBlog and people building on top of WordPress, building their own site and so on.
The founding principle of that community (build what fixes your problem, don't try and solve everyone's problems) keeps it quite well focussed.
Actually, that's a misquote from an incident at Stanford involving USENET in the early 1980s. There was an administration attempt to censor alt.humor.funny over some forgotten PC issue. The group was blocked at the main Stanford USENET gateway.
It didn't work. USENET nodes, when they connect, automatically exchange any messages in any group common to both that the other node hasn't seen. There were a few other USENET low-bandwidth nodes on campus that connected to the outside world, and as long as there was at least one, the missing posts in "alt.humor.funny" would be re-inserted. USENET really did route around censorship automatically.
I still use NNTP every day. I've never used Usenet, but it's very useful (through Gmane and Gnus) as a way to follow mailing lists. Also, the internal forums at $DAYJOB support connecting through NNTP, which is extremely convenient.
Likewise I use IRC every day, through Bitlbee, to interact with people on Facebook Messenger and other platforms.
I can do all of this from my client of choice (I choose Emacs) without a problem.
Admittedly this doesn't solve the problem of topicality, but I think things are better than they seem.
As someone who remembers being uber frustrated at having to use trillian to talk to all of my friend, I disagree, there was major fragmentation back then as well. Also for which BBS you were on, which friends were on AOL or other nerfed internet.
Would trust this service, if I could read the source. I don't know what the plan with MeetFranz is, but since it isn't open source, something tells me that there at some point will be monetization involved. Monetization isn't bad, but it also will at some point cause misalignment between it and the users.
Maybe that'd help ditch Twitter, but not Facebook. There needs to be a better (de-centralized) solution for writing that's longer that also provides the ability to add photos, videos, etc. The substitute for Facebook should be like Facebook in almost every way in terms of features to get better adoption, with improvements in several areas where Facebook is weaker.
My own theory about why our ability to focus on what we want to on the internet is Attention Capitalism. Our attention is the latest resource to be exploited by capitalism, and each company is attempting to control what we focus on for their profit. I've noticed this trend; very little software let's me control what information I see anymore. It's almost all controlled by some algorithm making me see what they claim I want to see.
> It's almost all controlled by some algorithm making me see what they claim I want to see.
The algorithm is known as "doing what users want". Very popular with companies and even open source projects nowadays. It works like this: you do some "analytics" of your product, or look what sells for others, assume it reflects what users want (and the things that are missing are obviously not wanted by users), and do those.
This algorithm ignores the fact that a lot of the time, users don't get any way to express preferences. Users buy/use what's available and what's popular. Through this algorithm, producers make choices for the consumers, and then use statistics to justify them.
Don't forget the other part that's critical for the big players: lock-in. The first thing gets them to join willingly. The second coerces them to stay. A one-two punch for a acquiring and keeping market share with high margins.
More of these website are just becoming data silos and not sharing their data. Unfortunately I don't know of a better place to write programming content. https://sergiotapia.me
If you're a developer and you care about this, like I do, where would you recommend we write articles sans-self hosting?
> Medium is quickly becoming the central place for programming articles
How? There are many programmers on medium, maybe. But it's a website I categorically avoid (too big and clunky, and hardly ever any quality content found there), and yet I can follow many programming blogs.
Frankly I think what is missing is an understanding of statistics.
Even having “1000 friends”, which sounds amazing, is statistically insignificant when we are talking about populations of millions or billions. Think about the last “really long thread” you read on Reddit or something, and think about when you tuned out: was it a few dozen comments, maybe a few hundred? Still statistically insignificant, or at the very least severely biased.
People are regularly exposed to, and worse respond to, statistically biased samples. This is a really, really bad thing. What we need is a way to almost forcibly blend samples from many different populations so that the number of comments you can “stand” is a more representative sample. That way, when you get up in arms about “what people think”, it might actually represent what “people” think instead of “my friends” or whatever other biased sample is out there.
I'd like to see better moderation systems. Moderation done by people who are actually good at it rather than just upvoting or downvoting comments depending on whether they agree with them or not (moderating the moderators). Ways to mark comments as duplicates so they are all grouped into one bundle. A way to avoid the "first posts" effect, where the earlier posts are likely to be the highest rated regardless of their merits.
Aggregators like Hacker News are already part of a solution; putting interesting articles in one place so that we don't have to subscribe to 10^8 personal blogs and fringe tech sites.
I always thought slashdot had a pretty good moderation system. Would in most cases separate the wheat from the chaff leading to excellent comment threads
Yeah meta-moderation, limited mod points and a secret algorithm for earning mod points based on reputation works pretty well. Oh.. and a maximum score for comments.
I'm very much for this. I have five messaging apps, two email clients, three file storage apps and a news reader all on my iPhone - just to stay in touch. It's hard to stay organised when businesses use different technologies across different platforms and there's no single place just for "Messages".
I don't care how the message is sent, I just want to make sure it gets to the right person.
Whenever I come across a new Twitter handle I think of following I first run it through http://www.tweetstats.com. I check one simple data there:
Number of tweets per day (tpd); and also maybe tweets per month (tpm)
If the handle is churning out more than ~10 tweets per day I tend to stay away. Because if even a couple of hanldes I follow crosses 30 tpd or so my feed will pretty much become pointless unless I am glued to my feed all the time and I would miss a lot of good content.
Another big problem is repeated content as the article suggests - I follow a quite some literary Twitter handles and longform handles. Everyday I come across just too many tweets that link to the same article. What is worse sometimes the same handles tweet about the same articles repeatedly (to get more views I reckon) with different texts. I usually end up unfollowing many of those handles and that means I actually going to miss a lot of content that I would have otherwise liked but in a moderate dose, at a slower pace.
I request my friends/family members to remove me form there WhatsApp broadcast lists and after sometime I simply tell them if they don't stop with that daily "Good Morning/Evening" forwards and all that crap I am simple gonna block them. I wish WhatsApp let me remove myself from all the broadcast lists I am added to or let me choose that I don't want to receive broadcast messagegs at all (if they can't/won't make it granular).
I've completed given up on Facebook. Sometimes it shows me posts that I've marked hide like five times. It never keeps my friends photos, self written text posts on top but all those video and silly article shares, those annoying and mostly unfunny memes. In fact they have a limit I guess (haven't really used them in a while) and the personal posts get drowned in the mass market noise.
Maybe the problem is we talk of Internet being decentralised but we are all try to find that decentralized Internet at any one place or vert few places - be it Twitter, or Facebook, Google. We are adopting the social networks, content sources wrong... maybe.
Elon Musk said that as time passes, technology continuously degrades, not improves, if people don't make a hard effort to keep improving it. The Egyptians knew how to build giant pyramids, and forgot how. The Romans built fantastic aqueducts, and forgot how. We built vehicles that landed on the moon in the 60s, and the next vehicle we built could only go into earth orbit, and then we no longer even had a vehicle that could do that.
Today, for some reason, everyone thinks that writing software for a platform on a platform on a platform with no interconnecting standards or protocols is a great idea. Instead of trying to improve people's lives, we're just making things needlessly complex, buggy, and bloaty. You need 8 gigs of ram minimum just to browse the web, and god forbid you want to do something like back up your data. I was just in a meeting where nobody could get Google's video conferencing to work.
Instead of building internet technology, we build web technology. The web is harder to write software for. It's an ephemeral, inconsistent, difficult thing. But if we try really hard, we can turn the tide on the unnecessarily complex box we've forced ourselves into.
The proposed fixes are good ideas, but they're bandaids on axe wounds. There are much deeper problems going on that won't be fixed by a feature add or a pivot. We need a re-evaluation of all internet-based technology, how we develop it, how we incorporate it into our lives, and what we want as a society from it.
I'm skeptical the Egyptians simply "forgot" how to build giant pyramids (especially as pyramid construction seems to be well-documented today). It seems more likely that the age of pharoahs ended and giant pyramid construction was no longer an important skill. Technology hadn't "degraded", it had adapted.
Just like we didn't forget how to build lunar vehicles, we just decided that it wasn't worth it in its current state. It's still very possible to do another moon landing, it will just cost a ton of money that corporations won't risk and governments won't spend without a clear political goal like the old space race.
Space technology isn't degrading, it's becoming more cost-oriented. A SpaceX rocket only reached orbit but it was relatively very cheap. Cost efficiency helps space technology reach more people and create greater impact, ultimately making the technology more valuable to society.
It's the same with the web. You say that the web is bloated "instead of trying to improve people's lives" but the reason the web platform is so heavy is because the platform is rapidly adding features and APIs and tools that make it easier for developers of all skill levels to make complicated applications. Letting more developers make stuff in less time improves people's lives. Maintaining backward compatibility at the cost of bloat also improves people's lives.
Society has been constantly evaluating the web and it has consistently reached the same conclusions: RAM is cheaper than developers, a slower platform with a bigger audience is worth dealing with over multiple native platforms, and weird legacy stuff is fine if it means we can run 15 year old websites.
Those two views are equivalent - humans don't forget technology; what was once invented can be reinvented (the more of the old invention remains, the cheaper it is to reinvent it). So it's always about incentives. But GP (and Elon) has a point here.
Infrastructure projects are important and what enables society to grow. Moon landings were plain awesome and had important effects on both the zeitgeist and growth of technology. That we're not incentivized to grow, or even maintain them, says a lot about us, and not much of it good.
And I do think GP is right when they say: "instead of trying to improve people's lives, we're just making things needlessly complex, buggy, and bloaty." Any actual improvement to people's lives that falls off this process on the web is incidental. The conclusions you say we've reached, like "RAM is cheaper than developers, a slower platform with a bigger audience is worth dealing with over multiple native platforms" are not conclusions. They're excuses. Excused for doing shitty engineering in order to get to the market first. And anyone trying to do the things the right way moves too slow to keep up with the hype train.
Ultimately, the complaints about the way technology develops today stem from this: some people see the suboptimal solutions created by our incentive structures, and also notice there are much better solutions available only a small step from the path we're on - solutions we could reach if only we could relax the market pressures a little bit. This is true as much for the web (e.g. for interoperability issues and bloat issues) as it is for transportation (where electric cars finally got viable after one player basically started to shove a product into peoples' throats until the market forces gave up and accepted it).
It also brought a lot of people online who previously were not. Desktops were stashed in some room, one per household, and laptops were still seen as somewhat nerdy or work related.
240 comments
[ 2.1 ms ] story [ 343 ms ] threadTotally de-centralised - your post on your local NNTP server, and it gets replicated across the NNTP mesh to all the other servers.
Built in threading... Actually HN is very similar to how a fairly busy newsgroup would look back in the day.
Weaknesses:
Posts used to take hours (sometimes days) to replicate across the whole mesh. Not so much nowadays tho.
Text only; no native support for rich media or Unicode (although some might see that as a Strength)
https://groups.google.com/forum/feed/comp.misc/topics/rss.xm...
I'm on the hunt for a good dedicated Windows NNTP reader, failing that I might have a go at writing my own (that supports text posts only - none of that dodgy binaries rubbish!).
A good newsreader needs to be able to thread and block and filter and preview binaries. Few apps ever did this well, and many that did have since become obsolete (Pan2, Unison, etc).
With the rise of Windows and the decline of Usenet, the demand for newsreaders faded fast, leaving few or no supported freeware newsreaders alive today. Even commercial newsreader apps are increasingly rare.
Just as it has everything MSN used to have ill have it shut down, rename itself and its website and start all over again.
Seriously?
What I meant was that IRC is still alive and kicking.
I've seen this affect sites like Stack Overflow as well. At the bottom of any popular question are usually very low-effort answers from posters who really shouldn't have answered at all. I wonder if it's an example of their otherwise well-executed reputation system creating noise that might have been avoided.
http://buildingreputation.com
There's a blog, book, and wiki.
Some keyword searches: http://www.worldcat.org/search?q=kw%3A+online+reputation+sys...
https://scholar.google.com/scholar?q=online%20reputation%20s...
You can't have "democracy" and "voting" and "algorithms" and then have high content.
Subreddits, users, posts, searches, and a number of other elements.
Edit: I don't know if there's a general name for this phenomenon, but I believe it is related to network decay. [0]
[0] http://tvtropes.org/pmwiki/pmwiki.php/Main/NetworkDecay
https://en.wikipedia.org/wiki/Eternal_September
Another idea related to the general phenomenon is gentrification: the cool artist colony suddenly overwhelmed with yuppies who drive up the rent, forcing the artists to move out.
Is there not a general term for this whole phenomenon?
"I don't go to that bar anymore, it's too wankerised."
"I don't visit Facebook anymore, it's become wankerised."
"The neighbourhood is too wankerised, I can't afford to live there."
The other main problem is that like Digg, that there is a powerful cabal of moderators in control of most mainstream subreddits, including those concerned with serious topics like news and politics and they do a lot of "curation" of content by removing stuff, rather than letting the users decide on which links and topics are relevant to discussion (by, ya know... upvoting/downvoting) with chilling effects on those respective subs. That this was allowed to continue, without overhauling and maybe even democratising the moderation system of default subs, was Reddit's choice. I can only suspect that this either (a) plays into the hands of advertisers directly via corporate censorship or (b) indirectly, by relegating content from users who fall outside of the "mass-market" demographic to niche subreddits or other websites.
They'll hide behind "But people are voting for this mindless content" but that's not the full picture. A vote after 5 seconds is actually weighted higher than a vote after 5 minutes.
It's specifically because Reddit the company is against too much engaging content. They want people to smirk, chuckle, acknowledge, and move on. Engaging content means fewer pages viewed, which means fewer ad views.
If that's built into the algorithm, that's one thing. If the social effect of early moderation is amplified, well, that's still a problem (and a major one), but ... of a different sort.
Citations requested.
So no, the algorithm doesn't intentionally punish dense material. But it does happen incidentally.
https://medium.com/hacking-and-gonzo/how-reddit-ranking-algo...
Is the monkeysphere is the root cause of the endless bifurcation of communities, culture? https://en.wikipedia.org/wiki/Dunbar%27s_number
John Dvorak once opined that social networks stop being cool when that coworker you secretly hated tries to connect.
Of course, pushing the other way, is the power law distribution of attention. The best explanation I've read for our current winner takes all economy. http://www.shirky.com/writings/herecomeseverybody/powerlaw_w...
Heck, in the world of webmaster forums, the ease in which they got filled with spammers and low quality content meant that the same people usually ended up being members of dozens of different forums and moving to a new site every month or two.
See Hannah Arendt's "The Crisis in Culture" (by way of ... Reddit):
https://www.reddit.com/r/philosophy/comments/50umjw/hannah_a...
http://www.celinecondorelli.edu/files/arendtcrisisinculture_...
There's the "evaporative cooling effect":
http://web.archive.org/web/20101012105003/http://blog.bumble...
I see these as mechanisms based on Gresham's Law, though that's based on a longer argument.
But the realities of the threading style & upvote democracy is that lowest common denominator spreads easily.
Related discussion in same thread: https://news.ycombinator.com/item?id=14699133
I was more lamenting the lousy comments eg. puns, comments that add nothing etc. These are possible with 0second old accounts.
Comments that get downvotes also don't get upvotes, so it's not just the small group who can downvotes who are causing the comments to be gray, it's also the large group who could upvotes but who chose not to do so.
But it's also worth remembering they're just internet points, there's no real value. Keep contributing good stuff & you'll eventually get past the 500 karma mark. After that it's just a vanity number.
Because I don't think I've seen too many low effort comments here recently, or at least not much more than there were in the past.
And while political threads get a bit out of control, my experience there is that the most volatile, disruptive posts tend to get downvoted to hell before they can cause too much of an issue. So even if a certain percentage of alt-right/SJW type content is a bit more common than it was before, it tends to get rightfully buried under hundreds more thoughtful ones.
On HN, quite a few of them seem to get heavily downvoted (sometimes enough to mark them as dead, meaning you won't see them unless you explicitly opt in to see them).
Groupthink is another significant problem, particularly on topics which tend to gravitate toward ideology. Many economics subs operate this way.
jff 9 minutes ago [-]
If anyone else is using uBlock Origin, at least for me the page displayed completely blank until I disabled uBlock temporarily.
Nice one.
reply
sluggg 10 minutes ago [-]
what if I want two? I can't have two third thumbs
reply
piyh 44 minutes ago [-]
When I was a kid, I dreamed of having a functional third arm, this seems like the first step in that direction.
reply
googlryas 25 minutes ago [-]
This is what my ex-gf would call my manhood. Guess why we aren't dating any more...
SimilarWeb says HN gets approximately 20M visits a month:
https://www.similarweb.com/website/news.ycombinator.com
Which all goes to show that people, in general, believe the best time was when they arrived/discovered/attended whatever. In other words, "pull the ladder up after I get to the top."
Eternal September on Usenet would hence be different from Eternal September on Reddit, Facebook or Hacker News.
http://www.radioshackcatalogs.com/html/1993/hr159.html
The bandwidth of a box of floppy disks on a bicycle was higher.
[1] https://en.wikipedia.org/wiki/Slirp
Now, having large number of people makes many of the features of these silos more desirable, they're not unrelated. But if you could find a way to scale up the "me-centric" view of the web to modern needs, then everyone -- early adopters and the mass audience -- would both benefit, I think.
Even assuming there were adequate mindshare to develop and maintain such an effort-- let's even just pick a modern day USENET-- how are you going to pay the bills?
Though RSS is kinda dying, I'll give the author that.
Not saying it's the only factor, but when connections are built via real-life relationships instead of communities of interest and then sprinkled with automated personalized content aggregation perhaps you end up with an environment where everyone seems to live in their own bubble of information overload.
If only it where not for those meddling open devs, distributing all the nice things among the clueless.
I suspect it's only a workable strategy if you either have few friends, or stop following all the ones with a low signal to noise ratio, or if you use Facebook somewhat frequently. Otherwise, there'd just be too many updates to finish scrolling through them in a reasonable amount of time.
[1] https://zapier.com/blog/facebook-twitter-rss-feed/
What isn't fine is that users are (intentionnaly, could be argued) led to believe that the media part is equivalent to traditional "RSS-like" media like AFP/Reuters/Bloomberg who carried the responsibility of being trustable.
There's a place and implementation for everything and social media is not an appropriate platform for diffusion of factchecked information. How we make use of the different information dispersion platforms is on us!
Ultimately, it's SaaS webapps that's the problem. The model has legitimate benefits (lack of installation, cross-platform), but it also is the reason the Web centralizes so much.
Even if, that's probably solvable (by turning it into cat-and-mouse game, TPB/SciHub-style). Have a non-turing-complete set of conversion rules that do scrapping -> standardized-data-format via a general-purpose scrapper. Distribute those through P2P. There might be problems with malicious actors, but generally this should be possible to implement in a way that's mostly transparent for end users.
> And you still need to account for upstream changes.
Upstream changes will never be too extreme to handle manually, if we're talking scrapping solutions - because any such extreme changes would also piss off the regular browser users.
The Computer Fraud and Abuse Act (“CFAA”), 18 U.S.C. § 1030, is an amendment made in 1986 to the Counterfeit Access Device and Abuse Act that was passed in 1984 and essentially states that, whoever intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains information from any protected computer if the conduct involved an interstate or foreign communication shall be punished under the Act. In 1996 the CFAA was, again, broadened by an amendment that replaced the term “federal interest computer” with the term “protected computer.”18 U.S.C. § 1030. While the CFAA is primarily a criminal law intended to reduce the instances of malicious interferences with computer systems and to address federal computer offenses, an amendment in 1994 allows civil actions to brought under the statute, as well.
https://ilt.eff.org/index.php/Computer_Fraud_and_Abuse_Act_%...
Anyone who's been on Reddit for the majority of the site's existence will probably have noticed monumental shifts in popular content, which I feel is essentially a movement away from serious diverse discussion towards entertainment and self validation.
I don't think I'd be going out on a limb to say a large amount of HN readers are interested in serious discourse, and I wouldn't be surprised to find there are many like me who feel completely alienated by modern Reddit. Previously it was enough to just ignore the default subs, but the momentum has become so strong that it has become pervasive even in niche subs.
I've been thinking about it a lot recently, probably mostly stemming from nostalgia of what Reddit used to be, but I feel something that is quite important is identifying that different people are interested in different content, an that nobody is really entitled to stop others enjoying the category of content they want to consume.
I don't care about entertainment on Reddit, and I particularly don't care about image macros, in-jokes and one liners. I absolutely love the high effort informational posts and discussions though, and some of them are truly insightful. I yearn for a mechanism to reliably filter out most of the entertainment things and promote the high effort content. I'd love for it to stay some form of voting mechanism though as it seems really effective when the community votes using congruent inclinations.
The Slashdot voting system worked using a classification system (interesting, funny, etc.) but that is hardly good supporting evidence for content classification as it didn't work particularly well, perhaps because it was so limited (votes themselves were limited in quantity.)
My current line of thinking is using emoji reactions might be an interesting option like you see on GitHub comments. If I want to avoid the entertainment posts I could filter out posts which predominantly have the laughing emoji. There is evidence that emoji reactions themselves are compelling enough to be used, and possibly more compelling than the style of categorisation that Slashdot used.
I feel that it's fundamentally important that a system like this needs to integrate into existing communities somehow, as there's no point having it if there's no content to apply it to. This is probably the hardest part but possibly also a very compelling part, having a content indexing / classification system might also be a way to centralise a lot of this content.
This post got a little long winded and ranty so apologies for that, but I've been wanting to get these thoughts in writing for a little while. If anyone has any ideas around this general topic I'd love to hear. I've gotten to the point where I'd like to invest energy into attempting to solve it.
I think the solution (as much as there can be one) is to just stick to your own communities where you find the discourse that you enjoy. For me, it's Hacker News, certain subreddits, and some forums.
Wasn't there an RFC finalized recently for a site-independent comment system?
But we arrived here neither out of pure accident or grand overarching malice, but simply out of the necessity of the now-commercial web and its lack of good business models, and perverse incentives.
Back when most of the traffic on the Internet was academic, institutional, government, or hobbyist, the incentives were to get the content out there and interoperate with others, because the dissemination of the information was seen as having intrinsic value, and the expenses were covered by out-of-band means (i.e. not monetized through the content or the consumers).
On today's particular flavor of a commercial web, especially when your next funding round depends on showing user numbers, easy out-migration is a liability, easy in-migration is a feature, so even the use of APIs frequently helps the product and the company more so than it helps the user.
When commercial entities started appearing on the Internet, particularly on the World Wide Web, some sites were pure billboards containing only an about page and contact information, e-commerce sites funded the operation through selling actual wares, but news/media/entertainment sites brought with them their previously trailblazed business model of giving away content and trying to recoup some of the cost with advertising. Later, VC-funded content silos took a page from early hobbyist web forums (that were mediocre reimagining of BBSes) and created login-walled playgrounds that funnel the content inward, making it easier to track, analyze, and monetize. It's no surprise that today's four largest display ad servers are Google, Facebook, Verizon/Yahoo/AOL, and Twitter.
Other business models only work for specialized players who can command some brand awareness and attract a discriminating customer willing to pay for quality (e.g. big-name or niche newspapers, streaming media sites, data brokers); and, as the HN meme goes, micropayments get much more interest from those who want to collect them than those who want to pay for them, so the everyman's market is full of me-too sites vying for limited attention, or captive content silos that re-create everything on the inside. The battle is largely lost, unless realistic progress is made in the web monetization space.
Luckily, there are no technical barriers to people banding together and making interoperable services like the way things used to be -- and keeping up the protocols that make that a reality. It's just that they will have to contend with the realities of playing in that space and competing with similar offerings that don't. Havens on the old web, or the old Internet for that matter still exist, just like amateur radio still exists along with public access television. It's just not where the mainstream attention-hours go.
The solution was in front of us all this time. It's the semantic web. What was missing is an accessible client, and it will be available soon.
Imagine one giant unified semantic decentralized database of everything. Add smart contracts, quantified self, intent inferring, and binary input (à la Tinder or Akinator). Organize this on a timeline, so that your OS becomes a task management system. That's it.
Unfortunately, my strategy is to target kids that haven't yet be corrupted by current communication paradigms. I think 5 to 12 year old is the sweet spot. Older people will be able to use it, but I don't know if they'll be able to grasp the full potential. Only time will tell.
This will be as true ten years from now as it was ten years ago.
That's Google (almost; not everything is online). No need for the semantic web at all, or to wait for your vaporware 'solution'.
As a (presumably) end user, do you think that the interface provided by Google is database-like?
Just this week there was a "Show HN" that combined data from Twitter et al in interesting ways. It was offline less than 24 hours later as all the comments indicated that it was a clear violation of the policies of various content providers.
Why should we care about those policies? Well, because under the CFAA, we've made it a federal crime to send packets to a server in a way that displeases the server's owner.
There is no technical limitation stopping people from collecting and curating data from many sources and combining/filtering them according to their own interest. The internet is already open in principle (cue Schneier: "trying to make digital bits not copiable is like trying to make water not wet"). The issue is that we've given the Facebooks and Googles of the world the right to hold our data and our personal networks hostage.
Start a decentralized, standardized protocol like email or IRC and companies will cooperate insofar as they must to ride the wave, and then they will work aggressively to corner things off into their own little world.
Understand, all software companies want one thing: lock-in. They want to make it so that there is as much pressure as possible to remain on their platform. It's the age-old story of someone who can't move to Mac, even though they greatly envy it, because their greeting card program from 1997 won't work on it. Instead of importable programs, it's importable personal networks -- but now, with everything server-side, it's usually illegal to try to bridge that gap on the user's behalf (insofar as doing so involves contacting the server of a competitor).
The situation with potential copyright and patent violations was precarious enough when it was all occurring on the user's local machine (WINE is in a big legal grey area, for example; my instinct is WINE would lose if MS ever decided to seriously try to squash them), but once you cross the line into some company's IP space, all bets are off. The CFAA allows them to define "authorized access" to their servers on their own terms, including "people trying to access our server to provide data portability". This has already been litigated with specific regard to Facebook in Facebook v. Power Ventures.
As long as we give companies the legal tools to exert effective ownership over user-generated data, we are destined to see well-designed, decentralized protocols that maximize availability, resiliency, and portability get whittled away by the overriding corporate interest in establishing some element that can be used to keep users locked in.
The Halloween memos may have caused a stir in the late 90s, and they're all but forgotten now, but their sentiment is more alive than ever.
I missed that. What's the HN URL?
> Why should we care about those policies? Well, because under the CFAA, we've made it a federal crime to send packets to a server in a way that displeases the server's owner.
There are ways to avoid such oversight. Consider, for example, Sci-Hub or TPB or third-generation dark markets. What's hard is doing that at scale. And that, I believe, is a huge niche that's aching to be filled.
TPB and BitTorrent helps move files. Sci-Hub helps move a particular type of files (scientific papers). So maybe we need a TPB/BitTorrent for data streams? Some way of moving around generalized access to and results from APIs of server platforms?
https://news.ycombinator.com/item?id=14639451 ; looks like the linked site has come back online (for now).
>There are ways to avoid such oversight. Consider, for example, Sci-Hub or TPB or third-generation dark markets. What's hard is doing that at scale. And that, I believe, is a huge niche that's aching to be filled.
The TPB saga should scare all of us, because it shows how far corporate interests will go to remove a thorn in their side. TPB lives on because the swarm refuses to let it die, but its founders were rewarded with a witch hunt led by the top brass in the United States, exile, and imprisonment.
Anyone who wants to take this on: do not expect a more favorable outcome than TPB's founders got, especially if you live in the U.S., where they won't have to threaten sanctions against your homeland to get you brought down.
Yup, that's exactly what made the internet great in the good old days.
But how do you monetize users who get to use their favoirite tools instead of yours?
People adopted schema.org so their search ranking improved. Adding markup for machine consumption doesn't make sense otherwise.
We need to build systems that benefit both parties.
It would be pretty neat if everyone could simply buy or configure some low-power hardware (like a router in both size, cost, and energy consumption) and have that become their internet 'home'. When participating in a community that supports the protocol used, that device simply allocates an amount of bandwidth and does its share of distributing content to the swarm. All a community would need is a central place to host the data needed to get started. Anyone could set up a new community with a minimum of means.
If technique used is standardized (and open) and gained popularity, service providers could even create hosting packages that do the hosting of your internet 'home' for you for a small monthly fee, so you don't have to bother with setting up your own device (although you could, and should always be able to).
This kind of concept probably exists of course. The challenge lies in getting it both technologically feasible for mass-scale adoption, and simple enough for anyone to participate. Still, doesn't this make sense for a future were we won't be as depended on a handful of commercial silos?
- Spam. When the cost of bandwidth goes to zero, the most useless freeloaders are enabled with no negative consequences for their pollution.
- Media storage. There is a huge demand for storage of (pirated) movies, tv, music or video games, which sucks up the vast bulk of volume. Entertainment needs monopolize communication resources to the detriment of knowledge preservation and freedom of expression.
- Legal and other liability. Aside from the piracy, there's child porn, private information, politically sensitive topics, etc. Society wants an accountability mechanism and will destroy or discredit platforms that lack it.
Commercial silos tend to address all three. Social media in particular is effective at #1 by leveraging the social graph as a filtering and reputation system. Which means once people are invested in their profile or channel, you can wield it as leverage to ensure compliance with #2 or #3.
There are still huge pieces missing to do this in a decentralized and federated way. For starters, a reputation and identity system that is not fully public and transitive. And also, a return to the willingness to tolerate violations of #2 and #3 on a local scale for the sake of intellectual freedom, which the contemporary web sorely lacks.
Email isn't even an exception. Becoming a reputable sender is hard, the data volume is still limited, and the privacy protection is laughable in the wild.
I am. Not in the sense of go back to using them for everyting (they're still there if you want to), but int eh sense of building some new protocols on those existing foundations. NNTP was really good given its technical limitations. TBH I think part of the secret of Facebook's success is that it hews to the same ethos of standardization and simplicity, as opposed to myspace which quickly feel victim to its own customizability and ended up being as chaotic and hard to navigate as the web itself.
Or is facebook to blame? John Gilmore famously said "The Net interprets censorship as damage and routes around it." -- he was wrong, censorship just has to wait for the right monopoly to appear. And with the success of monopoly interoperability becomes its casualty
We're here because it's convenient, not because it's good.
I always felt that Google should have created simple tools for web publishing and creating personal walled-gardens - make the web personal for 7 billion non-technical humans, and the brands they consume. Surely there must be something in between Facebook and Jekyll/Hugo.
G+ copied Facebook when they should have studied why people were using Facebook instead of the open web. Don't make the problem worse, fixed those problems; take the role of guardian of the internet. Create or support standard APIs, release relevant IP/research, and bankroll SV startups that build the tools to publish or consume through the open web.
Google is still an ad/search company that relies on the open web. They need to protect it and drive further innovation. What happens after the coming adpocalypse - micropayments? Subscription bundles? Who knows? Or, perhaps we'll be chained to ads forever because we'll be locked into walled-garden apps... what a hell that would be.
Are there any companies working in this space? Closest I've seen is Squarespace, and even then... not really...
I guess Wordpress is an example. It's got a hosted plan, which requires little more computer literacy than Facebook, and you can also leave at any time and self-host the software, which keeps the hosted offering honest. But it's just not as contagious as something like Facebook. That's the problem, Facebook isn't just more 'convenient' than self-hosted tools, it also has your friends tagging you in photos.
I envision this is a two part problem: the "back-end", one-click install or oauth register services to create your private space and identity (possibly a VPS or agglomeration of AWS services) for sending/receiving emails, direct messages, tags (push), and friend posts, group updates, news subscriptions (pull), plus APIs for search, access control, creating/managing your own content, so on; and the front-end, a glorified RSS reader for using the above services through standard APIs.
That's the challenge. The solution is way more complex then the existing offerings, like WP or other simple blogs, but all that complexity must be buried away so it's at least as easy as registering to Facebook.
It's possible, all of the constituent problems have been tackled separately... what's lacking is a unified standard, comprehensive tools, and stable (corporate) backing.
There's an effort to try and build some standards for open web sharing at indieweb.org - cross-post comments, syndication to existing social networks. A few of those standards have now been given the W3C stamp (MicroPub, WebMention).
There's some services built on top of it like WithKnown, MicroBlog and people building on top of WordPress, building their own site and so on.
The founding principle of that community (build what fixes your problem, don't try and solve everyone's problems) keeps it quite well focussed.
It didn't work. USENET nodes, when they connect, automatically exchange any messages in any group common to both that the other node hasn't seen. There were a few other USENET low-bandwidth nodes on campus that connected to the outside world, and as long as there was at least one, the missing posts in "alt.humor.funny" would be re-inserted. USENET really did route around censorship automatically.
Likewise I use IRC every day, through Bitlbee, to interact with people on Facebook Messenger and other platforms.
I can do all of this from my client of choice (I choose Emacs) without a problem.
Admittedly this doesn't solve the problem of topicality, but I think things are better than they seem.
The algorithm is known as "doing what users want". Very popular with companies and even open source projects nowadays. It works like this: you do some "analytics" of your product, or look what sells for others, assume it reflects what users want (and the things that are missing are obviously not wanted by users), and do those.
This algorithm ignores the fact that a lot of the time, users don't get any way to express preferences. Users buy/use what's available and what's popular. Through this algorithm, producers make choices for the consumers, and then use statistics to justify them.
A/B testing can't test for the users who aren't present.
Check it out: https://github.com/Medium/medium-api-docs/issues/91
More of these website are just becoming data silos and not sharing their data. Unfortunately I don't know of a better place to write programming content. https://sergiotapia.me
If you're a developer and you care about this, like I do, where would you recommend we write articles sans-self hosting?
How? There are many programmers on medium, maybe. But it's a website I categorically avoid (too big and clunky, and hardly ever any quality content found there), and yet I can follow many programming blogs.
Even having “1000 friends”, which sounds amazing, is statistically insignificant when we are talking about populations of millions or billions. Think about the last “really long thread” you read on Reddit or something, and think about when you tuned out: was it a few dozen comments, maybe a few hundred? Still statistically insignificant, or at the very least severely biased.
People are regularly exposed to, and worse respond to, statistically biased samples. This is a really, really bad thing. What we need is a way to almost forcibly blend samples from many different populations so that the number of comments you can “stand” is a more representative sample. That way, when you get up in arms about “what people think”, it might actually represent what “people” think instead of “my friends” or whatever other biased sample is out there.
Aggregators like Hacker News are already part of a solution; putting interesting articles in one place so that we don't have to subscribe to 10^8 personal blogs and fringe tech sites.
Also useful is https://www.reddit.com/r/all/top/?sort=top&t=day, a bit US/tech-biased but definitely a few million.
I don't care how the message is sent, I just want to make sure it gets to the right person.
Another big problem is repeated content as the article suggests - I follow a quite some literary Twitter handles and longform handles. Everyday I come across just too many tweets that link to the same article. What is worse sometimes the same handles tweet about the same articles repeatedly (to get more views I reckon) with different texts. I usually end up unfollowing many of those handles and that means I actually going to miss a lot of content that I would have otherwise liked but in a moderate dose, at a slower pace.
I request my friends/family members to remove me form there WhatsApp broadcast lists and after sometime I simply tell them if they don't stop with that daily "Good Morning/Evening" forwards and all that crap I am simple gonna block them. I wish WhatsApp let me remove myself from all the broadcast lists I am added to or let me choose that I don't want to receive broadcast messagegs at all (if they can't/won't make it granular).
I've completed given up on Facebook. Sometimes it shows me posts that I've marked hide like five times. It never keeps my friends photos, self written text posts on top but all those video and silly article shares, those annoying and mostly unfunny memes. In fact they have a limit I guess (haven't really used them in a while) and the personal posts get drowned in the mass market noise.
Maybe the problem is we talk of Internet being decentralised but we are all try to find that decentralized Internet at any one place or vert few places - be it Twitter, or Facebook, Google. We are adopting the social networks, content sources wrong... maybe.
Today, for some reason, everyone thinks that writing software for a platform on a platform on a platform with no interconnecting standards or protocols is a great idea. Instead of trying to improve people's lives, we're just making things needlessly complex, buggy, and bloaty. You need 8 gigs of ram minimum just to browse the web, and god forbid you want to do something like back up your data. I was just in a meeting where nobody could get Google's video conferencing to work.
Instead of building internet technology, we build web technology. The web is harder to write software for. It's an ephemeral, inconsistent, difficult thing. But if we try really hard, we can turn the tide on the unnecessarily complex box we've forced ourselves into.
The proposed fixes are good ideas, but they're bandaids on axe wounds. There are much deeper problems going on that won't be fixed by a feature add or a pivot. We need a re-evaluation of all internet-based technology, how we develop it, how we incorporate it into our lives, and what we want as a society from it.
Just like we didn't forget how to build lunar vehicles, we just decided that it wasn't worth it in its current state. It's still very possible to do another moon landing, it will just cost a ton of money that corporations won't risk and governments won't spend without a clear political goal like the old space race.
Space technology isn't degrading, it's becoming more cost-oriented. A SpaceX rocket only reached orbit but it was relatively very cheap. Cost efficiency helps space technology reach more people and create greater impact, ultimately making the technology more valuable to society.
It's the same with the web. You say that the web is bloated "instead of trying to improve people's lives" but the reason the web platform is so heavy is because the platform is rapidly adding features and APIs and tools that make it easier for developers of all skill levels to make complicated applications. Letting more developers make stuff in less time improves people's lives. Maintaining backward compatibility at the cost of bloat also improves people's lives.
Society has been constantly evaluating the web and it has consistently reached the same conclusions: RAM is cheaper than developers, a slower platform with a bigger audience is worth dealing with over multiple native platforms, and weird legacy stuff is fine if it means we can run 15 year old websites.
Infrastructure projects are important and what enables society to grow. Moon landings were plain awesome and had important effects on both the zeitgeist and growth of technology. That we're not incentivized to grow, or even maintain them, says a lot about us, and not much of it good.
And I do think GP is right when they say: "instead of trying to improve people's lives, we're just making things needlessly complex, buggy, and bloaty." Any actual improvement to people's lives that falls off this process on the web is incidental. The conclusions you say we've reached, like "RAM is cheaper than developers, a slower platform with a bigger audience is worth dealing with over multiple native platforms" are not conclusions. They're excuses. Excused for doing shitty engineering in order to get to the market first. And anyone trying to do the things the right way moves too slow to keep up with the hype train.
Ultimately, the complaints about the way technology develops today stem from this: some people see the suboptimal solutions created by our incentive structures, and also notice there are much better solutions available only a small step from the path we're on - solutions we could reach if only we could relax the market pressures a little bit. This is true as much for the web (e.g. for interoperability issues and bloat issues) as it is for transportation (where electric cars finally got viable after one player basically started to shove a product into peoples' throats until the market forces gave up and accepted it).