Brilliant technical solutions are solutions to problems that people have, not problems we want to solve. As soon as there will be a serious problem, which can be solved by decentralized or federated social service — it will be solved graciously, designs are in place already.
Decentralized technology is in fact trying to solve a serious problem, but sadly it is a problem that most people choose to ignore in favor of the incentives offered by Facebook and other monolithic centralized services.
However, Facebook et al. provide and maintain huge, single-point-of-entry, solely owned, walled gardens with hostile privacy policies and yet they successfully serve millions of users...
>Decentralized technology is in fact trying to solve a serious problem, but sadly it is a problem that most people choose to ignore in favor of the incentives offered by Facebook and other monolithic centralized services.
So people trying to solve these problems better focus on root causes rather than exciting technical problems which do not change the rules of the game anyway, unfortunately.
In that case we better have a clear definition of what a real-world problem is. Is a real-world problem specifically any issue that impedes ones ability to fully capitalize on incentives? If that is the case then issues that do not impede incentive gain such as mass surveillance and corporate data collecting are not real-world problems.
This is one of thr few essays about decentralization that understands that the problem isn't technical.
However, I'd go further than his explanations of incentives and say that the fundamental problem is that decentralized technical protocols do not solve the centralization of how money is spent.
Examples of that misunderstanding:
- SMTP the protocol is decentralized (technical) and yet we have giant email providers GMail/Hotmail/Yahoo which is centralized (money).
The big providers spent $$$ on 1 gigabyte mail storage + backups + convenience. SMTP specifies how fields are laid out but it doesn't put money in everyone's bank account so they can run residential SMTP servers so the email ecosystem stays decentralized.
- Git the protocol is decentralized (technical) but Github the service is centralized (money). Why? Because Git the technical protocol is not a bank fund that gives every programmer a free $10 VPS account to host their own git repo. The centralization of money spent (Github invests in a datacenter but individual programmers do not) results in centralization.
- Bitcoin protocol is decentralized (technical) and yet the phenomenon of giant China "mining pools" emerges which is centralization (money). The ability to spend money on liquid cooled ASIC chips in a datacenter located near the Artic Circle is "centralized" to the entities that can spend that vast amount of money. The exceeds the ability for the home enthusiasts to compute hashes on a spare computer in their bedroom.
The common theme: technical protocols can be decentralized but the real-world implementation of those protocols end up centralized because physical things like cpus, harddrives, network bandwidth, etc cost money.
This pattern of decentralized technical protocols vs centralized economic behavior is ignored by virtually all decentralization enthusiasts.
So the real puzzle to decentralization is, "How do we _decentralize economic behavior_ when everybody doesn't have the same amount of money to spend?" Nobody I've read about so far has figured that out . That includes Sandstorm/IPFS/Filecoin/Mastadon/Diaspora/Ethereum etc.
The bitcoin example is an especially clear one. You could generalise that even more, and say that the reason why companies exist is because you can make more money by working together with other people than to try to do everything yourself. In other words, the fact that a company is more than the sum of its parts is why this kind of centralisation happens.
My prediction would be that we would see this centralisation overturned at the same point that companies disband as people realise they can do just as well, self-employed. (Which is happening a little bit, actually.)
> The common theme: technical protocols can be decentralized but the real-world implementation of those protocols end up centralized because physical things like cpus, harddrives, network bandwidth, etc cost money.
I agreed with you up until here. Not convinced that centralization is due to the economics of running a computer on a network. It's also a due to reputation and trust.
People use gmail and hotmail, not just because they don't want to pay for the computer, it's because they trust these companies will do it properly and reliably.
Github isn't the best example either. It didn't spring up because of the economics of running a git server. It got big largely because of its social features, and also because it was a pretty trustworthy git repository. I feel far more confident pushing to a git repo than to some physical server somewhere. There's a trust aspect.
I'll skip Bitcoin and got to another mostly decentralized system: paper currency. Exchanging paper currency is largely decentralized, but most people deposit it in a bank, in part due to interest, but largely due to the trust the bank provides.
The common theme to centralized services isn't the cost of computing hardware, but rather raising trust/reliability in the service, outsourcing expertise, and risk mitigation. Big central service providers are good at exactly this.
Seems there's an angle of trust that includes Zipf's law.
Given that attention is limited, if there are some n services, any one individual's awareness of the ith service is likely 1/i. And the general population awareness will also fall with 1/i. If trust is a function (among other things) of familiarity, then there's simply limited trust in a universal-service market (that is: no subpopulation-specific grouping of service, e.g., by geography).
Exactly. This is how society works. One does not go to twenty farms to get grocery. We go to a store or two to get what we need. The source of the food are decentralized, but the platform for selling products are centralized.
SMTP, Git, and Bitcoin are really and truly decentralized in their actual implementations, even as economies of scale favor a concentration of service providers.
"Because Git the technical protocol is not a bank fund that gives every programmer a free $10 VPS account to host their own git repo."
No, but the protocol does ensure that every individual clone of a repository is a full mirror, and that everyone can work offline and synchronize via USB drives if they should need to.
I'm not sure that the kind of decentralization you're talking about is possible or even desirable.
>No, but the protocol does ensure that every individual clone of a repository is a full mirror, and that everyone can work offline and synchronize via USB drives if they should need to.
Correct, but what I was addressing was the recurring sarcasm of "why did we settle for a centralized service like Github when Git itself is decentralized?" ... everytime Github has an outage.
Some people truly don't connect the dots as to why source code repo centralization exists despite the fact that Git is decentralized. To them, Github centralization seems like a weird irrational phenomenon. No, it's actually a quite natural and expected phenomenon!
Yeah. I think part of the problem is that centralized vs decentralized becomes a dualism and people get fixated with an ideal of pure decentralization.
In my view, decentralization is about building resilient systems where it's not a big problem if everyone uses the same provider, or the same three providers, because the content is de facto independent from the operators.
But it's also true that decentralized systems are under constant threat of recentralization because lock-in benefits monopolists...
That's what I'm talking about (in rather indirect language - this is just a quick rant) - the incentive structure drives even decentralized systems (like the web as a whole) to centralization.
And yes, mainly it has to do with the problem that tech people aiming for decentralization are yet to accept that this isn't a tech problem, but economic/incentive problem.
Agreed. I'd even go a level up and simply say that centralized systems generally allow the service to have more control over user behavior. This allows them to monetize user behavior. This profit potential allows them to raise capital. The capital allows them to do all kinds of things that a nonmonetized platform can't, like hire 30000 employees, buy the competition, and engage in big marketing campaigns.
Ultimately though, it's a local optimum. Facebook acts in its own interest, not in the interests of users. The things that Facebook does to maximize its revenue cause great harm to its users. I'm not sure what's going to break the logjam, but something will.
This is true not just of social networks, but decentralized technology in general. There's incentive structures that make it cheaper and easier for everyone to outsource everything, and outsourcing in the realm of technology has large economies of scale. Even though email is listed as a premier example of decentralization, it is also de facto centralized around gmail and a few other major providers.
Although, the open standardization and lack of lock-in that email provides is something to shoot for, and seems to be attainable.
Because BT solved a real-world problem for problem for uploaders and there was a relatively small population of people that needed to be convinced to switch to BitTorrent for the whole community to follow suit. Once a good chunk of uploaders switched to BitTorrent all of the downloaders had to follow them. For social networks you have to convince a critical mass of individual consumers that not only is there a problem with their current situation, that this specific other thing is the solution in a saturated market, and that's it's worth the tradeoffs since it's not quite as polished as what they're using now.
Bittorrent itself may or may not exhibit power law issues. I'm guessing that it does... certainly the ratio of seeders to downloaders has always been bad.
But even if that were not true, Bittorrent is just a subcategory of filesharing in general, and it is abundantly obvious that the vast majority of filesharing occurs through centralized networks. (Google, Dropbox, Amazon, Facebook, Apple, etc)
So, in this case, Bittorrent is akin to these bespoke decentralized social networks that very few people use, and doesn't contradict the article at all.
Nowadays, Netflix alone is about 40% of traffic. Throw in youtube and you're already over half.
But even pirated content had largely been migrated to centralized streaming services. Megaupload started that trend, although I'm not sure what the leaders are now.
When people search for pirated content, they use streaming. When people want to listen to a song, they use youtube or spotify. When people want porn, they use PornHub.
I use bittorrent for linux ISOs and such, but I certainly haven't used it for video content in years, and most people I know don't use it.
> Even though email is listed as a premier example of decentralization, it is also de facto centralized around gmail and a few other major providers.
A few is a lot more than one, though. Even if 90% of email users are on gmail, you can still run your own sever and interoperate with them. Not so with closed networks like Facebook. A decentralized network with big players is a very different thing than a centralized network.
In Germany we have public and private TV, why can't we have something like this for decentralized social services?
The public TV/radio is meant to help people with opinion making independent of the government and private corps.
A public corp could develop the software, promote it and inform people about the usage, but the software could run decentralized on home PCs or smartphones etc.
> A public corp could develop the software, promote it and inform people about the usage, but the software could run decentralized on home PCs or smartphones etc.
Virtually any device with an internet connection can be used as a server. So the solutions are already there, it's just that people trade freedom and independence of decentralized solutions for convenience of centralized apps.
I don't think the focus on incentives is the best way to look at the problem.
A different perspective is that decentralized social services fail because of the lack of capital/revenue and sufficient management structure to compete with their centralized, commercial counterparts.
Money can fund a team of great designers, engineers, testers, marketers, sysadmins, writers, translators, product managers, to name a few. Together they can roll out and maintain a service that meets high standards of quality, uptime, usability, reliability, engagement, customer support... The things that actually matter to people.
Decentralized solutions really don't stand a chance against a well-funded company.
There've been a slew of attempts. A handful, and I mean that literally, hit the big time at any moment: Facebook, Instagram, Pinterest, VK, Twitter, Reddit.
(That handful's composition may change, but its count largely doesn't. Zipf's Law may hold a clue as to why.)
Even massively financed efforts fail: Google+. I'm one of the ghosts in the ghost town, and measured the activity to boot.
Services who've studied other failures ... fail. Imzy.
Services who've gotten big ... fail. MySpace.
Services who sell for near a billion dollars ... fail. Bebo.
Even services that ... sort of succeed, fail to generate sustainable revenues. Twitter and Reddit.
There are many problems, and initially, they tend overwhelmingly to be social -- who is on the network. Several of the noteable successes hit big, in particular Usenet, Slashdot, and Facebook, at least in their day. Draw the wrong initial crowd, and you'll be hampered by it forever.
There's a whole slew of other challenges: spam, abuse, asshats, network and system attacks, costs, UI/UX, performance, relevance, utility, and more. Whilst there are technical components, many of the elements are dominated by soft-skills.
Commercial and centralised systems have tended to be the more successful, through technical simplicity and funding, though there are noncommercial successes: Wikipedia and the Wikimedia foundation, MetaFilter.
And some interesting efforts at a slow boil: GnuSocial, Diaspora, Mastodon.
I'm not convinced decentralisation is the core problem.
Yeah, as much of a fan as i am of decentralization, i have to acknowledge that certainly - and unfortunately - any challenges to wider participation are not merely technically, but do consist of "soft" challenges, such as those related to behavior, incentives, and such. Its a tough nut to crack, no doubt. I'm sure early adopters of other similar systems like email brought up similar questions back in the day.
Although, one has to wonder, do the folks leading the charge for these decentralized services really want each and every single person on facebook to over-night migrate to the new communities? Maybe; but i would gather maybe not. Maybe it started with some programmer simply scratching their own itch, and offering others to "come on over and have fun at our party"? It's fun and encouraging to hear recent popularity with some decentralized platforms. And my being a fan of such platforms does bias me; triggering delight in me when i hear such news. But I don't believe that the default (or only) goal of any decentralized social service is to amass the maximum number of users. Wouldn't that be more of a goal for a centralized silo, or a shiny, new startup? If i set up my own Gnu Social server/instance and my family, and i use it to post fun stuff for ourselves, maybe that's my only goal.
29 comments
[ 3.5 ms ] story [ 65.4 ms ] threadDecentralized technology is in fact trying to solve a serious problem, but sadly it is a problem that most people choose to ignore in favor of the incentives offered by Facebook and other monolithic centralized services.
However, Facebook et al. provide and maintain huge, single-point-of-entry, solely owned, walled gardens with hostile privacy policies and yet they successfully serve millions of users...
err... try billions of users
There's no new thing in technology, just variations on themes.
So people trying to solve these problems better focus on root causes rather than exciting technical problems which do not change the rules of the game anyway, unfortunately.
This is one of thr few essays about decentralization that understands that the problem isn't technical.
However, I'd go further than his explanations of incentives and say that the fundamental problem is that decentralized technical protocols do not solve the centralization of how money is spent.
Examples of that misunderstanding:
- SMTP the protocol is decentralized (technical) and yet we have giant email providers GMail/Hotmail/Yahoo which is centralized (money). The big providers spent $$$ on 1 gigabyte mail storage + backups + convenience. SMTP specifies how fields are laid out but it doesn't put money in everyone's bank account so they can run residential SMTP servers so the email ecosystem stays decentralized.
- Git the protocol is decentralized (technical) but Github the service is centralized (money). Why? Because Git the technical protocol is not a bank fund that gives every programmer a free $10 VPS account to host their own git repo. The centralization of money spent (Github invests in a datacenter but individual programmers do not) results in centralization.
- Bitcoin protocol is decentralized (technical) and yet the phenomenon of giant China "mining pools" emerges which is centralization (money). The ability to spend money on liquid cooled ASIC chips in a datacenter located near the Artic Circle is "centralized" to the entities that can spend that vast amount of money. The exceeds the ability for the home enthusiasts to compute hashes on a spare computer in their bedroom.
The common theme: technical protocols can be decentralized but the real-world implementation of those protocols end up centralized because physical things like cpus, harddrives, network bandwidth, etc cost money.
This pattern of decentralized technical protocols vs centralized economic behavior is ignored by virtually all decentralization enthusiasts.
So the real puzzle to decentralization is, "How do we _decentralize economic behavior_ when everybody doesn't have the same amount of money to spend?" Nobody I've read about so far has figured that out . That includes Sandstorm/IPFS/Filecoin/Mastadon/Diaspora/Ethereum etc.
My prediction would be that we would see this centralisation overturned at the same point that companies disband as people realise they can do just as well, self-employed. (Which is happening a little bit, actually.)
I agreed with you up until here. Not convinced that centralization is due to the economics of running a computer on a network. It's also a due to reputation and trust.
People use gmail and hotmail, not just because they don't want to pay for the computer, it's because they trust these companies will do it properly and reliably.
Github isn't the best example either. It didn't spring up because of the economics of running a git server. It got big largely because of its social features, and also because it was a pretty trustworthy git repository. I feel far more confident pushing to a git repo than to some physical server somewhere. There's a trust aspect.
I'll skip Bitcoin and got to another mostly decentralized system: paper currency. Exchanging paper currency is largely decentralized, but most people deposit it in a bank, in part due to interest, but largely due to the trust the bank provides.
The common theme to centralized services isn't the cost of computing hardware, but rather raising trust/reliability in the service, outsourcing expertise, and risk mitigation. Big central service providers are good at exactly this.
Yes I agree and I've written previously that non-physical costs such as "trust/reputation/skills" have to also be included:
https://news.ycombinator.com/item?id=14125730
https://news.ycombinator.com/item?id=10405945
Given that attention is limited, if there are some n services, any one individual's awareness of the ith service is likely 1/i. And the general population awareness will also fall with 1/i. If trust is a function (among other things) of familiarity, then there's simply limited trust in a universal-service market (that is: no subpopulation-specific grouping of service, e.g., by geography).
"Because Git the technical protocol is not a bank fund that gives every programmer a free $10 VPS account to host their own git repo."
No, but the protocol does ensure that every individual clone of a repository is a full mirror, and that everyone can work offline and synchronize via USB drives if they should need to.
I'm not sure that the kind of decentralization you're talking about is possible or even desirable.
Correct, but what I was addressing was the recurring sarcasm of "why did we settle for a centralized service like Github when Git itself is decentralized?" ... everytime Github has an outage.
Some people truly don't connect the dots as to why source code repo centralization exists despite the fact that Git is decentralized. To them, Github centralization seems like a weird irrational phenomenon. No, it's actually a quite natural and expected phenomenon!
In my view, decentralization is about building resilient systems where it's not a big problem if everyone uses the same provider, or the same three providers, because the content is de facto independent from the operators.
But it's also true that decentralized systems are under constant threat of recentralization because lock-in benefits monopolists...
And yes, mainly it has to do with the problem that tech people aiming for decentralization are yet to accept that this isn't a tech problem, but economic/incentive problem.
Ultimately though, it's a local optimum. Facebook acts in its own interest, not in the interests of users. The things that Facebook does to maximize its revenue cause great harm to its users. I'm not sure what's going to break the logjam, but something will.
Although, the open standardization and lack of lock-in that email provides is something to shoot for, and seems to be attainable.
But even if that were not true, Bittorrent is just a subcategory of filesharing in general, and it is abundantly obvious that the vast majority of filesharing occurs through centralized networks. (Google, Dropbox, Amazon, Facebook, Apple, etc)
So, in this case, Bittorrent is akin to these bespoke decentralized social networks that very few people use, and doesn't contradict the article at all.
Not an issue usually.
> it is abundantly obvious that the vast majority of filesharing occurs through centralized networks
Whoa whoa, that's a strong clause, I remember times when BitTorrent was responsible for 70% of traffic (period).
> bespoke decentralized social networks that very few people use
You're definitely living in a parallel reality.
But even pirated content had largely been migrated to centralized streaming services. Megaupload started that trend, although I'm not sure what the leaders are now.
When people search for pirated content, they use streaming. When people want to listen to a song, they use youtube or spotify. When people want porn, they use PornHub.
I use bittorrent for linux ISOs and such, but I certainly haven't used it for video content in years, and most people I know don't use it.
A few is a lot more than one, though. Even if 90% of email users are on gmail, you can still run your own sever and interoperate with them. Not so with closed networks like Facebook. A decentralized network with big players is a very different thing than a centralized network.
The public TV/radio is meant to help people with opinion making independent of the government and private corps.
A public corp could develop the software, promote it and inform people about the usage, but the software could run decentralized on home PCs or smartphones etc.
Virtually any device with an internet connection can be used as a server. So the solutions are already there, it's just that people trade freedom and independence of decentralized solutions for convenience of centralized apps.
A different perspective is that decentralized social services fail because of the lack of capital/revenue and sufficient management structure to compete with their centralized, commercial counterparts.
Money can fund a team of great designers, engineers, testers, marketers, sysadmins, writers, translators, product managers, to name a few. Together they can roll out and maintain a service that meets high standards of quality, uptime, usability, reliability, engagement, customer support... The things that actually matter to people.
Decentralized solutions really don't stand a chance against a well-funded company.
There've been a slew of attempts. A handful, and I mean that literally, hit the big time at any moment: Facebook, Instagram, Pinterest, VK, Twitter, Reddit.
(That handful's composition may change, but its count largely doesn't. Zipf's Law may hold a clue as to why.)
Even massively financed efforts fail: Google+. I'm one of the ghosts in the ghost town, and measured the activity to boot.
Services who've studied other failures ... fail. Imzy.
Services who've gotten big ... fail. MySpace.
Services who sell for near a billion dollars ... fail. Bebo.
Even services that ... sort of succeed, fail to generate sustainable revenues. Twitter and Reddit.
There are many problems, and initially, they tend overwhelmingly to be social -- who is on the network. Several of the noteable successes hit big, in particular Usenet, Slashdot, and Facebook, at least in their day. Draw the wrong initial crowd, and you'll be hampered by it forever.
There's a whole slew of other challenges: spam, abuse, asshats, network and system attacks, costs, UI/UX, performance, relevance, utility, and more. Whilst there are technical components, many of the elements are dominated by soft-skills.
Commercial and centralised systems have tended to be the more successful, through technical simplicity and funding, though there are noncommercial successes: Wikipedia and the Wikimedia foundation, MetaFilter.
And some interesting efforts at a slow boil: GnuSocial, Diaspora, Mastodon.
I'm not convinced decentralisation is the core problem.
Although, one has to wonder, do the folks leading the charge for these decentralized services really want each and every single person on facebook to over-night migrate to the new communities? Maybe; but i would gather maybe not. Maybe it started with some programmer simply scratching their own itch, and offering others to "come on over and have fun at our party"? It's fun and encouraging to hear recent popularity with some decentralized platforms. And my being a fan of such platforms does bias me; triggering delight in me when i hear such news. But I don't believe that the default (or only) goal of any decentralized social service is to amass the maximum number of users. Wouldn't that be more of a goal for a centralized silo, or a shiny, new startup? If i set up my own Gnu Social server/instance and my family, and i use it to post fun stuff for ourselves, maybe that's my only goal.