74 comments

[ 2.3 ms ] story [ 175 ms ] thread
In some way, neither one project fails entirely - as it serves to distributes wealth from those that have too much to those that do not have enough.

In some sense it would be even better for society if those who have lot of wealth risk even more into wild ideas and moonshot projects.

But that is a broken window fallacy. It would be better for everyone if those who have a lot of wealth invest in useful projects.
What is definition of useful? Many successful projects are not necessary useful, but still they create wealth, and many useful project never succeeded to make money to become sustainable...

For example many paintings have huge price tag, and we could live without them. Many CO2 removal could devices are very useful but not financially sustainable ...

Let's skip "useful" and just go with projects that earn money for the person that invested in it. Then that person earns more and can hire 2 more devs or 3. When project fails devs have to search for new jobs and investor is probably not going to spend his money as easy again.
I think this is a parody?
Some may misinterpret this article as satire. Surely it is not really desirable for software projects to fail. But the facts speak for themselves.

Quite fun to read regardless!

No. I think he's serious. The same author also advocates that the manufacturing and distribution of high-performance processors should be outlawed and regulated as nuclear weapons in order to save humanity from a potential AI singularity [0]. Also, since the computing power we currently have may already be sufficient to run an AGI via distributed computing, destroying existing computers may be necessary. Finally, since he concluded that software projects don't really improve social productivity (in this article), that (high-performance) computers isn't a good idea to humanity in general, therefore, reverting back to the early 90s and asking everyone to use 10 MHz computers is an acceptable solution - you also solve the problem of software bloat simultaneously. In addition, he also suggested that patent trolls might actually be the saviors since they slow down the development of computing and AI.

Later he extended these ideas to an entire book [1].

The author's ideas are highly unconventional, but I won't say it's unreasonable or even uncommon in tech - Bill Joy's Why The Future Doesn't Need Us shares similar concerns, the author simply took a much stronger position.

[0] https://berglas.org/Articles/AIKillGrandchildren/AIKillGrand...

[1] http://computersthink.com/

I saw it more as a challenge. As with zeno’s paradox the reader is challenged to find the flaw in the logic.

Maybe they are right. I often wonder about the point of building another CRUD form. We’ve taken paper forms and digitized the process, but haven’t really figured out how to avoid needing the form in the first place. Massive digital bureaucracies exist that are all about pushing around digital paperwork. It’s as if there was a twist of history that caused the development of mechanical horses pulling rubber tired carriages, instead of just inventing cars.

> but haven’t really figured out how to avoid needing the form in the first place.

Most forms get digitized, then reworked later. Many digital agencies try to think beyond just forms and try to create smarter ways of getting data.

Naturally this is done for clients that spend ludicrous amounts of money to get there. So you'll never see them in governments.

> In this way the heavy burden of guilt can be lifted from the shoulders of the numerous project managers that have subconsciously devoted their careers to ensuring that projects rarely, if ever, succeed.

Was totally thinking the same thing when I read this article. How could the above line be written seriously - PMs don't have guilt about destroying projects!

For a recent confirmation of this, witness the enormous amount of effort expended on GPDR compliance.

None of it could have been done before computing power became so cheap that it needed to be wasted on doing it.

Everyone complains about useless procedures, but I think the real culprit is that we have no framework for discussing value. By that I mean the fundamental economic problem of what should we do, who should do what, how should stuff be shared?

In effect our current idea of value is a sort of unstated default: whatever you can get someone to do is valuable, unless it's illegal. And what becomes illegal is also governed by never-stated assumptions.

So we end up with this society where anything that can take up productive capacity is done. There's always some sort of justification that might sound reasonable in some context - privacy of personal data is indeed important - but we end up not balancing the costs with the benefits, and you end up with absurd situations where schools can't send out mail to their alumni (yes, this actually happened to me).

The reason the GDPR and other privacy legislation was introduced is because of the widespread abuse of people's private data for commercial and political gain.

Attempting to address this isn't a waste of time or useless.

It makes no sense to blame the people trying to solve this problem, rather than those who created it in the first place.

If anything, the GDPR didn't go far enough. Though, that's often the case with legislation trying to stop corporate malfeasance.

In my opinion - and speaking more generally, not just regarding privacy - not enough large companies have been obliterated for their harmful behaviours. They just pay a fine, or bribe some officials, and move on.

Attempting to address this isn't a waste of time or useless.

It is if you fail.

It makes no sense to blame the people trying to solve this problem, rather than those who created it in the first place.

Now we have two problems: the original problem that GDPR failed to solve and GDPR useless warnings.

It hasn't failed. There's a lot more to the GDPR than the warnings you see on some websites - many of which likely don't constitute proper informed consent anyway and so would be unlawful.
When I criticize GDPR is exactly for the warnings you see on most websites. They're a nuisance and don't prevent data collection. I need to use technical measures to have some privacy and now I also need to use additional technical measures to avoid the crappy warnings.

You say warnings are unlawful, and that's even worse, because that means they've created an unenforceable law. Or what are they waiting to prosecute those violations?

I'm not against data protection. I'm again incompetence.

It makes companies really think: Are we allowed to have this data? Do we need this data? How do we handle this data? Can we delete someone's data? And so on.

The consent forms are just a tip of iceberg and the stupid thing. But under everything it forces companies to do things they might have not done before. And it's also a tool for developers who want things to be better. As there is finally slight risk to point out.

> If anything, the GDPR didn't go far enough.

Exactly. If it went far enough the solution could have been much simpler. But instead of the real solution, politicians preferred the appearance of the solution which created enormous complexity and didn't solve the problem.

what would the solution you wanted look like?
We do have that framework, it's called supply and demand. Collectively, we buy what we need the most, at the best price. We sell what we can offer, at the best price. Value is determined in this way. Prices encompass not just currency, but also preferences.

Bureaucracies grow because people demand them to grow, it's just not that obvious. Something somewhere is always wrong and unjust, so people petition politicians for change. The tool of a politician is lawmaking, so more laws are created. Implementing a law requires a bureaucracy. A bureaucracy requires managers. Managers want to hire more subordinates, not less. Subordinates are paid by the hour and promoted according to plan, so there is no incentive to work faster. In effect, everyone involved is always busy with something and nobody appears lazy or irresponsible.

I would argue that the tax system of (in this example) Australia is exactly what the Australians asked for, the system that they deserve. They just don't know they asked for it.

What you have written is in fact a circular argument that confirms what I wrote about value. There is no extrinsic value in the economically orthodox description of value that you describe in the first line.

So when the Aussies get their tax system, they must have asked for it. Why? Because they got it.

This is an absurdity. Have you ever complained about the proliferation of utterly terrible reality TV shows? Very few people think it's anything but mindless garbage. But it's there, so it must be wanted, because it's there.

Plenty of other examples in society of things that exist but nobody appreciates them.

Anyway, been reading Mazzucato recently.

> What you have written is in fact a circular argument that confirms what I wrote about value.

It is not a circular argument, it is a connection between cause and distal effect.

> There is no extrinsic value in the economically orthodox description of value that you describe in the first line.

I'm not sure what to make of this statement.

> So when the Aussies get their tax system, they must have asked for it. Why? Because they got it.

This is a misrepresentation. By demanding a more just system of taxation, they asked for laws, which ask for bureaucracy, which ask for labor resources, which consume tax revenue. All of these are logical implications, there is no circularity at all.

Similarly, if I ask you to serve me a hamburger, there's the implication that someone somewhere has slaughtered an innocent cow. I don't want the innocent cow slaughtered, I just want to eat the hamburger. Yet, it would be fair to say that I asked for the cow to be slaughtered.

> This is an absurdity.

Not at all. Usually there's a plausible rational explanation for why things had to end up the way they did, but such explanations might be uncomfortable.

> Have you ever complained about the proliferation of utterly terrible reality TV shows?

I don't watch TV.

> Very few people think it's anything but mindless garbage. But it's there, so it must be wanted, because it's there.

Very few people think McDonald's serves good food, yet a lot of people buy it. Similarly, nobody would continue to spend money to produce more and more TV shows that nobody wants to watch. Being "mindless garbage" clearly isn't an impediment to success with the right audience. Consider the pareto principle.

> Plenty of other examples in society of things that exist but nobody appreciates them.

Just because it isn't "appreciated" doesn't mean there isn't demand for it.

> Anyway, been reading Mazzucato recently.

Doesn't seem to have done you any good.

> Similarly, if I ask you to serve me a hamburger, there's the implication that someone somewhere has slaughtered an innocent cow. I don't want the innocent cow slaughtered, I just want to eat the hamburger. Yet, it would be fair to say that I asked for the cow to be slaughtered.

I really wish there was a 1-3 word phrase for this concept, because I keep needing one. I am hopeful that the field of AI Safety will give us one.

This just confirms what I said above. The cognitive dissonance of not having a real theory is causing you to lash out at me, for no reason other than pointing out your missing theory.

It's pretty simple really, if there's a way to determine value then there must be a way to say that it's correct in one way or another. If things are always just worth whatever someone will pay for them "because of the market" then you aren't saying anything at all. Plus you can't explain a number of common observations.

> This just confirms what I said above.

Why? Because you said so?

> The cognitive dissonance of not having a real theory is causing you to lash out at me, for no reason other than pointing out your missing theory.

I wasn't aware that you're a mind reader.

> It's pretty simple really, if there's a way to determine value then there must be a way to say that it's correct in one way or another.

There's a shortcut: I simply define value in terms of supply and demand, the exchange of goods and services, based on subjective disposition.

If you are clamoring for some other theory of value, feel free to do so. I think that's a waste of time.

> If things are always just worth whatever someone will pay for them "because of the market" then you aren't saying anything at all.

If you believe that, then any further discussion must be fruitless.

> Plus you can't explain a number of common observations.

I probably can, but you won't like the explanation.

> For a recent confirmation of this, witness the enormous amount of effort expended on GPDR compliance. None of it could have been done before computing power became so cheap that it needed to be wasted on doing it.

Everyone complains about useless procedures, but I think the real culprit is that we have no framework for discussing value.

We do - money. That's why GDPR breaches cost lots of money - to make sure organisations don't allow breaches to happen.

The GDPR does not limit the possibility of schools sending out emails to their alumni, it just imposes requirements for doing it right. The GDPR recognizes the risk of the school or someone with access to the school's email db spamming, selling the addresses or harassing students / alumni, and the risk of someone hacking the school servers and stealing the data or targeting students. These are not theoretical risks... they are things that happen, and become more frequent as more of our lives move online and our emails get tied to critical processes.
This is of course absolutely true, and I did say that there is some merit to the goals of GDPR. It is merely that in adding this extra work, little is gained. Let me expand a bit, it was one-liner at the end.

My school in fact had my contact details from way before GDPR was a thing. Recently, they were forced to write to confirm that they were allowed to continue contacting me. If I had not responded, they would be compelled to delete my information.

So if I'd missed the message, I would unknowingly be disconnected from further communications.

And what is to be gained from this? All the risks of someone hacking the school system are the same, someone could still spam everyone, and so on. Why not just have an opt-out system, defaulting to stay-on-the-list? Then people who care to be removed can do so, but we can assume quite confidently that most alumni of a school want to remain in contact.

It just smells like someone had a use case they were thinking of that perhaps made sense to have a default-remove, but they ended up imposing it on everyone.

The title is missing "(2008)"
> As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function effectively.

Not only failed projects can prevent bureaucratic complexity, they can also save lives

> automated system assigns a unique identifying code to each incoming call before passing it on — a method of keeping track of phone calls as they move through the system. But on April 9, the software responsible for assigning the codes maxed out at a pre-set limit; the counter literally stopped counting at 40 million calls. As a result, the routing system stopped accepting new calls, leading to a bottleneck and a series of cascading failures elsewhere in the 911 infrastructure.

Had that automated system failed at the development phase...

https://www.washingtonpost.com/news/the-switch/wp/2014/10/20...

You can blame the software, but you can also blame the organization for not having a backup system.
> Any backup 911 call centers that could have helped with the bottleneck never did, because they themselves were suffering from the same problems everyone else was, according to the FCC.

Go figure.

They mean a backup way to take phone calls, not backup 911 centers.
Wow, it sounds like an apocalyptic story that comp.risks has warned in the 1990s. (From those who don't know: comp.risks was a well-known Usenet forum on the systematic risks of computing technologies in society, and they publish a newsletter since the mid-80s.)
I don't know, the contents of this article sound really inaccurate. There's no software counter that idiosyncratically stops at 40 million - maybe 4 billion was the real answer, and the reporter misread the digits.

This sounds like a 32-bit TXID exhaustion bug in a Postgres database, which is a notorious and very difficult to recover from problem in live systems. It is probably the #1 arcane Postgres cataclysm. It would be consistent with how this is a "single coding error" (i.e., the call to VACUUM), additionally why it was so difficult to recover from ad-hoc. I'm generously assuming that everything else went well, that people who are building 911 systems aspire to do a really good job, and that it didn't fail anyone how important their work was.

Which is funny, I mean what if Postgres "failed" for every TXID exhaustion cataclysm it had. Arguably the best database product in the world "failed" because of this. It probably provides more than a few billion dollars to our tech economy, on top of being the hub for research in databases - research whose goal is the same as the one as this paper, to use automation to reduce bureaucracy and improve society.

Anyway we always imagine these rinky-dink consultant shops as writing all this custom stuff, and in actuality they are almost always combining a bunch open source and off the shelf things - which I would like to not see fail.

Numeric limits like this are far more common than you might think.

Besides the well known Y2K mitigation, there were some problems on day 10,000 when four characters to represent days since the epoch overflowed, mid 90s for Unix derived systems and I believe VMS as well. None were civilization threatening.

We also had Second One Billion in early 2000s. Also no public catastrophes.

Next big one is 2038, when 32-bit Unix timekeeping overflows.

> We also had Second One Billion in early 2000s. Also no public catastrophes.

Thanks for the reminder! We had a small fun party to celebrate!

I find myself fighting this fight all the time - we introduce a new tool, and all of a sudden a dozen processes (bureaucracy in a nutshell) spring up from its outputs or to control its inputs.

Unnecessary processes should be considered like unnecessary code - maintenance overhead - and so the usefulness of any process should with some frequency be questioned.

Interesting as food for though. Completely absurd in reality.

It takes me 1 minute to review, all data is already filled in, my yearly income tax (not much ongoing a part from my salary). And, two clicks in my phone approve, sign and send the declaration to the tax agency. No writing in paper, no looking for numbers, no finding old mail with my payslips. I would avoid living in any country that makes it more complicated that this.

If your public administration software sucks, you are voting the wrong people into public office. Do not blame the software.

Interesting. What country is that?
It works this way in Sweden. Not sure which country he was referring to.
And Norway, except that if you do not need to make any changes you do not need to do anything at all.
My taxes in Spain were pretty much like that. You get a letter with the government's estimate. If you agree, you don't have to do anything else.
Could be Australia? Looking at others commenting though could be a lot of other countries.

The USAs tax system where the online portion was essentially privatized by a single company (Turbotax) was pretty jarring.

Several European countries. In the Netherlands taxes are just logging in to a web interface and verifying the provided data.

It's usually filed correctly already. So many times you'll just be clicking next.

Works the same way in France, too, for regular salaried income.

If you have other income, like dividends, there are a few more boxes to fill, but it's still fairly simple.

Once you start having serious money, you better have a tax lawyer doing this for you, but I suppose this is the case everywhere.

Not the USA for sure. One reason taxes are complicated and full of loopholes is to allow rich people to game their taxes and pay next to nothing. They have enough power and influence and their incentive is to keep things as they are: complicated.
if only the "right" people would run for office...
Where I live, it is possible to elect people into public office that weren't officially running, at least on a local level. There was a prominent example in a nearby town (not a big town, around 50,000 inhabitants) a few years ago in which only the current mayor and some dubios candidate ran for mayor. At the end, a prominent local opposition figure was elected because thousands of people wrote his name + address on the ballot. He accepted and is now mayor.
Most politicians are not around long enough to have to deal with the consequences of their policies. So, they tend to be focused on short term wins. The rare good ones combine that with long term vision that actually makes sense. Often that's not even recognized until decades later.

Four years, maybe eight in some cases. Not much more. That's all they have to prove their worth. Most complex projects easily take longer than that to plan and execute. Civil servants on the other are typically there until they retire. In many countries civil servants enjoy special job protections that ensure it is typically hard to fire them. And since they are large organizations, there's a bit of a rat-race to get promoted to the lucrative top positions. That requires pleasing the people holding the purse strings. Politicians in other words. And politicians need to get stuff done and that requires warm relations with the established civil servant leadership because otherwise nothing gets done and they lose the next elections.

This adds to the problem because politicians basically rely on an already compromised bureaucratic machinery that resists changes to make it smaller. So any new policy just translates into more budget, more people, and more bureaucracy. The old ones never go away.

You see the same pattern of behavior in large corporations except those periodically get forced by their shareholders and the market to do something about it. Sometimes that fails and they get acquired, lose market-share, or even just implode and disappear. If you look at the fortune 500 through the years, you'll see there's a lot of churn there. New companies take up spots vacated by older failing corporations. But pretty much all of them employ way more people than the competitors that ultimately displace them.

Banks were mentioned in the article a few times. Most banks have actually been reducing head count for years. It used to be that they had to have offices in every small city, village, and even neighborhoods. Not any more. Those days are long gone. There are now several startups disrupting the space with online only banks. Any bank looking to survive in that space is pretty much copying that organizational model at this point. And as consumer banking is a this point a low margin product, a lot of the older and bloated banks are clearly struggling.

>If your public administration software sucks, you are voting the wrong people into public office.

1. It's very difficult for voters to know how much a given politician will prioritize digital services.

2. Even if a politician appears to have a good digital strategy, will they implement it well?

3. Most decisions about public admin software are made by bureaucrats who aren't elected anyways.

4. Most elections will likely have more important issues than public admin software. Although I agree it probably does have huge returns on investment.

> 1. It's very difficult for voters to know how much a given politician will prioritize digital services.

Sort of but not really. Just keep voting in progressives that believe in government until the bureaucracy gets the message that it pays to be progressive and fix things.

I think the point being made is that public admin software has to be complicated because the underlying laws, policies, regulations, and procedures are complicated. Sometimes, they are over-complicated. And, sometimes this is deliberate (e.g., if applying for benefits is too complicated, fewer people will complete the process).

If we elect politicians in favor of simplifying government operations, and they can actually succeed, then it follows that government software would naturally simplify as well.

Note: This isn't necessarily the same as reducing government's role.

> more important issues than public admin software

In my experience, there is a correlation between parties that follow the science, invest in education, and create useful software.

My critique is towards people that gives up and says that governments cannot make good software. And, that is false. It will never be perfect, like anything else, but it can be good.

> It will never be perfect, like anything else, but it can be good.

As a US -> UK immigrant:

1. This is true.

2. This is made harder or easier by the complexity of the underlying abstractions it is trying to model.

I suspect the software that the NHS has to calculate how much to charge medical patients is very simple.

(comment deleted)
This reminds me of the time that I tried to remove a little used feature of a product that I worked on. I could demonstrate that a total of four people had used the feature infrequently in the last year. It took forever to get the approval to remove it. I think that may also contribute to bureaucracy size. It’s next to impossible to remove something from the system as soon as it gets introduced. In other words, bureaucracy is monotonically increasing.
> As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function effectively.

This is a short-sighted and overly pessimistic view of the role of software. Let's bring in some sorely needed optimism that's especially relevant to the current pandemic:

https://blog.cleancoder.com/uncle-bob/2020/04/05/ANewHope.ht...

Another area where software is actually good for the world is accessibility. Lots of people with disabilities, including me and several of my friends, are able to live independent and happy lives largely thanks to software such as screen readers, speech synthesizers (useful for both blind people and people who can't speak), optical character recognition, and speech recognition. Not to mention that many products and services that are merely convenient for able-bodied people are life-changing for us.

So I feel bad for people who are so caught up in the world of software that merely increases bureaucratic complexity, that they've lost sight of all of the good that software is doing, or at least don't get to contribute to that kind of software.

Yes, I think that the highlighted statement is overly hyperbolic. A better one would be something like: "A major effect of software is to facilitate bureaucratic complexity, therefore it is beneficial for some projects to fail."

This revised statement would then lead to an interesting discussion about which software projects are beneficial to society and which should be allowed to fail. I would agree that a lot of enterprise software tend to increase workflow complexity. I think that's because in a corporate setting, the people with purchasing powers often don't understand how software technologies works, and they aren't the ones using the system.

I agree, the accessibility point is important precisely because a lot of the growth of these bureaucracies is to accommodate more people. Being able to identify why these bureaucracies are growing (hint: it's not just because they can) allows us to better solve the problem, which is not software getting better (imo).

An old(er) post that is vaguely relevant: https://news.ycombinator.com/item?id=21848860 An interesting look at the end of what makes projects slow down, and possibly insights into why bureaucracies grow.

If you could guarantee every individual in a system would be competent enough to be afforded a greater degree of independence, we would not need bureaucratic complexity. Unfortunately as the number of individuals grows larger, you cannot. Bureaucracy provides resilience against incompetence.
(comment deleted)
> Well, there was one group of people who don’t often get mentioned; but without them the Pandemic would have been a hundred times worse than it was.

> Really? Who?

> The programmers.

Wow. This is the pinnacle of narcissism.

How about another group of people who don't get mentioned as often, say Chinese workers who slave away at foxconn factories.

Maybe this piece wouldn't read as bad if it wasn't written by a programmer.

> As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function effectively.

What about to play Go, to dispatch Uber drivers, to translate languages, to convert natural language queries into web searches?

Red flags: Author has already claimed to be "bold" by the third word of the abstract. Author includes an abstract on a two page self-published paper where they further propose their own name to decorate a newly discovered economic "law". Author uses phrases "statistically significant" and "empirical proof" without betraying any familiarity with their normal meaning in formal prose.

And, most importantly, author apparently believes that the observation that efficiencies introduced by new processes are invariably erased by increases in associated bureaucracy was novel and useful in 2008. People have been complaining about that since at least the Roman Republic.

For better reading experience, add this to body in console.log:

font-family: sans-serif; width: 80%; margin: auto; color: #1A202C; background-color: #F7FAFC; line-height: 1.5rem;

Is there a Chrome extension that can just do this sort of thing automatically for content that's linked to from forums like HN?
Not that I know of. Wouldn't be hard to build but I don't know how well this generalizes.
Amazing write-up, some really good kicks in there. The bloat in bureaucracy can be seen in the software itself too. Much of today's consumer software fundamentally performs the same function it did 20 years ago, yet it suffers from the same kind and number of bugs, runs slower, and uses more compute resources. Whatever advances have been in hardware are many times counteracted by the bloat of software. "Below 1% is easily affordable, so the bureaucracy will naturally grow beyond that size as predicted by Parkinson's law" applies to the software itself as much as it does to the bureaucracy it enables.
Satire is at its best when it doesn't obviously look like one.
It's mostly a legal form of money laundering stealing tax payer funds. The Fed prints millions of dollars of fiat currency out of thin air, the government creates some bonds out of thin air and sells them to the Fed for the money they've just printed, then they give it to a big corporation via huge government contracts... The executives then use the free government money to buyback their company shares, which drives up stock prices, which leads to them getting huge bonuses... Then the project fails, the executives take the blame, get a golden handshake worth millions... Then the government will award equally large software contracts to that same company again in the future and repeat.