Ask HN: Have coders made stuff complicated on purpose?

YAGNI has an acronym for a reason. Engineering, and over-engineering, is a mentally stimulating activity for many.

To avoid making a complicated response, the answer is "Yes".

Modern day Kubernetes and React feels like a make work program

> is it some kinds of fallacy to believe that simplicity has a rather obvious form

There definitely aren't enough people deliberately obfuscating to make it a generally visible cause of complexity; your quote here rings far more true to me.

For example, "refactoring" as a task very frequently just trades one set of things, often aesthetic things, for another -- it is rare to find any obvious best expression.

I recently talked with a colleague about documentation. We both studied math and were frustrated about the lack of documentation from our predecessors. I discussed with them very often the topic but both we resisted. One was claiming he writes "self documentated code" and the other one was smart but wrote none approachable code.

I guess the difference that (in Germany) math exercises cares more about reasoning and explaining while in CS / informatics solution is only the tested result. So yes, coder make the world more complicated ;)

People unfortunately do this all the time for job security.

I think it's a fallacy to believe to that simplicity has a rather obvious form. From the view point of a SaaS developer, at some point you have to add a bunch of business logic that is determined by what's needed to satisfy requirements. Those requirements can be pretty complex.

Do you have an example of what you mean to make this a bit less abstract?

Some of it is deliberate. The quickest thing to hit the current deadline often involves leaving complexity behind.

Some of it is incompetence.

Occasionally there are claims that it's deliberate attempts to make the author look more valuable though I haven't seen that in practice. Hopefully it's a myth.

Some of it is indifference. You're sufficiently beaten down by the environment that you no longer care about the product.

Also the underlying problems are sometimes very complicated in which case there may be no way to make the solution simple.

There are two forces at odds with each other that are making things overly complicated.

1. A need for standards

2. The industry being too immature for standards to exist and those that are created rarely stand up to the test of time

Look at standards from the late 80s / early 90s, most of it was not prepared for what software has NEEDED to change to.

Look at most programming languages, designed to only run on one machine, when a vast amount of software is designed with high-availability in mind.

Look at most databases, designed to have only one writer which is a huge bottle neck.

Now we need software to coordinate all this software that was designed to run by itself...

Finally, in the pursuit of perfection, we test new ideas in the open, the latest UI framework, a new database, a new paradigm... We are doing the scientific method on NPM and your organization is a part of it :)

Look at how much small teams accomplished in the 80s and 90s. I don’t mean rockstar teams where you know the names of some of the developers on them, today—normal-ass boring line of business stuff, applications that you maybe know but with no famous names attached, that kind of thing.

Consider how much more efficient we should be with allegedly-better processes and allegedly much better tools.

Look at some of those and estimate the team size and time to complete it now. Consider whether you could even deliver the same thing (say, native applications on two or three platforms) with double the team size and thrice the time, or whether you’d have to compromise and deliver something worse to make that happen.

There’s your answer.

[edit] though actually I think part of this is also because we’re on-average worse at managing companies overall, too.

    We are slower because we do much more.

Yes, and, with a handful of them being mouthpieces backed by cloud providers, they've actually convinced most people that complexity is the way.

It’s hard to do anything on purpose. While certainly possible, people tend to spend time on what they view as actually important, and I suspect most people don’t view creating something intentionally overcomplex as being important. So I’d say it’s unlikely.

No I don't believe so. After a ridiculous amount of time thinking about this over the last couple of decades, I really think that it's just harder to see and think the simple, especially when corner cases start showing up. Combine this with the fact that we have (and continue to produce) abstraction layers over in order to "simplify" it to a manageable level, and you quickly get to high complexity even when you don't want to.

It takes more effort and time to produce simpler solutions, and generally speaking as an industry, we're way balacned toward speed over quality. We basically only care about "quality" as far as it has direct customer impact. It's relentless optimization for the short-term over the long-term, and it leads to complexity.

Embracing imperfection is part of having simpler systems. Even for the simplest of domains, a perfect solution will be complex.

I’ll put this link for the archivists.

“Hanlon’s Razor”

https://en.m.wikipedia.org/wiki/Hanlon's_razor

I don't think simplicity has a rather obvious form without experience. In my opinion, one needs to go through many years of experience to be able to predict when something is too simple for the purpose, and when it's not simple enough.

That gut feeling only comes through pain, aka experience, botching a refactoring you thought was brilliant to simplify code, and realise there were reasons for it to be overly complex. Experience working with overengineered code that takes a while to realise it's actually much simple than what the code is telling you, and can be thoroughly simplified.

Over many cycles of those experiences I believe one can start developing a sense of form for what's good simplicity.

Another point is it comes from the experience of reading vast amounts of code, from many different skill levels/sets.

Simplicity takes time and consideration from experience, I don't think you can jump the step of building experience to be proficient at it.

Typically what I’ve see most often is: considering coding as a satisfying hobby, combined with the tendency to generalize into a properly sorted ontology, which leads to prioritizing the pleasingness of the code rather than the fitness of it to the current purpose. This is excellent, if the coding is in fact for a hobby, but it is often misapplied in other contexts. https://xkcd.com/974/

“Simplicity” can also be defined any number of ways, but is most often defined as “Maximally pleasing simplicity” rather than “Fewest moving parts simplicity”, in part due to misestimating how total volume of moving parts is an exponential complexity modifier when one is not already familiar with the mechanisms. https://xkcd.com/2501/

I have personally been in teams where, when I questioned whether or not we really needed Shiny New Thing, I got a ton of pushback, most of it not very persuasive. I have even heard programmers criticize the state of the code when they joined the company, saying "there just wasn't much code...".

You get what you reward. Our industry rewards more keywords in your CV. Therefore, people try to get as many keywords as they can.

Complexity stems from ignorance.

if someone isn't knowledgeable about the problem they're not going to be able to select the most simple solution.

if someone isn't knowledgeable about the available solutions there's a higher chance they'll implement an unnecessarily complex solution.

now multiply this over time and you get a code base that is really good for everyone's job security.

multiply this by millions of code bases and you get a really good tech economy.

More legacy code

I think complexity slowly developed over time to address certain needs of the time. Partly because software has been needing to do more. People and institution have become completely dependent on ease-of-use. When that happens software needs to be coded to be dummy proof and to have a large feature set. The second part is time. Programmers have need to find ways to decrease the amount of time programming takes to have free time to do other stuff, that causes programmers to rely on frameworks that are already established and creates a new market for coders to want to save time. Frameworks need to be complex to address a myriad of needs.

It comes down to the power of incentives. As a developer, there is no incentive to make code simple. It's subconscious. Coming up with near-optimal solutions that are more maintainable in the long run takes time and effort. Most developers will happily sacrifice long-term maintainability to deliver fast short term results... Anyway, if the developer churns out code at an impressive rate for a few months, they'll probably be promoted to management and someone else will have to take over the mess they created.

Depends upon what system you work on. I always worked on "large" systems that do not have a GUI type environment.

But today's "get it done now" outlook encourages complexity. That was always around, but ....

About 20 years ago or so, it was OK to get things done right and be a little late. Now seems this "get it in now, do worry about consequences", then "Well we have issues, create a request and will will get to it later". But later hardly ever comes, so the users are stuck.

So here we are, everything is complex, no time to think about what you are doing.

Unix had the KISS principle for a reason -- people weren't "keeping it simple, stupid".

In the '10s, everyone was talking about "webscale". VCs noticed that we could write software for hundreds of thousands of occasional users instead of a core group of a few thousand users.

We thought we were making UIs to help people do stuff, but we were really making little feeler apps for vertical integration for corporate interests.

Stuff like big-data and kubernetes looks overcomplicated if you believe the sales pitch. The "tech set" is negotiating for control over business.