Ask HN: Why did back-end development explode in complexity?
Follow up question: is that complexity necessary or artificially inflated?
Note: this is a fun jab back at https://news.ycombinator.com/item?id=34218003
Frontend feels like it gets incredible amounts of uninformed hatred here, and you could copy/paste many of the defensive replies here over there verbatim. My take is backend isn’t too complex, just sometimes over-complicated in specific cases. Just like frontend! I just wish there weren’t so many non-frontend developers in every frontend thread hating on it while obviously not being very experienced with it!
Happy new year!
77 comments
[ 2.7 ms ] story [ 64.5 ms ] thread> "Why can't you do it?"
pattern i saw is teams do not build customer feature but work on “efficiency” or “cost saving” project. Ok, so you build something that other teams should use because it will save them time. Sounds very good on paper.
so you do these things to entrench your team DEEP into the tech stack. now people depend on you you use it as reason to get even more head count. ok now the engineers got their promotion but they struggle to support these teams when things not working. So they leave. they got promo, why stay?
managers now use all this ammo to ask for even more hc. it is self serving cycle. Now..development harder. harder to push new features. new people from college come. first they have imposter syndrome and think this is all great and they just not smart to get it. within few year they get it..see processes and tools are idiotic. they learn to play the same game. circle of life continue
Try finding a starter kit for that.
I think Devops has a played a huge role in why things have gotten more complicated. They want Kubernetes to be the solution to all problems. Heroku has fallen out of fashion for literally no reason. They won't even consider ElasticBeanstalk or App runner for even the smallest of projects. Companies would rather hire a 200k plus benefits person instead of paying a few hundred dollars more a month on hosted solutions. I've honestly never seen anything like it.
IMO, Kubernetes solves a lot of issues for both camps but most of the Kubernetes detractors I've met are only hostile to it because of ignorance, not any reasonable criticisms. "It's different, it's hard or it's something I don't know" aren't valid excuses.
Things were never a problem when devs just managed this stuff. We used Heroku or Capistrano on plain EC2 and it was fine. Once you hit scale and things become more expensive then Docker on ECR / Fargate is fine. This whole dev ops fad just has to go.
So for every story I hear about "it was better back then", I just remember how much it was a horror for us and find it really hard to believe people voicing these options are actually arguing in good faith.
My experience is: devops is introduced to 2 orgs I was at (much smaller dev teams) to save money. Even though hiring devops negates any savings, but that's another story. They introduce Kubernetes (even though existing solutions were fine) as the flavor of the month to build up their resumes. They all leave after 6 months to a year and dev teams are stuck with the cleanup trying support and be in call for things we had no say in. It's harder to see logs, get a rails console, deploy, debug, etc.
Meanwhile they also spent 6 months trying to get PR apps and staging environments at one company. Something that is one click on Heroku.
If we didn't have Heroku at these companies when we started out we would have literally failed. End of story.
This isn't the future I wanted, let's solve these problems in some other way. K8s seems to be the second-system effect in all the worst ways.
But this Java shit, that has to all come from enterprise solutions right? I can’t imagine there were tons of influencer-like blog posts that JavaScript had. What hell hole did those Java ideas even come from?
Blah. One way or another, people manage to fuck everything up. We just can’t have nice things.
Like, if you want to build your résumé and get offered some of the cushier programming-adjacent gigs and get to present at prominent conferences and generally become A Name, how likely is that to happen if you try to launch some big web framework, or an auth library, or an ORM—in Ruby? Or Java? Nah, too crowded.
But if you pick some up-and-coming language (that's up and coming in part because everyone involved is doing exactly what you're trying to do) or hook your wagon to some new framework's (or database's, et c.) hype cycle, well, now you're talking. As a bonus, many of these projects require little or no novelty—it's all well-trod territory. Just half-competently do what hundreds of others have already done, succeed at promoting it (tellingly, this is the harder part) and you're a rockstar. And hell, maybe doing all that does demonstrate some kind of remarkable and valuable skill, for all I know!
But the point is, if that's the kind of thing you're trying to do, you need churn, and a bunch of people all trying to accomplish the above drives that very churn. Very little of that activity actually advances the industry, and the churn itself is a huge drag on everything.
If you take all the scary Java Enterprise buzzwords in one of the previous posts on this thread, and look them up, you'll find that they all came about, roughly, between 1999 and 2007.
I don't fully remember what all the hip influencer tech blogs were during that era (I recall "Joel on Software" having been important), but I do remember that people who paid more attention to such things were really into "Enterprise Java Beans" at the time.
So the JVM world might suffer from a bit of an analogous problem. But that world's complexity explosion was decades ago.
My feeling is that backend has always been complex, depending on the scale you were trying to operate at (and how much scale your framework and ecosystem forced you to think about whether you wanted to or not). The best you can do to mitigate it is continually ask "Do I really need my backend to handle the scale I'm designing it for?" and "Is all of the complexity in my preferred stack still necessary, or can some of it be hidden/automated?"
When are we ever going to get hip to that reality?
And that's the outcome for the successful stuff!
(Doesn't mean you shouldn't get into it though -- if I'd jumped on enterprise java 20 years ago, I'd probably be richer than I ended up by just working on the software I was interested in)
We were also trying really cool things with many of these java efforts. Rather than a main() routine that imperatively did everything, there were future-sightes notions that we could have a bunch of ambient systems available & interconnected. OSGI solved some thorny library versioning problems, and as a bonus allowed for a very strongly manageable runtime for modules. EJB brought objects up to a higher level, a greater management tier with more interoperability of objects across instances. We were interested in trying more, in tackling harder problems, that microservice era has masked over, deliberately chosen to ignore.
Over time we got more refined forms of many of these systems. Maven calmed much of the ad-hoc madness of Ant. CDI was a more reasonable EJB with event more competent management. JSP kept getting better component libs. OSGi actually was... decent all along, sometimes legit useful.
But the idea that Java was absurdly troubled seems to endure & be enormously popular hatred/FUD to spread. The fads of today are exactly the opposite of the better managed more flexible runtime trend, where the JVM keeps rising as a more capable, flexible, malleable, adaptable, controllable way to tackle processes than the processes at an OS level. And certainly, there's a lot of ease we've bought ourselves by doing much much less. Statically compiled binaries are an example of yet another means of doing less, of finding easy routes; powerful simplifications that have greatly eased many folk's lives.
Less is great, sure. Isolated container instances with dedicated apps is pretty easy to deal with. But there are still constant undercurrents, pushing us towards more interesting complected runtime arraingements, where virtual machines (not fully virtualized os'es but systems like glassfish or erlang, where many sub-processes are co-resident) come back. The idea of very quick very easy to spawn sub-processes allows for interesting security & modularity, where-as today services are almost all monolithic app servers that have complicated multi-user authentication & authorization concerns. We could make simpler safer software if we had more complex runtimes; it's a trade-off, and im deciding we have gone too-deep, unleashed monsters, we have perhaps too polarized ourselves against many of the great capabilities we one tried for.
I also used both DCOM and CORBA at later jobs for Soft-Realtime/Embedded distributed systems.
I’d say statically typed IDLs are much better than all these REST/HTTP/JSON/CRUD approach. That’s why I prefer GraphQL and gRPC/ProtoBuffers.
Not the least important thing is that having a statically-typed and well documented schema reduce the amount of required communication between the developers, and also reduces the number of misunderstandings, mistakes, and bugs.
That was some convoluted shit. Class path errors, coupled to blueprint XML and fancy IDE stuff.
I’ve worked in different languages and frameworks since and love the simplicity of Go. Excited to learn Rust
People literally use hundreds of different tools for no reason at all.
Slightly Disagree. Resume driven development (RDD) is a thing.
Now, I can write a BE, configure a Dockerfile, and deploy to a server nearly anywhere in the world without leaving my chair.
I don't know how decision makers think, but I get a general sense that Oracle is the safe "you can't be fired for choosing..." choice.
I don't think these places have just one server, though, far from it.
When I started out decades ago, systems were far less complex.
Note, the IRS is our best friend. Without them, most of us would be doing something else (if we are very lucky).
I strongly believe that we're now at the limit of human cognitive ability (as evidenced by the tight, and tightening, employment market) and we human programmers are painting ourselves into a complexity corner and we will weep when our employers happily hire the only minds capable of dealing with mechanical complexity, artificial ones.
I say this, half tongue-in-cheek.
I've met many full stack devs who are comfortable from DB (db design and stored procedures, sql) - backend code (java, spring framework) backend services (consuming first party and third party APIs) - frontend code (javascript, typescript, jquery, angular, react) - frontend services (Google analytics, GTM, etc. )
Edit: forgot to add in CI, CD, deployment platform skills, server less functions etc
That is not full-stack, that is knowing a syntax that happens to have ended at both ends of the stack.
Some folks will blame "the system" or "the man", others will claim it's not hard and you just need to RTFM, still others will explain how much harder it was "back in their day", and some folks will question the question (like me).
But nobody is going to drop some single piece of information or knowledge that'll "answer" the question; "In 2004 Linus Torvalds declared that all backend systems must contain three additional technologies than the currently mandated two." If only it were that simple.
Things are hard until they aren't, complex until they're simple, convoluted until they're straightforward as can be. The line is different for everyone, so nobody knows what your line is, which means nobody's going to answer the question in a way that will satisfy you, and if they do, it'll only be helpful to a few folks (your folks) and possibly actively harmful to others.
Maybe if you had a specific question that illuminated where you're coming from, folks could provide better answers...
I tend to agree with you; stuff that felt impossible for me to understand at one point is now second nature to me, and over time I've learned how to do more. UI and BE, as well as orchestration and infra.
AWS used to scared me, and now I maintain a dozen microservices (and I may slightly regret breaking everything up lmao) for my startup in AWS by myself because Terraform is pretty easy to pick up and at this point AWS makes a lot of sense (hello Dunning-Kruger, my oldest colleague).
Frontend terrified me because I didn't know how to compose UI cleanly (not just JS), but then I learned React and now feel confident I can build, maintain, and deploy complex frontends because honestly it wasn't that hard to grok.
The single strongest piece of advice I've been given, the single idea that gets me through like... 99% of work related strife is this:
How you feel now is not permanent and will not last forever.
I guess the more poetic way people say this is, "This too shall pass"[0] but I just think of Gandalf so...
[0] https://en.wikipedia.org/wiki/This_too_shall_pass
Worst case complexity is incentivized in more ways than this comment has room for.
We live in a society.
I've worked on everything from dinosaurs older than me to new greenfield projects using all the latest tech. I think over-complexity emerges in many ways, notably: too much abstraction, ignorance, dogma, and just straight up carelessness.
I think the abstraction problem is the most common and severe though. In enterprise software, I've spent an entire day in the past combing through several layers of services and repositories just to get the full picture of how a response object is being created for a single controller method.
Let me give another example: think about the great debate over ORM vs No-ORM. It's fundamentally a battle about the balances and costs of abstraction in software development. On one hand, you have a (hopefully) simple interface for defining a db's schema, but do I really want to obfuscate the developer from the implementation details of that `findBy` method and trust that the ORM will be 1) used correctly and 2) generate a performant, sensible query? What about security handling? Should I have my developers concerned about preventing potential SQL injections, or just let the ORM take care of that too? The answer depends on a lot of things.
Anyway, I believe for every abstraction:
- something significantly useful should be added (this seems like a truism as I write this, but i've seen a startling amount of useless abstractions)
- the cost should be carefully calculated
- understand where complexity is being added and subtracted
I believe thinking about abstraction in this way would've cut down a lot of the complexity I've seen in softwares past.
Backend suffers from some similar issues, but not to the same degree. Frontend seems to be subject not just to the effects of the apparently-eternal fat/thin client cycle, as the backend also is, but to some other pressures that lead it to be constantly changing but not really getting better, and even getting worse over time.
The "microservices by fad" based architecture usually comes with a huge decrease in performance and an even large one in reliability. The NoSQL for no reason fad came with horrible performance and some very bad data losses. Every once in a while we get some data leakage story here, certainly many are due to complexity (but nobody can tell which ones).
Frontend is not alone suffering for the complexity.
Good point, that one was a big mess.
> The NoSQL for no reason fad came with horrible performance and some very bad data losses.
100% agreed. My perception at the time, though, was that it was largely driven by frontend folks who'd come to the backend via NodeJS buying (especially) MongoDB's marketing hype. There was a real "finally, all those barriers the backend folks had for no reason whatsoever are gone and this is easy now—better, even, you dinosaur, have you seen the marketing, it makes it so clear that this is the future—so I can be full-stack without having to learn much!" vibe to the whole thing. But maybe that's just what I happened to see and that wasn't really the main cause of that going as far as it did.
- 100x in growth, aka scaling.
- Needing to set up reporting data pipelines. Metrics, auditing, 3rd party uses.
Does the frontend need to worry about either of these? Maybe for scaling, they need to do some CDN work, or is that still the backend developer's job?
Basically, backend dev is so complicated because it's done by fools. When it's not done by fools, it's either not complex or has some very difficult requirements, perhaps sometimes both.
Edit to add, I say this as a foolish backend Dev myself, no offence meant to my own kin, of course!