This seems rather alarmist because it's so focused on the bad things and ignores the improvements that come along with the churn.
With each new language and new library, there's a chance to do it better. It's quite a wasteful process, but that's different from things getting worse.
You can read it as alarmist, or you can read it as a rather sobering reflection on the current state of affairs. The world will not end, nothing will explode. What however I think will happen is that the next generation of engineers is going to judge us by our creations and learn from it.
Many of these have been an issue since I started programming many decades ago on minis. Seems as the hardware gets more powerful we find ways to stress out the hardware more and more. This making your list true to a greater extent year after year.
'We are destroying software telling new programmers: “Don’t reinvent the wheel!”. But, reinventing the wheel is how you learn how things work, and is the first step to make new, different wheels.'
and this
'We are destroying software pushing for rewrites of things that work.'
I think that's actually pretty pragmatic. Sometimes the answer is to reinvent the wheel. Sometimes the answer is to keep improving the system that already works.
The problem, IMO, is globally applying either rule.
> We are destroying software pushing for rewrites of things that work.
And sometimes the reverse problem is what is destroying software: being unwilling to push for rewrites of things when they desperately need one. I think we may have over indexed on "never rewrite" as an industry.
> We are destroying software mistaking it for a purely engineering discipline.
And I'm also seeing this the other way: engineers are beginning to destroy non software things, because they think they can blindly apply engineering principles to everything else.
All the statements in that post are trade offs. In all cases you are sacrificing something to gain something else. Therefore in a way you are always "destroying" something.
Sometimes it is valid to not reinvent the wheel. Sometimes wheel needs to be reinvented to learn. Both actions are done. Sometimes the decision was right. Sometimes not.
Overall as a whole we are creating things, more than we are destroying. I don't see the need to take a negative stance.
> Overall as a whole we are creating things, more than we are destroying. I don't see the need to take a negative stance.
Fair point: each one of us can think about the balance and understand if it's positive or negative. But an important exercise must be accomplished about this: totally removing AI from the complexity side.
Most of the results that neural networks gave us, given the hardware, could be recreated with a handful lines of code. It is evident every day that small teams can rewrite training / inference engines from scratch and so forth. So AI must be removed from the positive (if you believe it's positive, I do) output of the complexities of recent software.
So if you remove AI since it belongs to the other side, the "complicated software world" what gave us, exactly, in recent times?
If we discard the AI, which I don't think we should, but if we do - my life has been enriched a lot in terms of doing things I want to do vs things I don't want to. Very quick deliveries, I never have to go to a physical store, digital government online services, never having to wait in any queue, ability to search and find answers without having to go to libraries or know specific people. Online films, tv shows on demand, without ads. There are tons of those things that I feel have made my life so much easier.
I am not sure I understand you then. The post was saying we are destroying something, but I feel like we are constantly gaining and that things are getting better.
The services that enable the things you desire also create harm (Amazon's problems are well documented, digital government services are often a divide that sometimes exclude freedom-minded indivuduals who don't use a "mainstream" OS, to name a couple).
AI has the potential to make the situation much worse, as many laypeople confer it an air of "authority" or "correctness" that it's not really owed. If we're not careful, we'll have an AI-driven Idiocracy, where people become so moronic that nobody can do anything about the system when it takes a harmful action.
Sure, there are trade offs and risks to everything and everything new. Cars made us move faster, but can pollute and cause injury or death. But summing all of those things together, I would not pick any other time before now to live. And same with software development.
I'm sure factory owners said the same thing in England in the early 1800s.
It needs to be noted that the average person's lot didn't improve until 150 years later. There's no reason why technology can't be decided by democratic means rather than shoved in our faces by people that just want to accumulate wealth and power.
I may have worded it poorly, but everyone can choose the content they consume. And activities they do. You can choose mindless things or things that allow you to learn about the World and understand. Both are easier.
"Destroying software" is broader than the creation of new, working software artifacts in the moment. The phrase refers to changes in engineering culture in software and it's long term effects, not the immediate successes.
Writing a new green field project using 10.000 npm dependencies for an electron based front end is shockingly easy. But how do you keep that running for the next 15 years? Did the project need npm? Or a web browser? How do all the layers between the lamguage of choice and the bare metal actually behave and can you reason about that aggregate accurately?
The field has come to a point where a lot of projects are set up with too many complexities that are expedient in the short term and liabilities in the long term.
The current generation of junior devs grows up in this environment. They learn that these mistakes as "the right thing to do" when they are questionable and require constant self-reflection and reevaluation. We do not propagate a hacking culture enough that values efficiency and simplicity in a way that leads to simple, efficient, stable, reliable and maintainable software. On a spectrum of high quality craftsmanship to mass-produced single use crap, software is trending too much to the latter. It's always a spectrum, not a bunary choice. But as a profession, we aren't keeping the right balance overall.
World is full of abstractions on many different levels. Something being on a lower level doesn't inherently mean superior. You can go in any direction on the scale or spectrum. Do you know how exactly atoms behave that computers are made out of? There are plenty of people working on all different sorts of abstractions, new abstractions appear and demand for lower level increases when it is needed. You could say that as more abstractions are built on top of lower level the balance of all the field will go higher in abstraction level on average, but that is the natural way to evolve. Abstractions allow you to build faster and the abstractions are possible because of lower level elements. In the end if you are measuring what an average level of abstraction for current industry is you can draw the line arbitrarily. You could include the people who use website builders and you can calculate the average to be even higher. We need people working on at all different levels of abstraction. We could divide the groups with 2 different naming convention for lower level engineers and higher level, then technically you could go back to calculating that average is still where it used to be.
I definitely use npm (or rather pnpm) because I know it will allow me to build whatever I want much faster.
Abstractions are only part of the whole issue. Maybe I focused too much on that. But I'll argue that point once more.
How much complexity is actually required? What changed in software in the last 20 years so that the additional bloat and complexity is actually required? Hardware has become more powerful. This should make software less reliant on complicated optimizations and thus simpler. The opposite is happening. Why? What groundbreaking new features are we adding to software today that we didn't 20 years ago? User experience hasn't improved that much on average. In fact, measurements show that systems are responding more sluggishly on average.
Intrinsic complexity of the problems that software can solve hasn't really changed much as far as I can see. We add towers of accidental complexity on top that mostly aren't helpful. Those need to be questioned constantly. That isn't happening to the extent that it should. Web-based stuff is the poster child of that culture and it's hugely detrimental.
Backends handling tens / hundreds of thousands or more of concurrent users rather than locally deployed software on a single machine or a small server with a few 10s of users?
Mobile?
Integration with other software / ecosystems?
Real time colaboration amoung users rather than single user document based models?
Security?
Cryptography?
Constant upgrades over the web rather than shipping CDs once a year?
I'll pass on AI for the moment as it's probably a bit too recent.
> Something being on a lower level doesn't inherently mean superior. You can go in any direction on the scale or spectrum. Do you know how exactly atoms behave that computers are made out of?
This is a false equivalency; no one is suggesting that it’s necessary or even useful to understand electron drift to write programs. It can, however, be extremely useful to understand memory allocation and retrieval, and how your languages’ abstractions over it work.
Take UUID generation, for example. I needed to generate millions of them for a tool I’m building, and I found that it was massively slower on Linux than Mac, even taking raw CPU performance into account. I eventually tracked this down [0] to BSD using arc4random, while until a fairly recent glibc release, Linux was using ul_random_get_bytes. Most people are never going to need or notice this difference, because for a single UUID, it simply doesn’t matter. But it did for me, and the only reason I was able to figure it out and solve it was by having a passable understanding of implementations buried behind abstractions.
I've been a backend engineer for about 10 years, with my last job doing an aws lambda stack.
I started a job in manufacturing a few months ago and having to think that this has to work for the next 20 years has been a completely different challenge. I don't even trust npm to be able to survive that so web stuff has been been an extra challenge. I landed on lit web components and just bringing it in via a local CDN.
Agreed and well said. Furthermore, a lot of the statements in the post are making opposing tradeoffs when you put them together. A bunch of them value experimenting and breaking things, and a bunch of others value using what we already have and not breaking things.
A few of them aren’t decisions any individuals have control over. Most coders aren’t jumping onto new languages and frameworks all the time; that’s an emergent group behavior, a result of there being a very large and growing number of programmers. There’s no reason to think it will ever change, nor that it’s a bad thing. And regardless, there’s no way to control it.
There are multiple reasons people write software fast rather than high quality. Because it’s a time/utility tradeoff, and time is valuable. It’s just a fact that software quality sometimes does not matter. It may not matter when learning or doing research, it may not matter for solo projects, it may not matter for one-off results, and it may not matter when software errors have low or no consequence. Often it’s a business decision, not an engineering decision; to a business, time really is money and the business wants engineering to maximize the utility/time ratio and not rabbit hole on the minutiae of craftsmanship that will not affect customers or sales.
Sometimes quality matters and time is well spent. Sometimes individuals and businesses get it wrong. But not always.
I guess the rant should be renamed "business is destroying software" because several of the tradeoffs he mentions can be root caused to a commercial entity cutting corners and sacrificing everything on the altar of "developer time" in order to save money. Only a business would come up with the madness of "Move Fast And Break Things."
These are all complaints specific to a particular attitude and kind of web software development that just happens to be most prevalent within a place like SV.
I think the situation (loosely, wasting time spinning your wheels by modernizing things that are actually fine the way they are and may be made worse by adopting "hot new thing") looks worse when you see it through that lens than it actually is throughout the industry as a whole. There are plenty of opportunities for modernization and doing some of the things described in this article that actually make some sense when applied to appropriate situations.
In other words, I totally understand the vibes of this post, it's one of the reasons I don't work in the parts of the industry where this attitude is most prevalent. I would never feel the push to write a post like it though because the poster is I think being a bit dramatic. At least that's the case looking at the industry from my (id argue broader) vantage point of being an expert in and working quite a bit with "legacy" companies and technologies that maybe could stand to have a new UI implemented and some tasteful modern conventions adopted at least as options for the end users.
Yes. There are better and worse ways to lead a team, just as there are better and worse ways to prompt AI.
This post reads as a description of how the wheels come off the wagon if you don't do things well.
With the evolution of AI agents, we'll all be virtual CTOs whether we like it or not, mastering scale whether we like it or not. We need to learn to do things well.
If I had to guess, antirez was describing engineering managers and tech leads that have (mis)read “clean code” or similar works, and take them as commandments from on high rather than tools and approaches that may be helpful or applicable in some circumstances.
Or, more generally, the fact that most of what the software industry produces is much more in line with “art” than “engineering” (especially when looked at from a Mechanical Engineer or Civil Engineer). We have so much implementation flexibility to achieve very similar results that it can be dizzying from the standpoint of other engineering fields. consider
In my view it is about design that requires taste and creativity. Engineering is about function, design is about form. If I build something that solves a problem but if it isn't well designed it can mean that no one actually uses even if it is good piece of engineering.
I think the final point illustrates this one pretty succinctly: 'what will be left will no longer give us the joy of hacking.' Personally I build my own version of almost every software tool for which I regularly (like, daily) use a UI. So for e.g. personal note-taking, continuous integration, server orchestration, even for an IDE: I could use Apple Notes, CircleCI, Chef, VSCode, but I instead build my own versions of these.
I'm not a masochist; they're often built on top of components, e.g. my IDE uses the Monaco editor. But working on these tools gives me a sense of ownership, and lets me hack them into exactly the thing I want rather than e.g. the thing that Microsoft's (talented! well-paid! numerous!) designers want me to use. Hacking on them brings me joy, and gives me a sense of ownership.
Like an idealised traditional carpenter, I make (within reason) my own tools. Is this the most rational, engineering-minded approach? I mean, obviously not. But it brings me joy -- and it also helps me get shit done.
While I highly respect antirez, I think this post is full of good sounding, short statements, that wouldn't hold in a discussion.
One example: Newbies shouldn't reinvent the wheel. I think they should use the tools, that are available and common in the given context. When they want to tinker, they should write their own compiler. But they shouldn't use that in production.
Another: Backward API compatibility is a business decision in most cases.
Also, I think it doesn't help to start every sentence with "We are destroying software". This sounds much more gloomy, than it really is.
the shitstorms usually have a community behind it. Even if it sucks, it's supported and will be maintained to a point of it "working." If someone writes their own thing, chances are they won't go the extra mile and build a community for it. Then, when it comes to maintaining it later, it might grow untenable, especially if the original "tinkerer" has moved on.
This would be great if things like open source were more supported in the "real" world. Unfortunately, you're describing exactly why a community means nothing in this situation unless the community is giving back to the project. When the "original tinkerer" moves on, everything depending on that project breaks when everything else changes around it.
These systems also came from tinkering. Most programming languages even are really the investment of one person for a long time, doing apparently what you aren't supposed to do.
When it comes down to it, whatever works best and is usually the most simple, non-breaking, used to win out. That decision has been disconnected from the value creators to the value extractors. It is impossible to extract value before value is created.
Additionally, programming is a creative skill no matter how hard they try to make it not one. Creativity means trying new things and new takes on things. People not doing that will harm us long term.
>> But they shouldn’t use that in production.
> Why?
Generally speaking, because that’s very likely to end up being “pushing for rewrites of things that work”, and also a case of not “taking complexity into account when adding features”, and perhaps in some cases “jumping on a new language”, too.
This is an imagined scenario, but the likelihood of someone easily replacing a working compiler in production with something better is pretty low, especially if they’re not a compiler engineer. I’ve watched compiler engineers replace compilers in production and it takes years to get the new one to parity. One person tinkering and learning how to write a compiler almost for sure does not belong in production.
"parity" is the keyword here. Most of the time, the problem doesn't come from sloppy execution, but ever widening scope of the software.
For example, my own "Almost C" compiler[1] is 1137 lines of code. 1137! Can it ever reach "parity" with gcc or even tcc? No! That's specifically not the goal.
Do I benefit strongly for having a otherworldly simpler toolchain? hell yeah.
The key is scope, as always. Established projects have, by virtue of being community projects, too wide a scope.
Agreed, parity is a strong constraint. Since the premise under discussion was “production” environments where some kind of parity is presumably required, I think it’s a reasonable assumption. If there is no baseline and it’s not a production situation where parity is needed, then yeah scope can and should be constrained as much as possible. I like the phrase “otherworldly simple”, I might borrow that!
I don't think that people deciding what goes in "production" are immune to scope inflation. Thinking that we need parity with another software without realizing the cost in terms of complexity of that parity requirement is, I think, a big driver of complexity.
Those who do not know history are doomed to repeat it. Or re-re-reinvent Lisp.
There was this anecdote about storm lamp or something. New recruit comes to a camp and sees old guard lighting lamps are turned upside down and lit sideways with a long stick. But he knows better, he tells them and they smirk. First day he lights them the optimal way with a lighter. He's feeling super smug.
But next day he finds the fuse is too short to reach so he takes the long stick...
Few months later, he's a veteran, he's turning lamp upside down using lighter sideways, with a long stick.
And the fresh recruit says he can do it better. And the now old guard smirks.
I'm sure I'm misremembering parts, but can't find the original for the life of me.
If someone would hand me a project, that is full of self invented stuff, for example a PHP project, that invented its own templating or has it's own ORM, I would run. There is laravel, slim or symfony, those are well established and it makes sense to use them.
There are so much resources around those frameworks, people who posted about useful things, or packages that add functionality to those.
It just doesn't make sense to reinvent the wheel for web frameworks and thousands of packages around those.
Writing software is standing on the shoulders of giants. We should embrace that, and yes one should learn the basics, the underlying mechanisms. But one should make a difference between tinkering around and writing software, that will be in production for years and therefore worked on by different developers.
The JavaScript world shows how to not do things. Every two years I have to learn the new way of building my stuff. It is annoying and a massive waste of resources. Everyone is always reinventing the wheel and it is exhausting. I understand why it is like this, but we as developers could have made it less painful, if we would embrace existing code instead of wanting to write our own.
I strongly disagree. They should, and fail and try again and fail. The aim is not to reinvent the wheel, but to understand why the wheel they're trying to reinvent is so complex and why the way it is. This is how I learnt to understand and appreciate the machine, and gave me great insight.
Maybe not in production at first, but they don't reinvent the wheel in their spare time either. They cobble up 200 package dependency chains to make something simple, because that’s what they see and taught. I can write what many people write with 10 libraries by just using the standard library. My code will become a bit longer, but not much. It'll be faster, more robust, easier to build, smaller, and overall better.
I can do this because I know how to invent the wheel when necessary. They should, too.
> Another: Backward API compatibility is a business decision in most cases.
Yes, business decision of time and money. When everybody says that you're losing money and time by providing better service, and lower quality is OK, management will jump on it, because, monies.
> Also, I think it doesn't help to start every sentence with "We are destroying software". This sounds much more gloomy, than it really is.
I think Antirez is spot on. We're destroying software. Converting it to something muddy and something for the ends of business, and just for it.
I'm all with Antirez here. Software came here, because we developed the software just for the sake of it, and evolved it to production ready where needed. Not the other way around (Case in point: Linux).
> Yes, business decision of time and money. When everybody says that you're losing money and time by providing better service, and lower quality is OK, management will jump on it, because, monies.
Often that "saving money" is just externalizing the cost onto your users. Especially in mobile development. Instead of putting in the tiny amount of effort it takes to continue support for older devices, developers just increase the minimum required OS version, telling users with older hardware to fuck off or buy a new phone.
Another example is when you don't take the time to properly optimize your code, you're offloading that cost onto the user in the form of unnecessarily higher system requirements.
> I'm all with Antirez here. Software came here, because we developed the software just for the sake of it, and evolved it to production ready where needed. Not the other way around (Case in point: Linux).
Growing up in the 80s and 90s I understand viscerally how you feel, but this take strikes me as willfully ignorant of the history of computers, and the capitalist incentives that were necessary for their creation. The first computer and the internet itself were funded by the military. The PC wouldn't have existed if mainframes hadn't proved the business value in order to drive costs down to the point the PC was viable. Even the foundational ideas that led to computers couldn't exist with funding—Charles Babbage's father was a London banker.
I think a lot of what you are reacting to is the failed promise of free software and the rise of the internet, when the culture was still heavily rooted in 60s counter-culture, but it hadn't crossed the chasm to being mainstream, so it was still possible to envision a utopian future based on the best hopes of a young, humanitarian core of early software pioneers operating largely from the sheltered space of academia.
Of course no such utopian visions ever survive contact with reality. Once the internet was a thing everyone had in their pocket, it was inevitable that software would bend to capitalist forces in ways that directly oppose the vision of the early creators. As evil as we thought Microsoft was in the early 90s, in retrospect this was the calm before the storm for the worst effects of tech. I hear Oppenheimer also had some regrets about his work. On the plus side though, I am happy that I can earn enough of a living working with computers that I have time to ponder these larger questions, and perhaps use a bit of my spare time to contribute something of worth back to the world. Complaining about the big picture of software is a fruitless and frustrating endeavour, instead I am interested in how we can use our expertise and experience to support those ideals that we still believe in.
I think what he is trying to say is that the value or focus was better when it was placed on the endeavors, not the means behind the endeavors. I don't think anything has to be inevitable. What matters is what we decide to do when challenges like these present themselves and how can we have more of a positive impact on the world when things go awry.
I take issue with your use of the word "utopian" being used in this context. Its not a lost cause to see the world from the perspective of making the world better, by finding our way though this with a better mindset on the future.
And while you are taking the time to ponder these questions because you earn enough to take the time, the world is burning around you. Sorry if my tone is harsh, but these kinds of statements really rub me the wrong way. It feels like you are saying everything that is happening is how its suppose to be and I am strongly against that. We have enough of that perspective, we really don't need it, IMHO.
Fair criticism, but saying "we are destroying software" is not actionable. I want us to do better, but I also want to be effective, and not just sit around impotently wringing our hands about how bad everything is.
Fair point. I agree with you. Some times it just takes one person to say whats wrong with the world, to make people realize something can/has to be changed.
I'll kindly disagree. For me, seeing or accepting where we are currently is enough to gently motivate me to do whatever I can do to change the current state.
This gentle motivation is good, because it allows me to look inside and be rational about my ambitions. I won't go to a blind crusade, but try to change myself for the better.
Because, I believe in changing myself to see that change in the world.
I actually don’t agree. Maintaining or not maintaining backwards compatibility is often a decision made on the technical level, e.g. by a tech lead, or at least heavily based on the advice from technical people, who tend to prefer not being restricted by backwards compatibility over not breaking things for relying parties.
They absolutely should, or they will never even get to understand why they are using these wheels.
Fun fact, try to question modern web developers to write a form, a simple form, without a library.
They can barely use html and the Dom, they have no clue about built-in validation, they have no clue about accessibility but they can make arguments about useMemo or useWhatever in some ridiculous library they use to build...ecommerces and idiotic crud apps.
> Newbies shouldn't reinvent the wheel. I think they should use the tools, that are available and common in the given context. When they want to tinker, they should write their own compiler. But they shouldn't use that in production.
So basically they shouldn’t learn the prod systems beyond a shallow understanding?
Software technology is in decline despite appearances of progress. While hardware improvements and machine learning create an illusion of advancement, software's fundamental robustness and reliability are deteriorating. Modern software development has become unnecessarily complex, with excessive abstraction layers making simple tasks difficult. This complexity reduces programmer productivity and hinders knowledge transfer between generations. Society has grown to accept buggy, unreliable software as normal. Unless active steps are taken to simplify software systems across all levels, from operating systems to development tools, civilization faces the risk of significant technological regression similar to historical collapses.
I think that talk might be Jonathan Blow's most important work to date actually. I love The Braid and The Witness, but "Preventing the Collapse of Civilization" managed to articulate what at least me and my circle of devs have talked about and discussed for long time, but were never quite able to put into words. I'm very grateful for people like him and others like Casey Muratori, Mike Acton etc. who continue to point this out very real danger in the (at least) last decade.
Unfortunately my stance is that fundamentally things won't change until we get hit with some actual hardware limitations again. Most devs and people in general prefer a semblance of a working solution quickly for short-term gains rather than spending the actual time that's needed to create something of high quality that performs well and will work for the next 30 years. It's quite a sad state of affairs.
With that said I'm generally optimistic. There is a small niche community of people that does actually care about these things. Probably won't take over the world, but the light of wisdom won't be lost!
Oftentimes there isn’t a need for something to work for the next 30 years as the business will change in a much shorter timeframe and software exists to serve the business. While I agree that software quality varies wildly if the business can’t get off, the ground because the software isn’t ready, the software team will quickly stop existing with the rest of the business.
The way I interpret the decline of software is that part of the problem is because more software has been written with the mentality that it doesn't need to last 30 years.
It's a self-perpetuating issue: people build stuff saying "it won't/can't last 30 years" for various reasons (money, time, skill, resources, expectations, company culture, business landscape etc). So then software doesn't last 30 years for those same various reasons.
The idea that we think systems did used to last longer is probably some survivor bias. However, software that has survived decades was probably created with a completely different set of methodologies, resources and incentives to modern software.
Linux and Java has survived since 1995-1998, which is the dusk of times.
We swap JS frameworks constantly, but when we’ll reach a good paradigm, we’ll stick with it. At one point, React might be the final framework, or rather, one of its descendants.
If history had taught us anything in computing it’s that breaking backwards compatibility/rewriting it isn’t the panacea. See Perl, python, C++, longhorn, mosaic for example. My 4k, 120hz monitor with 32 bit color depth and 16 4.5GHz cpu cores attached to it still renders a terminal at 80 characters wide and chokes on cat bigfile.bin because we’re that married to not changing things.
was there a large breaking backwards compatibility break for C++? I was there for some of the Python2/3 transition but I’ve always thought C++ was adamantly backwards compatible (minor exceptions being stuff like auto_ptr/unique_ptr, which I don’t think was as big as a break as Python’s)
Sorry - I meant that C++ as an “improved” C never managed to remove C’s foothold, it just fractured the ecosystem,
When the c++11 abi break happened it was a big pain in the ass, but once MSVC decided in 2015 that they were going to stop breaking ABI I think it was the stability that c++ needed to fossilize…
Writing bad code for any reason is bad, the question is whether you can write good code to get the next release out. Or are you saying there's no such thing as good code that meets the next release' requirements?
That is untrue for almost all software written outside of companies who's primary product is tech or software. I work for a publisher and while there's a ton of disposable microsites and project work (that nobody ever cleans up or documents, so you never know if it's okay to remove), there's also ancient monoliths written in forgotten programming languages that are so important for production and have so many hidden dependencies and unknown users ("random FTP server" being my favorite), that you can barely put on new paint.
I'm writing software with the assumption that it'll be used for at least 30 years there, with a lot of guard rails and transparency/observability mechanisms, because I know the next person working there will thank me.
I don’t mind this philosophy, but in aggregate I think very slow applications that are cumbersome and widely deployed have cost humanity many human lifetimes because a handful of developers were not able (or not given the opportunity) to optimise them even a little.
I am aware that capitalism essentially dictates externalising cost as much as possible, but with software- much for the same reason capitalism loves it (a copy is cheap and can be sold at full price despite being constructed just once) means that these externalities can scale exponentially.
Teams in particular is an outlier as in most cases it is essentially forced on people.
It doesn't dictate externalising cost as much as possible unless you have a very short-term view.
Short-term view businesses get eaten pretty quickly in a free capitalist system.
People forget that half of capitalism's advantage is the "creative destruction" part - if businesses are allowed to fail, capitalism works well and creates net value.
It’s interesting that all three of the people you mention are very concerned with performance, something most programmers don’t even think about anymore or think they aren’t supposed to.
As a group, we have trained many, many programmers out of even considering performance with the proliferation of quotes like “premature optimisation is the root of all evil” ands ideas like “who cares, just get a faster computer/wait for hardware”.
Premature optimisation is bad, but there’s now so many devs who don’t do _any _ at all. They don’t improve any existing code, they’re not writing software that is amenable to later optimisation, inefficient architectures and unnecessary busywork abounds.
Are we surprised that years of “product first, bug fixes later, performance almost never” has left us with an ecosystem that is a disaster?
Yes, people think that writing performant software is something that's nice-to-have when in fact if you program with the intention of things being performant then that branches out to better overall design, better user experience, better quality etc. It means you actually care about what you're doing. Most people don't care and the results are apparent. We're surrounded by waste everywhere. Bad software wastes resources like money, electricity and most importantly - time.
The fact that people don't stay long enough in companies or work on a long project themselves to see the fruits of their labour down the line is a point that is discussed in some of the other comments here in this thread. I agree with it as well. In general if you job hop a lot you won't see the after effects of your actions. And the industry is such that if you want to get paid, you need to move. To reiterate - it's a sad state of affairs.
The problem is that we generally don't have anyone a good track record of what good software is valued at, it USED to around 300-500$ and with companies being incentivized to go subscription based who knows intuitively what that is.
I work in a two man team making software that is 500-1000 times faster than the competition and we sell it at ~40% of their price. Granted, this is in a niche market but I would be very careful in stating price/costs are the entire picture here. Most developers, even if you suddenly made performance a priority (not even top priority, mind you), wouldn't know how to actually achieve much of anything.
Realistically only about 5% or so of my former colleagues could take on performance as a priority even if you said to them that they shouldn't do outright wasteful things and just try to minimize slowness instead of optimizing, because their entire careers have been spent optimizing only for programmer satisfaction (and no, this does not intrinsically mean "simplicity", they are orthogonal).
Generalized? Probably not. Replicated to a much higher degree than people think? I think so. It wouldn't matter much to me personally outside of the ability to get hired because I have no desire to create some massive product and make that my business. My business is producing better, faster and cheaper software for people and turning that into opportunity to pay for things I want to do, like making games.
Disclaimer: Take everything below with a grain of salt. I think you're right that if this was an easy road to take, people would already be doing it in droves... But, I also think that most people lack the skill and wisdom to execute the below, which is perhaps a cynical view of things, but it's the one I have nonetheless.
The reason I think most software can be faster, better and cheaper is this:
1. Most software is developed with too many people, this is a massive drag on productivity and costs.
2. Developers are generally overpaid and US developers especially so, this compounds for terrible results with #1. This is particularly bad since most developers are really only gluing libraries together and are unable to write those libraries themselves, because they've never had to actually write their own things.
3. Most software is developed as if dependencies have no cost, when they present some of the highest cost-over-time vectors. Dependencies are technical debt more than anything else; you're borrowing against the future understanding of your system which impacts development speed, maintenance and understanding the characteristics of your final product. Not only that; many dependencies are so cumbersome that the work associated with integrating them even in the beginning is actually more costly than simply making the thing you needed.
4. Most software is built with ideas that are detrimental to understanding, development speed and maintenance: Both OOP and FP are overused and treated as guiding lights in development, which leads to poor results over time. I say this as someone who has worked with "functional" languages and FP as a style for over 10 years. Just about the only useful part of the FP playbook is to consider making functions pure because that's nice. FP as a style is not as bad for understanding as classic OOP is, mind you, but it's generally terrible for performance and even the best of the best environments for it are awful in terms of performance. FP code of the more "extreme" kind (Haskell, my dearest) is also (unfortunately) sometimes very detrimental to understanding.
I think I'd need a lot more than one claim with no evidence that this is true but I appreciate the thoughtful response all the same.
Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
I don't think either of us needs to convince the other of anything, I'm mostly outlining this here so that maybe someone is actually inspired to think critically about some of these things, especially the workforce bits and the cost of dependencies.
> Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
This seems like a generous take to me, but I suppose it's usually better for the soul to assume the best when it comes to people. Companies with explicit performance requirements will of course self-select for people capable of actually considering performance (or die), but I don't take that to mean that the rest of the software development workforce is actually able to, because I've seen so, so many examples of the exact opposite.
I guess I just see so many of these "the whole industry except for me is doing it wrong" kinds of posts but almost never see anybody really back it up with anything concrete that I've grown extremely skeptical.
I can't take Blow seriously after his meltdown on how difficult it is to test games on different OSes while some developers already released on multiple platforms... all in a one man's band (i.e. https://www.executionunit.com/blog/2019/01/02/how-i-support-...).
Aside, he's treated like a celebrity in the game developer niche and I can't understand why.
He has some good takes (like the talk he gave in the OP) but also has some questionable ones. How he feels on work-life balance comes to mind. He seems to legitimately hate having to employ other people to help create his games.
It can be difficult and at the same time be possible. Clearly he also released his games on multiple platforms. Braid I think was everywhere from Xbox 360, Windows, Mac, Linux, Switch and mobile phones.
Just because one person is willing to do everything needed to test on all platforms doesn't mean everyone should therefore be willing to put the time and effort into it.
Depending on what tech you use it can be easier or harder to do as well. I'm making a game with Love2D now, which has made supporting Mac and Linux rather trivial so far, although I've run into challenges with mobile platforms and the web even though it support them (it does work, but it takes more low-level glue code to support native phone features, and the web doesn't seem to be well maintained and my game is throwing webassembly errors currently when I try to run it).
And my previous game (which is on the backburner for now) was made with Monogame, and while that technically has support for Mac and Linux (as well as mobile), I've had quite a few issues even just getting the Mac version working well, like issues with resolution, rendering 3D properly, getting shaders not to break, etc. And they haven't kept up with the latest Mac updates the past few years and have had to make a special push to try to get it caught back up. I've probably sunk a good 20+ hours trying to get it working well before putting that aside to work on the actual game again and I still might have to rearchitect things a bunch in order to get that working.
Meanwhile Unity would probably be pretty dirt simple to port, for the most part, but it comes with other tradeoffs, like not being open source, and trying to pull a stunt a couple years ago where they pulled the rug out from under developers with changing licensing to something aggressive (that convinced other developers to port their games away from the platform), etc.
And there's Godot, which seems to be getting more support again (which is great, I even considered it or my current game, I just like coding in Love2D a bit better), but if you ever want your game on consoles you have to pay a third party to port your games to consoles for you.
The guy you linked makes their own engine (and to be fair, so does Jonathan Blow, who you're critiquing), which is great, but not everyone wants to get that low level. I would rather spend more time focusing on building the games themselves, which is already hard enough, rather than spending all that time building an engine.
It was for that reason that I spent several years focused on board game design instead (as I can make a working game with drawing on index cards and some colored plastic cubes in less than an hour), although that has its own frustrations with significant hurdles to get your game signed by publishers as an unknown designer (I did get one signed four years ago, and it's still not released yet), and large financial risks being made for manufacturing and distribution.
Edit: Also the person you linked to isn't even sure it was financially worth it to support all of those platforms, they just do it for other reasons:
"Do you make your money back? It’s hard to say definitely which could mean no. Linux and macOS sales are low so the direct rewards are also low. ...For Smith and Winston [their current game] right now I don’t think it’s been worth it financially (but it will probably make it’s money back on Launch)"
That's a feature. Permanent systemization of short term worldviews and lenses is pretty horrifying. That's how those techno-theocratic civilizations in pop culture happen.
Imagine if a company had been able to systemize the whims of a paranoid regime, allowing them to track and spy on their citizens with impunity in secret, and the population became so inured to the idea that it became an accepted facet of their culture.
Or what if a political party dominated a state and systematized a way to identify and detect oppositional thought, stamping it out before a counterculture could ever arise. Maybe that thought was tied to a particular religious, ethnic, and/or cultural group.
What if these companies are here today, selling those products to the highest (nation-state) bidders, and the methods they're employing to keep the conceptual wheels turning at scale rely on filtering that selects for candidates who will gladly jump through abstract hoops set before them, concerned only with the how and never the why of what they're doing.
> Software technology is in decline despite appearances of progress. While hardware improvements and machine learning create an illusion of advancement, software's fundamental robustness and reliability are deteriorating. Modern software development has become unnecessarily complex, with excessive abstraction layers making simple tasks difficult. This complexity reduces programmer productivity and hinders knowledge transfer between generations. Society has grown to accept buggy, unreliable software as normal. Unless active steps are taken to simplify software systems across all levels, from operating systems to development tools, civilization faces the risk of significant technological regression similar to historical collapses.
I haven't watched that talk by Blow yet so maybe he covers my concern.
I think you have to be mindful of incentives structures and constraints. There's a reason the industry went down the path that it did and if you don't address that directly you are doomed to failure. Consumers want more features, the business demands more stuff to increase its customer base, and the software developers are stuck attempting to meet demand.
On one hand you can invent everything yourself and do away with abstractions. Since I'm in the embedded space I know what this looks like. It is very "productive" in the sense that developers are slinging a lot of code. It isn't maintainable though and eventually it becomes a huge problem. First no one has enough knowledge to really implement everything to the point of it being robust and bug free. This goes against specialization. How many mechanical engineers are designing their own custom molding machine in order to make parts? Basically none, they all use mold houses. How many electrical engineers are designing their own custom PCB(A) processing machines or ICs/components? Again, basically none. It is financially impossible. Only in software I regularly see this sentiment. Granted these aren't perfect 1-to-1 analogies but hopefully it gets the idea across. On the other hand you can go down the route of abstractions. This is really what market forces have incentivized. This also has plenty of issues which are being discussed here.
One thought that I've had, admittedly not fully considered, is that perhaps F/OSS is acting negatively on software in general. When it comes to other engineering disciplines there is a cost associated with what they do. You pay someone to make the molds, the parts from the molds, etc... It's also generally quite expensive. With software the upfront cost to adopting yet another open source library is zero to the business. That is there is no effective feedback mechanism of if we adopt X we need to pay $Y. Like I said, I haven't fully thought through this but if the cost of software is artificially low that would seem to indicate the business and by extensions customers don't see the true cost and are themselves incentivized to ask for more at an artificially low price thus leading to issues we are currently seeing. Now don't misread me, I love open source software and have nothing but respect for their developers; I've even committed to my fair share of open source projects. As I've learned more about economics I've been trying to view this through the lens of resource allocation though and it has lead me to this thought.
I no longer think that's true. Instead, I think consumers want reliability, but more features is a way to justify subscription pricing segregation and increases.
Everyone has a different a tipping point. But generally I see folks want more features and don't value reliability unless it's something they use really often and has no workaround.
I play games with known bugs, and on imperfect hardware, because I unwilling to pay more. Some experiences are rare, so I tolerate some jank because there aren't enough competitors.
Operating Systems are quite mature. I suppose they do need to evolve to take advantage of newer hardware and new UI conventions. For example, swipe typing and H264 decoding are table stakes on mobile.
Adopting libraries and 3P solutions is like jumping in the pool, easy to do. Getting out of the pool is much harder. Or in some cases like jumping into quick sand. Sometimes it can be hard to tell which before you're in it.
Most large enterprise IT departments are fully aware that the cost of adopting yet another open source library is very high even if the price is zero. This cost comes in the form of developer training, dependency management, security breaches, patent troll lawsuits, etc. There is a whole niche industry of tools to help those organizations manage their open source bill of materials.
Interesting. My experience is that bulky abstraction layers are harder to maintain than own software.
In game development, whenever we go with highly abstract middleware, it always ends up limiting us in what we can do, at what level of performance, how much we can steer it towards our hardware targets, and similar. Moreover, when teams become so lean that they can only do high level programming and something breaks close to the metal, I’ve seen even “senior” programmers in the AAA industry flail around with no idea how to debug it, and no skills to understand the low level code.
I’ve seen gameplay programmers who don’t understand RAII and graphics programmers who don’t know how to draw a shape with OpenGL. Those are examples of core discipline knowledge lost in the games industry. Aka what we have now, we might not know anymore how to build from scratch. Or at least most software engineers in the industry wouldn’t. It cannot end well.
Building your own in my exp is a better idea — then you can at least always steer it, improve and evolve it, and fix it. And you don’t accidentally build companies with knowledge a mile wide and an inch deep, which genuinely cannot ship innovative products (technically it is impossible).
The notion that software defects could destroy civilization is just so silly. I think some people have the impression that the quality of tools and technologies was somehow better before. But since the beginning of the industrial age, most of our stuff has been kind of crap. You might see good quality stuff preserved by enthusiasts or museums but that's survivorship bias and what most people used was worse. People muddle through somehow.
It used to be everything ran on mechanical machines and analog electronics. That stuff failed all the fucking time. We're no worse off today with software.
I'm responding to the implication that "mechanical shit" is local and thus less damaging.
Since they mentioned nukes it seemed like an obvious example where local things can be catastrophic.
The theoretical risk of electronic things malfunctioning in some global way that they mentioned has never resulted in any nuclear weapons being deployed, but we've actually seen the local mechanical approach they disregard be devastating.
> Thus far, the electronic age has been much less bloody.
You're not looking at the right signs.
In terms of pollution and environmental damage, including with direct human impact such as microplastics, things have been getting worse pretty much exponentially quicker every year _since_ the end of WW2.
As an opposite view point I work on a 10 year old legacy iOS app which has almost no abstractions and it’s a huge mess. Most classes are many thousands of lines long and mix every concern together.
We are fixing it up by refactoring - many through adding abstractions.
I’m sure code with bad abstractions can scale poorly, but I’m not clear how code without abstractions can scale at all.
Abstraction is a an abstract word, but as an example, I would consider the process of refactoring a big piece of code which mixed together api requests and view presentation into a 2 classes (a presenter, and an api client) as adding abstraction, especially if those new components are defined by interfaces.
And I’d rather work in a codebase where the owners are constantly doing such refactors than not.
edit: if you feel the need to downvote, feel free to share why you think my question is problematic - I think that "poorly written" to describe excessive code file sizes is such a wooly phrase as to be basically useless in a discussion like this, and when examined more closely usually comes down to "poorly abstracted". But I stand to be corrected.
Well, the EU recognized this after pushback, but not all lawmakers do!
It works out okay to require compliance from paid vendors - if it costs more to comply with security regulations, that will be included in the price. But it's not reasonable to impose requirements on volunteers that way. After the modification, the EU regulation puts the burden on paid vendors who use free software (who can afford to pay for it), not on free software volunteers.
Paid software == a requirement to be of merchantable quality: to do what it claims, do it reliably, and keep doing it, so long as clearly stated requirements are met.
If this means all paid apps become tiny simplistic things in weird restrictive type-safe languages like Ada -- win!
Free software == you break it, you get to keep both halves. Our liability is what you paid, nothing more.
Think about what happens when two people video call each other from opposite sides of the world. How many layers of hardware and software abstraction are they passing through, to have a reliable encrypted video conversation on a handheld computer where the majority of the lag is due to the speed of light? How much of that would you like to remove?
I would venture an alternative bogeyman - "move fast and break things" AKA the drive for profits. It's perfectly possible (as illustrated above) to reliably extract great performance from a tower of abstractions, while building those abstractions in a way that empowers developers; what's usually missing is the will to spend time and money doing it.
> How many layers of hardware and software abstraction are they passing through, to have a reliable encrypted video conversation on a handheld computer where the majority of the lag is due to the speed of light?
And then the video player crashes trying to play an ad
The older I get the less stock I put in merely pointing out flaws without offering solutions.
You might say "I don't need to be able to propose a solution in order to point out problems", and sure, but that's missing the point. Because by pointing out a problem, you are still implicitly asserting that some solution exists. And the counter to that is: no, no solution exists, and if you have no evidence in favor of your assertion that a solution exists, then I am allowed to counter with exactly as much evidence asserting that no solution exists.
Propose a solution if you want complaints to be taken seriously. More people pointing out the problems at this point contributes nothing; we all know everything is shit, what are you proposing we do about it?
My rather broad solution has always been: let engineers own a part of a stack. Let an engineer own the UI for an app, own the database front-end. Let an engineer own the caching mechanism, let an engineer own the framework.
You give an engineer ownership and let them learn from their own mistakes, rise to the occasion when the stakes are high. This presumes they will have the last word on changes to that sand box that they own. If they want to rewrite it — that's their call. I'm the end they'll create a codebase they're happy to maintain and we will all win.
Defining or clarifying the specifics of the problem is a critical step in solving (or not solving) it. We don't have a good understanding of all of the factors and how they contribute to this problem so having more people take a stab at understanding the problem and sharing that is a net positive. You may think that "we all know it already" but we don't. I discover new and meaningful ways that systems and people are fucking up software just about every year and have been for 25-30 years so I take strong issue with your "we all know" when clearly we don't, and in fact very much still disagree on the details of that problem, the very things we need to understand in order to best solve the problem.
We are destroying software with Leetcode interviews, resume-driven development, frequent job-hopping, growth investment scams, metrics gaming, promotion-seeking, sprint theatre, bullshitting at every level of the org chart, and industry indifference.
Just how many FOSS projects there are that are dead? Just how many different ways to do things have been invented? Say Linux desktop environments. Or parts of subsystems.
Fair, but then again FOSS is born from individual hackers who want to learn how to build X, or who feel like the ecosystem doesn't provide the X they would like to have.
It fosters a culture where everyone can hack something together, and where everyone is knowledgeable enough to make responsible use of technology.
Working as a for-hire developer doesn't let you experience all of that because you're building a product that someone else wants you to build. No wonder one does not give a shit about writing good software at that point! You've taken all the fun and personal fulfillment out of it!
We can build anything we put our mind to -- but most of us are too busy churning out CRUD boilerplate like factory workers. That's depressing.
Because the software we are paid to write has an external purpose. As scale increases, supply chains lengthen, and roles specialize, inevitably more and more people are missing the forest for the trees—this isn't an indictment, it's an inevitable result of the pursuit of efficiency and scale at all costs. Many engineers would be happy to polish the thing that exists, perhaps adding complexity the form of scalability, modularity or reusability that isn't actually needed, and in fact may make it harder to adapt to broader changes in the ecosystem that the company operates in. "Impact" is just a suitably hand-wavy management buzzword to be used in lots of different situations where they deem ICs to not sufficiently taking the big picture into account.
> Many engineers would be happy to polish the thing that exists, perhaps adding complexity the form of scalability, modularity or reusability that isn't actually needed, and in fact may make it harder to adapt to broader changes in the ecosystem that the company operates in
When done correctly it absolutely adds business value and should not make it harder to adapt or change, that's the point of good engineering. The problem is that you need years, if not decades, of technical experience to see this, and it's also a hard sell when there is no immediate "impact". It's basically something that happens because consumers don't know any better, so then it becomes low priority for making profit... at least until a competitor shows up that has better software and then it has a competitive edge, but that's a different matter.
Sure. I'm just explaining why the term exists. Of course it is often applied by clueless managers, there's nothing you can do about that except go find a better manager. Just don't make the mistake of thinking its all bullshit because you've only dealt with muppets on the leadership side—I see this often with journeyman engineers who have never had the privilege of a good management team and it's very sad.
Because they’re all in the same tiny social circle and went to the same schools, joined the same clubs, intermarried, and just generally isolate themselves from the rest of us.
If you don't want to have any say in the work that you're doing, then sure don't judge based in impact. I'd rather have the responsibility and trust of letting me decide what to work on, rather than management tell me. The only way that works is that my judgement is good, I deliver impact, and I'm paid as such.
You are optimizing for your resume and pay, not the product's needs. That's what people are criticizing; what constitutes impact, how it is measured, and whether that's what's needed to make the product better.
In fact I'd argue it's the _exact_ opposite. I've seen countless engineers architect big complicated systems that are completely unnecessary. If anyone was measuring, they'd realize the impact of that system was: no incremental users, no incremental revenue, increased operational overhead, higher headcount, and more burden in further development. Is that what the user needs? You tell me.
There is implicit impact by engineering good systems and software. It's like hiring a plumber and telling them "you need to improve my house" rather than "you need to improve the plumbing in my house". When your objective as an engineer is so broad that you need to worry about customers and the product, then the engineering itself suffers... that said it might not suffer to the extent that it matters but that is the point here.
Bad software keeps happening because businesses can afford it, as well as hardware improvements. It's a combination of consumers not knowing what they are missing and hardware advancements allowing for bad software to exist.
> When your objective as an engineer is so broad that you need to worry about customers and the product, then the engineering itself suffers... that said it might not suffer to the extent that it matters but that is the point here
If you aren’t writing software with the customer and business in mind, why are you doing it? That’s what you are getting paid for.
To clarify, I don't mean having no concern for the business or customer, but rather having that as a priority and the primary way to determine contribution. Engineers should care about the engineering first, which implicitly has value if done correctly.
That said, AI is about to change all this... but ironically this justifies my position. If software can be so powerful such that you can have a general intelligence that can do almost any cognitive task... then it follows that all this time engineers can also engineer clever systems that can add a lot of value to the company if engineered correctly.
There is no ceiling of how much performance and value can be squeezed out of a software system, but this never happens because businesses are not investing in actual engineering but rather in technical entrepreneurs that can bring something to market quickly enough.
> To clarify, I don't mean having no concern for the business or customer, but rather having that as a priority and the primary way to determine contribution. Engineers should care about the engineering first, which implicitly has value if done correctly
There is no “implicit value” of software to a company. The only value of a software to a company is whether it makes the company money or saves the company money. That’s it, there is no other reason for a company to pay anyone except to bring more value to the company than they cost to employ them.
If software can be so powerful such that you can have a general intelligence that can do almost any cognitive task... then it follows that all this time engineers can also engineer clever systems that can add a lot of value to the company if engineered correctly.
It’s just the opposite. If AI can do any of the coding that a software engineer can do and I am not saying that’s possible or ever will be, what’s becomes even more important are the people who know how to derive business value out of AI.
> There is no ceiling of how much performance and value can be squeezed out of a software system
That may be true. But what’s the cost benefit analysis? Should we all be programming in assembly? Should game developers write a custom bespoke game engines and optimize them for each platform?
> That’s it, there is no other reason for a company to pay anyone except to bring more value to the company than they cost to employ them.
The implicit part is that if you engineer a good system then it saves money with less bugs and breaks less, and also makes money by allowing faster development and iterations.
There are plenty of examples here. I could point at how the PlayStation network just went down for 24 hours, or how UIs are often still very laggy and buggy, or I can also point at companies like Vercel that are (I assume) very valuable by providing a convenient and easy way to deploy applications... the fact that there are many SaaS out there providing convenience of development proves that this adds value. Despite this businesses are not having their engineers do this in-house because somehow they don't see the immediate ROI for their own business. I would just call that lack of vision or creativity at the business level, where you can't see the value of a well engineered system.
Businesses are free to run their company in whichever way they please, and they can create crappy software if it makes them money, but the point is that when this is industry-wide it cripples the evolution of software and this is then felt by everyone with downtimes and bad experiences, even though hardware is unbelievably fast and performant.
> Should game developers write a custom bespoke game engines and optimize them for each platform?
This is a good example actually. Most companies that want full creative control are making their own engines. The only exception here is Unreal (other smaller engines are not used by large companies), and from what I can tell the Unreal engine is an example of great software. This is one of those exceptions where engineers are actually doing engineering and the company probably can't afford to have them do something else. Many companies could benefit from this, but it's just not as straight line from the engineering to profit and that's kind of the root of why there is so much bad software out there.
Right, but it's an example that illustrates a broader problem with how software still has the same issues it had for decades, especially in network programming and web. In contrast hardware has made huge advancements.
I wonder in comparison what had more impact, a software made in the 80s still used today, or an app that will be replaced in a year, with an added note of "I don't know why I'm using it at all".
If the company didn’t make a continuous stream of income from the product, from the company’s perspective, it would be that awesome pay to win game with in app purchases…
The users often feel the "impact" like a punch in the gut, as their perfectly-fine software has now grown a useless AI appendage, or popup ads for an upsell to the next pricing tier. But hey, got the promotion!
Yes with promotions, bonuses and overall good reviews, which are only marginally based on technical achievements (when it should be absolutely based on that).
Yes, that's part of the reward, but normally you get raises, bonuses, etc. based on yearly or quarterly reviews. If you are not entry level, and you just focus on programming while getting a good pay, and you are not seen as underperforming, consider yourself lucky.
If you want a raise or promotion, you should talk to your boss or otherwise figure out how to remind them regularly of your value. Ask them for a quarterly or monthly one-on-one, and take an interest in what they do and what the team priorities are. I don’t know about your boss, they’re not all the same, but managers tend to like see initiative, engineers who make other engineers more productive, and engineers who have and spread an optimistic attitude. Promotions are about taking greater responsibility.
An alternative but dangerous approach is to make it known you’re looking elsewhere for work. Don’t do that if it’s relatively easy to replace you, and definitely assume the management thinks it’s easy to replace you, especialy if you haven’t been talking to your boss. ;) But there is the chance that they know you’re valuable and haven’t given you a raise because you seem content and they believe they have the upper hand - which may or may not be true.
The problem is that “your boss” usually has to conform to the budget set by their manager working alongside HR.
When I left a company for increased compensation, which funny enough has only been 3x in almost 30 years across 10 jobs, it’s been between a 25%-60% raise. It’s almost impossible for any manager to push that kind of raise for anyone without a promotion.
Even at BigTech promotions usually come with lower raises than if you came in at that level.
Don’t do that if it’s relatively easy to replace you, and definitely assume the management thinks it’s easy to replace you, especialy if you haven’t been talking to your boss.
Everyone is replaceable. If you are at a company where everyone isn’t replaceable, it’s a poorly run company with a key man risk. I never saw any company outside of a one man company or professional practice where one person leaving was the end of the company.
> The problem is that “your boss” usually has to conform to the budget set by their manager working alongside HR.
Very true! Though isn’t it also normal in that case for HR to be recommending inflation raises at least? The exception might be if you came in at a salary that’s higher than your peer group and/or high for the title range. Parent’s problem could be that - either peer group correction or not possible for manager to raise at all without a promotion by company rules. There’s lots of reasons I can imagine, but in any case I wouldn’t expect a change with status quo, right? If you haven’t been talking to your boss, continuing to not talk to your boss is unlikely to change anything.
It sounds like the combination of a poorly run company and a lack of initiative on the commenter’s side if they haven’t even spoken to their manager in two years.
But yeah, the company should be doing at least inflation raises. A company only has at most two full review cycles to not get me somewhere in the range I think I should be making before I start looking for another job.
But I do understand that it is a shit show out here right now. I was looking for bog standard enterprise dev jobs with AWS experience as a “Plan B” while waiting for the “Plan A” interviews to work their way through. I have never seen anything like this in almost 30 years.
It was not this hard for me to get software developer job interviews in either 2000-2001 or 2008-2010. Admittedly, that’s partially because I was only looking for remote jobs and there the competition is fierce.
Three years into my career and my total comp hasn’t risen once - it’s honestly exhausting at this point but I’m still learning a ton, feel as if there is solid job security, and love my team. At some point I’m going to have to make that scary jump though if it continues.
That’s fair. There have been plenty of points in my career where I chose leveling up over compensation and even chose a lower offer between two offers because it would prepare me for my n+1 job better.
Bu but...but..look at how elegant my rewrite of this part of the code that nobody uses for a product that has a handful of users is../s
There's really way too many developers that care way more about the code than the product, let alone the business.
It ends up like fine dining, where 99% of the times a big Mac would've been tastier and made the customer happier, but wouldn't justify the effort and price.
Why are they hiring the candidate to begin with? This entire "justify your existence every quarter" is one of the bigger downsides of how the economy works, because it allows people to be exploited. it's the metaphorical writing of the blank check. There are no real boundaries of when work starts and stops or when your responsibilities end because the task is to increase business value and that is technically endless. The company keeps piling more and more work, keeps growing and growing, meanwhile they reward you with a small fraction of the total profit that you contributed.
I keep hearing how 10x engineers make their companies millions upon millions but they only get paid a fraction of that. How does that even make sense as a fair exchange? Not to mention is completely unfeasible for most people to have this kind of impact... it is only possible for those in key positions, yet every engineer is tasked with this same objective to increase company value as much as possible. There's just something very wrong with that.
Because they feel they can extract more value from them then they are paying them.
There are no real boundaries of when work starts and stops or when your responsibilities end because the task is to increase business value and that is technically endless
I work 40 hours a week and they pay me the agreed upon amount. There was nowhere in our agreement the expectation of my working more than that. I also knew that they could put me on a plane anytime during the week.
The company keeps piling more and more work, keeps growing and growing,
That’s completely on the employee to communicate trade offs between time, cost and requirements. My time is fixed at 40 hours a week. They can choose to use my 40 hours a week to work with sales and the customer to close a deal, be a project manager, lead an implementation, be a hands on keyboard developer, or be a “cloud engineer”. It’s on them how to best use my talents for the amount of money they are paying me. But seeing that they pay my level of employee the highest of all of the ICs, they really should choose to have me working with sales and clients to close deals.
That’s not bragging. I make now what I made as a mid level employee at BigTech in 2021.
I keep hearing how 10x engineers make their companies millions upon millions but they only get paid a fraction of that. How does that even make sense as a fair exchange?
The concept of a 10x engineer except in very rare cases is a myth if you think of them as just being on a keyboard everyday. All of the things I listed I could do - project management, backend developer or a cloud engineer - I would say I’m only slightly better than average if that. My multiplier comes because i can work with all of those people and the “business” and they can put me on a plane or zoom call and I can be trusted to have the soft skills necessary and my breadth is wide enough to know what needs to be done as part of a complex implementation and how to derive business value.
If you are making a company millions of dollars and you are only getting a fraction of that - and I doubt someone is doing that on their own without the supporting organizational infrastructure - it’s on you to leverage that to meet your priority stack.
Not to mention is completely unfeasible for most people to have this kind of impact... it is only possible for those in key positions, yet every engineer is tasked with this same objective to increase company value as much as possible. There's just something very wrong with that.
If you are a junior developer, there isn’t much expected of you, you are told what to do and how to do it. You aren’t expected to know the business value.
If you are a mid level developer, you are generally told the business objective and expected to know best practices of how to get there and understand trade offs on the epic/work stream level.
If you are a “senior” developer, now you are expected to understand business value, work with the stakeholders or their proxy, understand risks, navigate XYProblems on the project implementation level and deal with ambiguity.
As you move up the more “scope”, “impact” and “dealing with ambiguity” you have to be comfortable with.
“Codez real gud” only comes into play in getting from junior to mid.
Right, being senior and above basically means doing less engineering. I hope that's something we can agree on because otherwise we would need to discuss the semantics.
> All of the things I listed I could do - project management, backend developer or a cloud engineer - I would say I’m only slightly better than average if that
I completely acknowledge this is a valid way to run a business, but the context here is how this sort of career progression is preventing the specialization of engineers in their domain and contributing to the widespread of software problems. Instead of investing in good engineers that specialize in their domain, companies move them away from engineering into more of an entrepreneur mindset by tasking them with adding value to the business directly, which is not something that you do as an engineer (it's nowhere in a CS degree, aside from say some electives).
A good metaphor here is a football/soccer team. What companies are doing is telling the goal keeper that he needs to score goals because more goals means winning. The team wants to win so everyone on the field has to score a goal. That obviously doesn't make sense even though the premise is true. You want a team composed of specialists and the more they specialize in their domain and work together the more you win. Even though there are only two or three offensive players that are scoring the goals, everyone is contributing to the success of the team if they specialize in their domain. Similarly, just because talking to clients and selling the product is directly contributing to the revenue of a business it doesn't mean that engineering at a higher level has no value.
And once again to stress the context here, companies can do whatever they want, but having engineers progress through their careers by moving AWAY from engineering is precisely why there is so much bad software out there. Letting engineers create better software should result in more profit in the long term, just probably not in the short term, and it's also hard for non-technical people to manage. So it is what it is.
Right, being senior and above basically means doing less engineering. I hope that's something we can agree on because otherwise we would need to discuss the semantics
Engineering is not only the physical labor. Aircraft engineers and building engineers don’t spend most of their time doing hands on work.
Designing and Planning: Engineers are responsible for designing and planning systems, structures, processes, or technologies. They analyze requirements, gather data, and create detailed plans and specifications to meet project objectives. This involves considering factors such as functionality, safety, efficiency, and cost-effectiveness.
When doing construction work, who adds more value?
The general contractor? (Staff software engineer),
The owners of the plumbing, electrical, and HVAC companies assuming they have the actual skills (senior level developers). The owners of the plumbing companies could very well be making more than the general contractors. This is where you can specialize and the sky is the limit.
The actual certified workers (mid level developers). This is the level that the completely head down people are. No matter how good they become at being a hands on plumber , there is a hard ceiling they are going to hit at this level.
The apprentices (juniors)?
I work in consulting. I am currently a staff architect (true - IC5) over a hypothetical project (not a real project). I know the project is going to need a cloud architect, a data architect , and a software architect. They are all specialists at their jobs and are all going to lead their “work streams”. They are all IC4s
I expect each architect to take the high level business objectives and work with the relevant technical people on both sides and lead their work along with some hands on keyboard work.
They will each have people under them that are not customer facing at all. While I know all of the domains at some level, I’m going to defer to their technical judgement as long as it meets the business objectives. I did my high level designs before they came on to the project. Was my design work, figuring out priorities, risks, making sure it met the clients needs, discussing trade offs, etc not “engineering”?
Each level down from myself IC5 to junior engineers (IC1) is dealing with less scope, impact and ambiguity. There is no reason that the architects shouldn’t get paid as much as I do. They bring to the table technical expertise and depth. I bring to the table delivery experience, being able to disambiguate, and breadth.
> Do you think “engineering” is only the physical labor? Do aircraft engineers and building engineers actually do the construction work?
No, but software is inherently different because you can leverage existing software to create more software. Every airplane has to be created individually, but software that already exists can be extended or reused by just calling functions or in the worst case copy/pasting.
> The actual certified workers (mid level developers). This is the level that the completely head down people are. No matter how good they become at being a hands on plumber , there is a hard ceiling they are going to hit at this level.
Yes, with hardware this can be the case as there is a small number of ways to build something. With software there is no ceiling, and the proof here is AI. We might soon see general intelligence that just codes anything you want. This means software can be designed to automate virtually anything in anyway shape or form, but it requires more and more expertise.
> I did my high level designs before they came on to the project. Was my design work, figuring out priorities, risks, making sure it met the clients needs, discussing trade offs, etc not “engineering”?
I agree what you're outlining is how the industry works. Perhaps the core of the issue here is how software engineering was modeled after other kinds of engineering with physical limitations. Software is closer to mathematics (arguably it's just mathematics). You can of course still design and plan and delegate, but once the role starts dealing with the high level planning, scheduling, managing, etc., there is less of a requirement for the technical details.
I've worked with architects that didn't know the specifics of a language or design patterns, not because they're bad at engineering but because they had no more time to spend on those details. These details are crucial for good software that is reliable, robust, extensible, etc. Junior and even mid level engineers also don't know these details. Only someone that has been hands on for a long time within a domain can hone these skills, but I have seen so many good engineers become senior or tech leads and then forget these details only to then create software that needs constant fixing and eventually rewriting.
I'm a senior myself and have no choice but to engage in these activities of planning, scheduling, etc., when I can clearly see they do not require technical expertise. You just need some basic general knowledge, and they just are time consuming. My time would be better spent writing advanced code that mid-level and junior level can then expand on (which has happened before with pretty good success, accelerating development and eliminating huge categories of bugs). Instead I have to resort to mediocre solutions that can be delegated. As a result I can see all kinds of problems accumulating with the codebase. It's also really hard to convince the leadership to invest in "high level" engineering because they think that you create more impact by managing an army of low to mid-level engineers instead of leveraging properly written software. I'm convinced that it does add value in the long term, it's just a hard sell. Ultimately I guess it comes down to the type of org and the business needs, which often does not include writing software that will not break. Most companies can afford to write bad software if it means they get to scale by adding more people.
> No, but software is inherently different because you can leverage existing software to create more software.
That’s true. But when I put my “software engineering”, “cloud engineer”, or “data engineer” (theoretically) hat on, I can only do work of one person. No matter how good I am at any of it, I won’t be producing more output than someone equally qualified at my own company. Distributing software does have zero marginal cost more or less and that’s why we get paid more than most industries.
but I have seen so many good engineers become senior or tech leads and then forget these details only to then create software that needs constant fixing and eventually rewriting.
This is just like both my general contractor analogy and my real world scenario. As an “staff architect”, I come up with the initial design and get stakeholder buy in. But I defer to the SMEs the cloud architect, the data architect and the software architect who still eats, sleep and breathe the details in their specialty.
Just like the owners of the plumbing company, HVAC company and electrical company are the subject matter experts. The general contractor defers to them.
In the consulting industry at least, there are two ways you can get to the top, by focusing on depth or breadth. But in neither case can you do it by being staff augmentation (the plumber, electrician, or HVAC person), you still have to deal with strategy.
> You just need some basic general knowledge, and they just are time consuming
Knowledge isn’t the issue, it’s wisdom that only comes with experience that I assume you have. Going back to the hypothetical large implementation. It involves someone who does know architecture, development and data. No matter how good your code is, high availability, fault tolerance, redundancy, even throughput comes from the underlying architecture. Code and hands on implementation is usually the least challenging part of a delivery.
Its knowing how to deal with organizational issues, managing dependencies, sussing out requirements, etc
I can grant all of this makes sense for contractors, given that it's not building a career within a single company. I was more talking about full time, long term careers within a single domain.
> No matter how good your code is, high availability, fault tolerance, redundancy, even throughput comes from the underlying architecture. Code and hands on implementation is usually the least challenging part of a delivery.
I agree to a certain extent. Unless the solution is short lived, designing the code to scale properly is extremely important, and this happens at the implementation level where details of how code is being added makes all the difference. I can see how contractors might not be thinking about this, but it's a recurring pattern with legacy systems. Additionally with software, implementation gets automated away if done correctly, so there is only an initial overhead that pays back as the codebase grows larger and larger.
I'm not saying there is no need for engineers that deal with org and management issues, but rather that companies underestimate the need for senior level engineers focusing on the software implementation details, only to pay the price as the codebase inevitably accumulates complexity later as it scales in size.
Anyway, great discussion, I appreciate your input and will be considering it more.
Look, I get that HN is startup-focused, but many companies DO drown themselves in tech debt of the sake of the business case.
Then a few years pass and they are unable to deliver features quickly enough - they can't keep Sr. engineers because their stack is a soul-sucking, career-killing tarpit. The Jr. engineers don't know how to fix the code and don't even see anything wrong with it because it's all they've known.
It's totally fine if you're a megacorp - Salesforce and so on can hire JUST ENOUGH high-level folks to keep the system manageable. It's also fine if you're so small that all your work is greenfield.
The type of company that I'm describing is generally Series D/E and probably worth a few 100M dollars.
I’m not saying don’t care about tech debt and well architected systems. I’m saying you have to tie that to business goals. If you can’t explain to a manager the business purpose of why you are doing something then does it really need to be done?
Your reasoning for tackling tech debt is sound. “We are moving like molasses. Our foundation is wobbly and we can’t build new features until we start tackling $X tech debt. These are the reasons that it will help us in the future.”
But on the other hand, if I achieve x, y, and z or someone else achieve it and we are told it didn’t achieve enough “impact”, yeah I am going to do like Google employees do and decide to work on the 5th messaging app.
I've been way too close way too often with lisp and Haskell or many other niches (I know well both Racket/Scheme and Haskell btw) the people that care that much about this correct and reliable and extensible software care about the code more than they care about the products.
That's why the languages that breed creativity and stress correctness have very little, if any, killer software to show when PHP/Java has tons of it.
I think you're conflating two different things. On the one hand you have how a product is made, and on the other there is the demand for it which will affect how the product is made. In the case of software these two things are unfortunately broadly disconnected for a number of reasons.
First, hardware has improved consistently, outpacing any need for optimal software. Second, the end user does not know what they really want from software until it is provided to them, so most people will easily put up with slow, laggy and buggy software until there is a competitor that can provide a better experience. In other words, if companies can make money from making bad software, they will, and this is often what happens because most people don't know any different and also hardware becomes more and more powerful which alleviates and masks the defects in software.
I think there is also another big factor, which is that businesses prefer to scale by adding more people, rather than by making better software. Because they want to scale along human engineers, they tend to prefer low-tier software that is easy to pick up by the masses, rather than the high-level specialized software that is a hard skill to acquire. This is understandable, but is also the reason why software is so slow in advancing forward compared to hardware (and by the same token, hardware engineers require more specialization).
What is wrong with frequent changing of jobs? It’s one of the easiest tools for increasing my compensation. The job market ideally should be so flexible you can switch to another company any time you want, no noncompetes.
That is the problem. That job-hopping is only way to get better compensation. The history, the domain knowledge and accountability is lost when one who made a programming decision is gone.
Why care about quality or maintainability if you are gone in year or two anyway...
You've hit the nail on the head -- the problem is systemic, not the implied sudden lack of virtue. People job hob in our industry b/c even the giants with hundreds of billions in the bank are caught up in absurd quarter-by-quarter performances for their shareholders, which sets the direction of the whole industry.
Exactly. On the level the parent is describing, having options is a good thing, but the side-effect of this systemic problem is that it fragments our ability to write good software because the process is not about developing good software, its about to profit. Which enshitifies the endeavor, devaluing the quality of the end result.. Slowly, over time, this enshifitication is going to, or already is causing problems in our technological infrastructure and maybe other areas of society.
The topic is “we’re destroying software”, not “we’re destroying techniques to increase your compensation”. Individual compensation and collective quality are not somehow inherently correlated.
I guess you could argue something along the lines of people never staying long enough to build complex things from start to finish. New people moving in and working on the project without the proper understanding, not caring since it will be someone else's problem in a few months...
It's harder to learn the impact of your design decisions -- Seeing how software evolves to meet changing business goals and how the design choices made play out in the long run helped teach me a lot.
Coming up with a neat API that turns out to be difficult to modify in the future, or limiting in ways you didn't imagine would when writing it is a good learning experience.
Or seeing how long a system can survive growing usage -- Maybe a simple hack works better than anyone expected because you can just pay more for RAM/CPU each year rather than rebuild into a distributed fashion. Or the opposite, maybe there's some scaling factor or threshold you didn't know existed and system performance craters earlier than predicted.
I don't get the downvotes. This is a rational point of view for an individual. The problem is higher up, where the incentives align to make it rational. It'd be better if people could stay longer and still grow their compensation.
I was just chatting with a friend of mine, this morning, about this kind of thing.
He works as a highly-skilled tech, at a major medical/scientific corporation. They have invested years of training in him, he brings them tremendous value, and they know it. He was just telling me how he used that value to negotiate a higher compensation package for himself. Not as good as if he swapped jobs, but he really has a sweet gig.
People who stay, take Responsibility for the code they write. They will need to face the music, if it doesn't work, even if they are not responsible for maintaining it.
They are also worth investing in specialized training, as that training will give great ROI, over time.
But keeping skilled people is something that modern management philosophy (in tech, at least) doesn't seem to care about.
Until corporations improve the quality of their managers; especially their "first-line" managers, and improve their culture, geared towards retaining top talent (which includes paying them more -but there's a lot more that needs doing), I can't, with good conscience, advise folks not to bounce.
> But keeping skilled people is something that modern management philosophy (in tech, at least) doesn't seem to care about.
I’m a founder for 10 people and this is the first thing we think about. Except for low performers; except that youngsters need a variety of experience to be proficient at life; except that the team is not performing well(1). 25% or 30% increases for half the workforce are frequent.
(1) The biggest remark from management coaches is that giving raises lowers employee performance, which I can fully witness in my company. It’s not even good for morale. I’m just happy that people exit the company fitter and with a girlfriend, even a kid and sometimes a permanent residency, but business-wise I’ve been as good as a bad leader.
I’m reaching the sad conclusion that employees bring it upon themselves.
Paying more is not the answer (but it is also not not the answer -we need to pay well). Improving the workplace environment, into a place people want to stay, is important.
My friend could double his salary, moving almost anywhere else, but he gets a lot of perks at his work, and is treated extremely well by his managers.
They just gave him a rave review, and that did more to boost his willingness to stay, than a 10% raise. He will still negotiate a better salary, but is more likely to be satisfied with less than he might have, if they tried to treat him badly.
Treating employees with Respect can actually improve the bottom line. They may well be willing to remain in difficult situations, if they feel they are personally valued.
I know this from personal experience. When they rolled up my team, after almost 27 years, the employee with the least tenure had a decade. These were top-shelf C++ image processing engineers, that could have gotten much higher salaries, elsewhere.
Yeah I agree with this. It really depends on the culture of the business. If the employee feels valued, giving raises increases feeling valued.
The problem is our current form of corporate culture. Employees don't feel like they matter, there efforts are a cog in a wheel. If you get a raise in this type culture, it only matters to the bottom line and there is no incentive produce because the employee is already unhappy in the first place.
Change your business culture and these problems will disappear, IMHO.
Is there any alternative to raises that don't lower employee performance, like some kind of bonus scheme? Or do you find some unfortunate relation between money vs motivation?
Truly unlimited PTO where you judge employees by their performance. I just spoke to my manager at the job I started late last year and he said it is customary for people to take 5-6 weeks kid a year.
Sounds like he's lucky though. Many companies are happy to let you specialize in their area of business, own the special software, get to know all the vendors & business contacts then really not pay you well. You dont get to find out until 5 years in when you have skills that aren't really transferable to a new job.
His skills are quite transferable. Many of the company’s customers would love to hire him away. The company doesn’t have a noncompete, and he’d probably quit, if they tried.
> But keeping skilled people is something that modern management philosophy (in tech, at least) doesn't seem to care about.
If they would care then job hopping would not exist. If staying at s company would be more. Beneficial to your salary, why would you ever want to change company, if you are otherwise happy?
> Not as good as if he swapped jobs, but he really has a sweet gig
If your main motivation for working is to exchange your labor for the maximum amount of money possible, I don’t see how that is the positive outcome you think it is.
I personally wouldn’t leave my current job if another one for $100K more fell into my lap. But the “unlimited PTO” where the custom is to take at least 5-6 weeks off during the year not including paid holidays and it being fully remote is hard to beat.
> It’s one of the easiest tools for increasing my compensation.
This is the root problem. None of the problems the GP pointed up were created by software developer.
Now, if you want to know the consequences, it causes an entire generation of people that don't really know what they are doing because they never see the long-term consequences of their actions. But again, it's not the software developers that are causing this, nor are they the ones that should fix it.
Most of what is said are symptoms not causes.
Leetcode interviews: lack of continuous certification and changing toolsets too fast
Frequent job hopping: lack of pay upgrades because software is considered a cost center
I could go on but in reality it’s a disconnect between what business thinks the software is worth as opposed to what the engineer wants to do with it.
You can say software is an art but commodity art doesn’t make much money. In reality, the ad driven software has greatly inflated salaries (not complaining but it’s reality). Now it’s going to be an ai bubble. But your rank and file business doesn’t care what software bubble is happening but unfortunately they are bound by the costs that come with it.
Have you seen the process that happens in defense or medical equipment industries. You probably won’t complain.
Why is LeetCode a symptom of a lack of continuous certification or changing toolsets? I ask because LC is about neither of those things. I agree LC is a symptom of something but I think it’s something else. I also don’t think ad driven software has inflated salaries, there are many, many more software companies than ad based ones, it even only compromises at most half of FANG, which is hardly the only game in software. For defense and medical, these are places where software is not tertiary concerns.
If you didnt have degree requirements and certification bodies for:
* accountants
* engineers
* doctors
* lawyers
What do you think hiring might look like?
Do you think they would build a hiring process to validate to the best of their ability your aptitude of the core fundamentals- except worse than certification and education bodies?
There is a bit of a difference between other professions and software. You wouldn’t hire an orthopedic surgeon for cardiology. Now, the human body doesn’t change that fast. So both are needed. In software the rate of change is much faster. So what happens is tools change and people who want to switch streams for better opportunity have to tweak resumes. Now the only way to validate basic proficiency comes down to leetcode style interviews - for better or worse. It’s pretty much the only common denominator between an interviewer and the candidate.
What makes you think certifications can’t be gamed? Brain dumps have been a thing since at least 2008 and just like you can have a dozen AWS certifications, it tells you nothing about whether they could actually be productive in the real world.
I'm just pointing out that certifications don't have to be meaningless. If somebody wants to use the title "Software Engineer", then perhaps we should require them to be actual professional licensed engineers.
Having spent quite a bit of time around a couple of those groups, I find most of those degree requirements and certifications as just ways to increase salaries, more than ways to increase quality. Many people pass state bars and are incompetent. Lawyers that go through residency get lazy and kill patients, and they aren't magically superior to someone that isn't allowed to work in, say, the US, because their medical training was done the wrong country.
Realistically, licensing boards are there to protect their members, and rarely do political things against people in the same body with unpopular opinions. You have to be catastrophic for most boards to do anything about you: Just like a police union will defend a union member that has committed gross negligence unless the evidence is public.
When you hire a doctor for something actually important, you don't look at the certification body: You look at long term reputation, which you also do in software. Only the largest of employers will leetcode everyone as a layer of fairness. In smaller employers, direct, personal references replace everything, which is what I'd go with if I needed an oncologist for a very specific kind of cancer. The baseline of aptitude from the certification body doesn't matter there at all.
> Many people pass state bars and are incompetent. Lawyers that go through residency get lazy and kill patients, and they aren't magically superior to someone that isn't allowed to work in, say, the US, because their medical training was done the wrong country.
So we need the barrier to entry to be even lower for such professions that deal with life-changing outcomes? I don't think so. In such high risk fields: "long term reputation" is totally dependent on hiring extremely qualified individuals.
The barrier to entry MUST be continuously raised with the bare minimum requirement of a degree. Only then the secondary requirements can be considered.
> When you hire a doctor for something actually important, you don't look at the certification body: You look at long term reputation, which you also do in software.
I don't think you can compare the two. Since one deals with high risk to the patient such as life and death and the other in most does not. (Unless the software being written deals with safety critical systems in a heavily regulated setting.)
From what you are saying, maybe you would be OK consulting a surgeon or an oncologist that has never gone to medical school.
Software may be important in defense and medical but I don't think this is reflected in how software engineering is done or how software engineers work in those industries.
I can only hope that the tech stack for server-served apps will be much more sane in a couple of decades (for the developers of the future; it’ll be too late for me). We are really in a very bad place now in that regard.
I think people vastly underestimate "the wheel". The wheel was something recreated independently over and over again for thousands of years across human history. Even your favorite programming language or web framework is not comparable to "the wheel".
Holy fuck. I internally explode in anger every time I hear Don’t reinvent the wheel.
To me this is the most sure way to identify the adults from the children in the room. People that can actually program, strangely enough, aren’t bothered by programming.
He is exactly right. Complexity is eating the world. Faster CPUs, larger memories and faster networks have brought great capability -- but removed the natural governor on software complexity.
- never ending upgrades
- unstable ui, unstable ecosystems
- fix it later
- shallow business models (subscriptions, freemium, adinfested)
- social everything (it used to be that my computer was a little escape space where and when i wanted, now i'm tethered to the web noise)
642 comments
[ 4.1 ms ] story [ 324 ms ] threadWith each new language and new library, there's a chance to do it better. It's quite a wasteful process, but that's different from things getting worse.
Many of these have been an issue since I started programming many decades ago on minis. Seems as the hardware gets more powerful we find ways to stress out the hardware more and more. This making your list true to a greater extent year after year.
BTW, I like your WEB Site format a lot!
'We are destroying software telling new programmers: “Don’t reinvent the wheel!”. But, reinventing the wheel is how you learn how things work, and is the first step to make new, different wheels.'
and this
'We are destroying software pushing for rewrites of things that work.'
But generally speaking I grok it.
The problem, IMO, is globally applying either rule.
And sometimes the reverse problem is what is destroying software: being unwilling to push for rewrites of things when they desperately need one. I think we may have over indexed on "never rewrite" as an industry.
> We are destroying software mistaking it for a purely engineering discipline.
And I'm also seeing this the other way: engineers are beginning to destroy non software things, because they think they can blindly apply engineering principles to everything else.
Arguing against 10x complexity & abstractions, with 1x coding :-)
Sometimes it is valid to not reinvent the wheel. Sometimes wheel needs to be reinvented to learn. Both actions are done. Sometimes the decision was right. Sometimes not.
Overall as a whole we are creating things, more than we are destroying. I don't see the need to take a negative stance.
Fair point: each one of us can think about the balance and understand if it's positive or negative. But an important exercise must be accomplished about this: totally removing AI from the complexity side.
Most of the results that neural networks gave us, given the hardware, could be recreated with a handful lines of code. It is evident every day that small teams can rewrite training / inference engines from scratch and so forth. So AI must be removed from the positive (if you believe it's positive, I do) output of the complexities of recent software.
So if you remove AI since it belongs to the other side, the "complicated software world" what gave us, exactly, in recent times?
AI has the potential to make the situation much worse, as many laypeople confer it an air of "authority" or "correctness" that it's not really owed. If we're not careful, we'll have an AI-driven Idiocracy, where people become so moronic that nobody can do anything about the system when it takes a harmful action.
It needs to be noted that the average person's lot didn't improve until 150 years later. There's no reason why technology can't be decided by democratic means rather than shoved in our faces by people that just want to accumulate wealth and power.
Writing a new green field project using 10.000 npm dependencies for an electron based front end is shockingly easy. But how do you keep that running for the next 15 years? Did the project need npm? Or a web browser? How do all the layers between the lamguage of choice and the bare metal actually behave and can you reason about that aggregate accurately?
The field has come to a point where a lot of projects are set up with too many complexities that are expedient in the short term and liabilities in the long term.
The current generation of junior devs grows up in this environment. They learn that these mistakes as "the right thing to do" when they are questionable and require constant self-reflection and reevaluation. We do not propagate a hacking culture enough that values efficiency and simplicity in a way that leads to simple, efficient, stable, reliable and maintainable software. On a spectrum of high quality craftsmanship to mass-produced single use crap, software is trending too much to the latter. It's always a spectrum, not a bunary choice. But as a profession, we aren't keeping the right balance overall.
I definitely use npm (or rather pnpm) because I know it will allow me to build whatever I want much faster.
How much complexity is actually required? What changed in software in the last 20 years so that the additional bloat and complexity is actually required? Hardware has become more powerful. This should make software less reliant on complicated optimizations and thus simpler. The opposite is happening. Why? What groundbreaking new features are we adding to software today that we didn't 20 years ago? User experience hasn't improved that much on average. In fact, measurements show that systems are responding more sluggishly on average.
Intrinsic complexity of the problems that software can solve hasn't really changed much as far as I can see. We add towers of accidental complexity on top that mostly aren't helpful. Those need to be questioned constantly. That isn't happening to the extent that it should. Web-based stuff is the poster child of that culture and it's hugely detrimental.
Backends handling tens / hundreds of thousands or more of concurrent users rather than locally deployed software on a single machine or a small server with a few 10s of users?
Mobile?
Integration with other software / ecosystems?
Real time colaboration amoung users rather than single user document based models?
Security?
Cryptography?
Constant upgrades over the web rather than shipping CDs once a year?
I'll pass on AI for the moment as it's probably a bit too recent.
Software can be distributed onto client machines and be kept up to date. That was first solved with Linux packages managers more than 25 years ago.
Before mobile we had a wide range of desktop operating systems with their own warts.
TLS 1.0 was introduced in 1999. So cryptography already a concern back then.
So what is really new?
This is a false equivalency; no one is suggesting that it’s necessary or even useful to understand electron drift to write programs. It can, however, be extremely useful to understand memory allocation and retrieval, and how your languages’ abstractions over it work.
Take UUID generation, for example. I needed to generate millions of them for a tool I’m building, and I found that it was massively slower on Linux than Mac, even taking raw CPU performance into account. I eventually tracked this down [0] to BSD using arc4random, while until a fairly recent glibc release, Linux was using ul_random_get_bytes. Most people are never going to need or notice this difference, because for a single UUID, it simply doesn’t matter. But it did for me, and the only reason I was able to figure it out and solve it was by having a passable understanding of implementations buried behind abstractions.
[0]: https://gist.github.com/stephanGarland/f6b7a13585c0caf9eb64b...
I started a job in manufacturing a few months ago and having to think that this has to work for the next 20 years has been a completely different challenge. I don't even trust npm to be able to survive that so web stuff has been been an extra challenge. I landed on lit web components and just bringing it in via a local CDN.
A few of them aren’t decisions any individuals have control over. Most coders aren’t jumping onto new languages and frameworks all the time; that’s an emergent group behavior, a result of there being a very large and growing number of programmers. There’s no reason to think it will ever change, nor that it’s a bad thing. And regardless, there’s no way to control it.
There are multiple reasons people write software fast rather than high quality. Because it’s a time/utility tradeoff, and time is valuable. It’s just a fact that software quality sometimes does not matter. It may not matter when learning or doing research, it may not matter for solo projects, it may not matter for one-off results, and it may not matter when software errors have low or no consequence. Often it’s a business decision, not an engineering decision; to a business, time really is money and the business wants engineering to maximize the utility/time ratio and not rabbit hole on the minutiae of craftsmanship that will not affect customers or sales.
Sometimes quality matters and time is well spent. Sometimes individuals and businesses get it wrong. But not always.
I think the situation (loosely, wasting time spinning your wheels by modernizing things that are actually fine the way they are and may be made worse by adopting "hot new thing") looks worse when you see it through that lens than it actually is throughout the industry as a whole. There are plenty of opportunities for modernization and doing some of the things described in this article that actually make some sense when applied to appropriate situations.
In other words, I totally understand the vibes of this post, it's one of the reasons I don't work in the parts of the industry where this attitude is most prevalent. I would never feel the push to write a post like it though because the poster is I think being a bit dramatic. At least that's the case looking at the industry from my (id argue broader) vantage point of being an expert in and working quite a bit with "legacy" companies and technologies that maybe could stand to have a new UI implemented and some tasteful modern conventions adopted at least as options for the end users.
This post reads as a description of how the wheels come off the wagon if you don't do things well.
With the evolution of AI agents, we'll all be virtual CTOs whether we like it or not, mastering scale whether we like it or not. We need to learn to do things well.
Managerial genius will be the whole ball of wax.
This one packs a lot of wisdom.
Or, more generally, the fact that most of what the software industry produces is much more in line with “art” than “engineering” (especially when looked at from a Mechanical Engineer or Civil Engineer). We have so much implementation flexibility to achieve very similar results that it can be dizzying from the standpoint of other engineering fields. consider
I'm not a masochist; they're often built on top of components, e.g. my IDE uses the Monaco editor. But working on these tools gives me a sense of ownership, and lets me hack them into exactly the thing I want rather than e.g. the thing that Microsoft's (talented! well-paid! numerous!) designers want me to use. Hacking on them brings me joy, and gives me a sense of ownership.
Like an idealised traditional carpenter, I make (within reason) my own tools. Is this the most rational, engineering-minded approach? I mean, obviously not. But it brings me joy -- and it also helps me get shit done.
One example: Newbies shouldn't reinvent the wheel. I think they should use the tools, that are available and common in the given context. When they want to tinker, they should write their own compiler. But they shouldn't use that in production.
Another: Backward API compatibility is a business decision in most cases.
Also, I think it doesn't help to start every sentence with "We are destroying software". This sounds much more gloomy, than it really is.
Why? We should stop saying others how they want to write/use their code ASAP.
Many established technologies are a total shitstorm. If it is ok to use them, it is ok if somebody wants to use their own compiler.
When it comes down to it, whatever works best and is usually the most simple, non-breaking, used to win out. That decision has been disconnected from the value creators to the value extractors. It is impossible to extract value before value is created.
Additionally, programming is a creative skill no matter how hard they try to make it not one. Creativity means trying new things and new takes on things. People not doing that will harm us long term.
Generally speaking, because that’s very likely to end up being “pushing for rewrites of things that work”, and also a case of not “taking complexity into account when adding features”, and perhaps in some cases “jumping on a new language”, too.
This is an imagined scenario, but the likelihood of someone easily replacing a working compiler in production with something better is pretty low, especially if they’re not a compiler engineer. I’ve watched compiler engineers replace compilers in production and it takes years to get the new one to parity. One person tinkering and learning how to write a compiler almost for sure does not belong in production.
For example, my own "Almost C" compiler[1] is 1137 lines of code. 1137! Can it ever reach "parity" with gcc or even tcc? No! That's specifically not the goal.
Do I benefit strongly for having a otherworldly simpler toolchain? hell yeah.
The key is scope, as always. Established projects have, by virtue of being community projects, too wide a scope.
[1]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/doc/co...
Those who do not know history are doomed to repeat it. Or re-re-reinvent Lisp.
There was this anecdote about storm lamp or something. New recruit comes to a camp and sees old guard lighting lamps are turned upside down and lit sideways with a long stick. But he knows better, he tells them and they smirk. First day he lights them the optimal way with a lighter. He's feeling super smug.
But next day he finds the fuse is too short to reach so he takes the long stick...
Few months later, he's a veteran, he's turning lamp upside down using lighter sideways, with a long stick.
And the fresh recruit says he can do it better. And the now old guard smirks.
I'm sure I'm misremembering parts, but can't find the original for the life of me.
If someone would hand me a project, that is full of self invented stuff, for example a PHP project, that invented its own templating or has it's own ORM, I would run. There is laravel, slim or symfony, those are well established and it makes sense to use them. There are so much resources around those frameworks, people who posted about useful things, or packages that add functionality to those. It just doesn't make sense to reinvent the wheel for web frameworks and thousands of packages around those.
Writing software is standing on the shoulders of giants. We should embrace that, and yes one should learn the basics, the underlying mechanisms. But one should make a difference between tinkering around and writing software, that will be in production for years and therefore worked on by different developers.
The JavaScript world shows how to not do things. Every two years I have to learn the new way of building my stuff. It is annoying and a massive waste of resources. Everyone is always reinventing the wheel and it is exhausting. I understand why it is like this, but we as developers could have made it less painful, if we would embrace existing code instead of wanting to write our own.
I’m in games we even rewrite standard libraries (see EASTL) so that they are more fit for purpose.
Of course, it’s your preference. And that is fine. But I don’t think it speaks to the situation in many tech companies.
I strongly disagree. They should, and fail and try again and fail. The aim is not to reinvent the wheel, but to understand why the wheel they're trying to reinvent is so complex and why the way it is. This is how I learnt to understand and appreciate the machine, and gave me great insight.
Maybe not in production at first, but they don't reinvent the wheel in their spare time either. They cobble up 200 package dependency chains to make something simple, because that’s what they see and taught. I can write what many people write with 10 libraries by just using the standard library. My code will become a bit longer, but not much. It'll be faster, more robust, easier to build, smaller, and overall better.
I can do this because I know how to invent the wheel when necessary. They should, too.
> Another: Backward API compatibility is a business decision in most cases.
Yes, business decision of time and money. When everybody says that you're losing money and time by providing better service, and lower quality is OK, management will jump on it, because, monies.
> Also, I think it doesn't help to start every sentence with "We are destroying software". This sounds much more gloomy, than it really is.
I think Antirez is spot on. We're destroying software. Converting it to something muddy and something for the ends of business, and just for it.
I'm all with Antirez here. Software came here, because we developed the software just for the sake of it, and evolved it to production ready where needed. Not the other way around (Case in point: Linux).
Often that "saving money" is just externalizing the cost onto your users. Especially in mobile development. Instead of putting in the tiny amount of effort it takes to continue support for older devices, developers just increase the minimum required OS version, telling users with older hardware to fuck off or buy a new phone.
Another example is when you don't take the time to properly optimize your code, you're offloading that cost onto the user in the form of unnecessarily higher system requirements.
This is why I believe slow cooked software. Works better, easier on the system, and everyone is happier.
Growing up in the 80s and 90s I understand viscerally how you feel, but this take strikes me as willfully ignorant of the history of computers, and the capitalist incentives that were necessary for their creation. The first computer and the internet itself were funded by the military. The PC wouldn't have existed if mainframes hadn't proved the business value in order to drive costs down to the point the PC was viable. Even the foundational ideas that led to computers couldn't exist with funding—Charles Babbage's father was a London banker.
I think a lot of what you are reacting to is the failed promise of free software and the rise of the internet, when the culture was still heavily rooted in 60s counter-culture, but it hadn't crossed the chasm to being mainstream, so it was still possible to envision a utopian future based on the best hopes of a young, humanitarian core of early software pioneers operating largely from the sheltered space of academia.
Of course no such utopian visions ever survive contact with reality. Once the internet was a thing everyone had in their pocket, it was inevitable that software would bend to capitalist forces in ways that directly oppose the vision of the early creators. As evil as we thought Microsoft was in the early 90s, in retrospect this was the calm before the storm for the worst effects of tech. I hear Oppenheimer also had some regrets about his work. On the plus side though, I am happy that I can earn enough of a living working with computers that I have time to ponder these larger questions, and perhaps use a bit of my spare time to contribute something of worth back to the world. Complaining about the big picture of software is a fruitless and frustrating endeavour, instead I am interested in how we can use our expertise and experience to support those ideals that we still believe in.
I take issue with your use of the word "utopian" being used in this context. Its not a lost cause to see the world from the perspective of making the world better, by finding our way though this with a better mindset on the future.
And while you are taking the time to ponder these questions because you earn enough to take the time, the world is burning around you. Sorry if my tone is harsh, but these kinds of statements really rub me the wrong way. It feels like you are saying everything that is happening is how its suppose to be and I am strongly against that. We have enough of that perspective, we really don't need it, IMHO.
This gentle motivation is good, because it allows me to look inside and be rational about my ambitions. I won't go to a blind crusade, but try to change myself for the better.
Because, I believe in changing myself to see that change in the world.
Completely and unjustifiably false.
Agree. That statement/sentiment though doesn't refute the point that it's destroying software.
They absolutely should, or they will never even get to understand why they are using these wheels.
Fun fact, try to question modern web developers to write a form, a simple form, without a library.
They can barely use html and the Dom, they have no clue about built-in validation, they have no clue about accessibility but they can make arguments about useMemo or useWhatever in some ridiculous library they use to build...ecommerces and idiotic crud apps.
So basically they shouldn’t learn the prod systems beyond a shallow understanding?
We're destroying software when we think documentation does not matter.
I was going to phrase it: We are destroying software by neglecting to documenting it and its design decisions.
Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc)
https://www.youtube.com/watch?v=ZSRHeXYDLko
Software technology is in decline despite appearances of progress. While hardware improvements and machine learning create an illusion of advancement, software's fundamental robustness and reliability are deteriorating. Modern software development has become unnecessarily complex, with excessive abstraction layers making simple tasks difficult. This complexity reduces programmer productivity and hinders knowledge transfer between generations. Society has grown to accept buggy, unreliable software as normal. Unless active steps are taken to simplify software systems across all levels, from operating systems to development tools, civilization faces the risk of significant technological regression similar to historical collapses.
Unfortunately my stance is that fundamentally things won't change until we get hit with some actual hardware limitations again. Most devs and people in general prefer a semblance of a working solution quickly for short-term gains rather than spending the actual time that's needed to create something of high quality that performs well and will work for the next 30 years. It's quite a sad state of affairs.
With that said I'm generally optimistic. There is a small niche community of people that does actually care about these things. Probably won't take over the world, but the light of wisdom won't be lost!
It's a self-perpetuating issue: people build stuff saying "it won't/can't last 30 years" for various reasons (money, time, skill, resources, expectations, company culture, business landscape etc). So then software doesn't last 30 years for those same various reasons.
The idea that we think systems did used to last longer is probably some survivor bias. However, software that has survived decades was probably created with a completely different set of methodologies, resources and incentives to modern software.
We swap JS frameworks constantly, but when we’ll reach a good paradigm, we’ll stick with it. At one point, React might be the final framework, or rather, one of its descendants.
Develop a good vm that can either be part of the browser or can be easily launched from it and get all of the browser/OS makers on the same page.
We only have what we have because of a lack of real leadership.
When the c++11 abi break happened it was a big pain in the ass, but once MSVC decided in 2015 that they were going to stop breaking ABI I think it was the stability that c++ needed to fossilize…
Spoken as a C++ fan.
Writing bad code to just get past the next sprint or release is madness.
I'm writing software with the assumption that it'll be used for at least 30 years there, with a lot of guard rails and transparency/observability mechanisms, because I know the next person working there will thank me.
I am aware that capitalism essentially dictates externalising cost as much as possible, but with software- much for the same reason capitalism loves it (a copy is cheap and can be sold at full price despite being constructed just once) means that these externalities can scale exponentially.
Teams in particular is an outlier as in most cases it is essentially forced on people.
It doesn't dictate externalising cost as much as possible unless you have a very short-term view.
Short-term view businesses get eaten pretty quickly in a free capitalist system.
People forget that half of capitalism's advantage is the "creative destruction" part - if businesses are allowed to fail, capitalism works well and creates net value.
Premature optimisation is bad, but there’s now so many devs who don’t do _any _ at all. They don’t improve any existing code, they’re not writing software that is amenable to later optimisation, inefficient architectures and unnecessary busywork abounds.
Are we surprised that years of “product first, bug fixes later, performance almost never” has left us with an ecosystem that is a disaster?
The fact that people don't stay long enough in companies or work on a long project themselves to see the fruits of their labour down the line is a point that is discussed in some of the other comments here in this thread. I agree with it as well. In general if you job hop a lot you won't see the after effects of your actions. And the industry is such that if you want to get paid, you need to move. To reiterate - it's a sad state of affairs.
Users have now been taught that $10 is a lot to pay for an app and the result is a lot of buggy, slow software.
Those big software packages are sold to admins anyway.
Realistically only about 5% or so of my former colleagues could take on performance as a priority even if you said to them that they shouldn't do outright wasteful things and just try to minimize slowness instead of optimizing, because their entire careers have been spent optimizing only for programmer satisfaction (and no, this does not intrinsically mean "simplicity", they are orthogonal).
Disclaimer: Take everything below with a grain of salt. I think you're right that if this was an easy road to take, people would already be doing it in droves... But, I also think that most people lack the skill and wisdom to execute the below, which is perhaps a cynical view of things, but it's the one I have nonetheless.
The reason I think most software can be faster, better and cheaper is this:
1. Most software is developed with too many people, this is a massive drag on productivity and costs.
2. Developers are generally overpaid and US developers especially so, this compounds for terrible results with #1. This is particularly bad since most developers are really only gluing libraries together and are unable to write those libraries themselves, because they've never had to actually write their own things.
3. Most software is developed as if dependencies have no cost, when they present some of the highest cost-over-time vectors. Dependencies are technical debt more than anything else; you're borrowing against the future understanding of your system which impacts development speed, maintenance and understanding the characteristics of your final product. Not only that; many dependencies are so cumbersome that the work associated with integrating them even in the beginning is actually more costly than simply making the thing you needed.
4. Most software is built with ideas that are detrimental to understanding, development speed and maintenance: Both OOP and FP are overused and treated as guiding lights in development, which leads to poor results over time. I say this as someone who has worked with "functional" languages and FP as a style for over 10 years. Just about the only useful part of the FP playbook is to consider making functions pure because that's nice. FP as a style is not as bad for understanding as classic OOP is, mind you, but it's generally terrible for performance and even the best of the best environments for it are awful in terms of performance. FP code of the more "extreme" kind (Haskell, my dearest) is also (unfortunately) sometimes very detrimental to understanding.
Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
> Outside of really edge case stuff like real time low level systems software optimizing performance is not that hard and I've worked with many engineers over my long career that can do it. They just rarely have the incentive. In a few particular areas where it's critical and users are willing to pay software can still command a premium price. Ableton Live is a good example.
This seems like a generous take to me, but I suppose it's usually better for the soul to assume the best when it comes to people. Companies with explicit performance requirements will of course self-select for people capable of actually considering performance (or die), but I don't take that to mean that the rest of the software development workforce is actually able to, because I've seen so, so many examples of the exact opposite.
Aside, he's treated like a celebrity in the game developer niche and I can't understand why.
Depending on what tech you use it can be easier or harder to do as well. I'm making a game with Love2D now, which has made supporting Mac and Linux rather trivial so far, although I've run into challenges with mobile platforms and the web even though it support them (it does work, but it takes more low-level glue code to support native phone features, and the web doesn't seem to be well maintained and my game is throwing webassembly errors currently when I try to run it).
And my previous game (which is on the backburner for now) was made with Monogame, and while that technically has support for Mac and Linux (as well as mobile), I've had quite a few issues even just getting the Mac version working well, like issues with resolution, rendering 3D properly, getting shaders not to break, etc. And they haven't kept up with the latest Mac updates the past few years and have had to make a special push to try to get it caught back up. I've probably sunk a good 20+ hours trying to get it working well before putting that aside to work on the actual game again and I still might have to rearchitect things a bunch in order to get that working.
Meanwhile Unity would probably be pretty dirt simple to port, for the most part, but it comes with other tradeoffs, like not being open source, and trying to pull a stunt a couple years ago where they pulled the rug out from under developers with changing licensing to something aggressive (that convinced other developers to port their games away from the platform), etc.
And there's Godot, which seems to be getting more support again (which is great, I even considered it or my current game, I just like coding in Love2D a bit better), but if you ever want your game on consoles you have to pay a third party to port your games to consoles for you.
The guy you linked makes their own engine (and to be fair, so does Jonathan Blow, who you're critiquing), which is great, but not everyone wants to get that low level. I would rather spend more time focusing on building the games themselves, which is already hard enough, rather than spending all that time building an engine.
It was for that reason that I spent several years focused on board game design instead (as I can make a working game with drawing on index cards and some colored plastic cubes in less than an hour), although that has its own frustrations with significant hurdles to get your game signed by publishers as an unknown designer (I did get one signed four years ago, and it's still not released yet), and large financial risks being made for manufacturing and distribution.
Edit: Also the person you linked to isn't even sure it was financially worth it to support all of those platforms, they just do it for other reasons:
"Do you make your money back? It’s hard to say definitely which could mean no. Linux and macOS sales are low so the direct rewards are also low. ...For Smith and Winston [their current game] right now I don’t think it’s been worth it financially (but it will probably make it’s money back on Launch)"
Imagine if a company had been able to systemize the whims of a paranoid regime, allowing them to track and spy on their citizens with impunity in secret, and the population became so inured to the idea that it became an accepted facet of their culture.
Or what if a political party dominated a state and systematized a way to identify and detect oppositional thought, stamping it out before a counterculture could ever arise. Maybe that thought was tied to a particular religious, ethnic, and/or cultural group.
What if these companies are here today, selling those products to the highest (nation-state) bidders, and the methods they're employing to keep the conceptual wheels turning at scale rely on filtering that selects for candidates who will gladly jump through abstract hoops set before them, concerned only with the how and never the why of what they're doing.
I haven't watched that talk by Blow yet so maybe he covers my concern.
I think you have to be mindful of incentives structures and constraints. There's a reason the industry went down the path that it did and if you don't address that directly you are doomed to failure. Consumers want more features, the business demands more stuff to increase its customer base, and the software developers are stuck attempting to meet demand.
On one hand you can invent everything yourself and do away with abstractions. Since I'm in the embedded space I know what this looks like. It is very "productive" in the sense that developers are slinging a lot of code. It isn't maintainable though and eventually it becomes a huge problem. First no one has enough knowledge to really implement everything to the point of it being robust and bug free. This goes against specialization. How many mechanical engineers are designing their own custom molding machine in order to make parts? Basically none, they all use mold houses. How many electrical engineers are designing their own custom PCB(A) processing machines or ICs/components? Again, basically none. It is financially impossible. Only in software I regularly see this sentiment. Granted these aren't perfect 1-to-1 analogies but hopefully it gets the idea across. On the other hand you can go down the route of abstractions. This is really what market forces have incentivized. This also has plenty of issues which are being discussed here.
One thought that I've had, admittedly not fully considered, is that perhaps F/OSS is acting negatively on software in general. When it comes to other engineering disciplines there is a cost associated with what they do. You pay someone to make the molds, the parts from the molds, etc... It's also generally quite expensive. With software the upfront cost to adopting yet another open source library is zero to the business. That is there is no effective feedback mechanism of if we adopt X we need to pay $Y. Like I said, I haven't fully thought through this but if the cost of software is artificially low that would seem to indicate the business and by extensions customers don't see the true cost and are themselves incentivized to ask for more at an artificially low price thus leading to issues we are currently seeing. Now don't misread me, I love open source software and have nothing but respect for their developers; I've even committed to my fair share of open source projects. As I've learned more about economics I've been trying to view this through the lens of resource allocation though and it has lead me to this thought.
I no longer think that's true. Instead, I think consumers want reliability, but more features is a way to justify subscription pricing segregation and increases.
I play games with known bugs, and on imperfect hardware, because I unwilling to pay more. Some experiences are rare, so I tolerate some jank because there aren't enough competitors.
I’m not even sure if this is true anymore. We got new features foisted on us most of the time.
In game development, whenever we go with highly abstract middleware, it always ends up limiting us in what we can do, at what level of performance, how much we can steer it towards our hardware targets, and similar. Moreover, when teams become so lean that they can only do high level programming and something breaks close to the metal, I’ve seen even “senior” programmers in the AAA industry flail around with no idea how to debug it, and no skills to understand the low level code.
I’ve seen gameplay programmers who don’t understand RAII and graphics programmers who don’t know how to draw a shape with OpenGL. Those are examples of core discipline knowledge lost in the games industry. Aka what we have now, we might not know anymore how to build from scratch. Or at least most software engineers in the industry wouldn’t. It cannot end well.
Building your own in my exp is a better idea — then you can at least always steer it, improve and evolve it, and fix it. And you don’t accidentally build companies with knowledge a mile wide and an inch deep, which genuinely cannot ship innovative products (technically it is impossible).
mechanical shit was local, electronic shit is global, not comparable at all
The first half of the 20th century excelled in mechanical destruction. Thus far, the electronic age has been much less bloody.
Since they mentioned nukes it seemed like an obvious example where local things can be catastrophic.
The theoretical risk of electronic things malfunctioning in some global way that they mentioned has never resulted in any nuclear weapons being deployed, but we've actually seen the local mechanical approach they disregard be devastating.
You're not looking at the right signs.
In terms of pollution and environmental damage, including with direct human impact such as microplastics, things have been getting worse pretty much exponentially quicker every year _since_ the end of WW2.
We are fixing it up by refactoring - many through adding abstractions.
I’m sure code with bad abstractions can scale poorly, but I’m not clear how code without abstractions can scale at all.
That's quite unrelated to abstractions. It's just poorly written code, for whatever reasons may have led there.
And I’d rather work in a codebase where the owners are constantly doing such refactors than not.
edit: if you feel the need to downvote, feel free to share why you think my question is problematic - I think that "poorly written" to describe excessive code file sizes is such a wooly phrase as to be basically useless in a discussion like this, and when examined more closely usually comes down to "poorly abstracted". But I stand to be corrected.
That’s really the entirety of the issue, right there.
People aren’t just accepting bad software, they’re paying for it, which incentivizes a race to the bottom.
It’s like “The War on Drugs,” that looks only at the suppliers, without ever considering the “pull,” created by the consumers.
As long as there are people willing to pay for crap, there will be companies that will make and sell crap.
Not only that, companies that try to make “not-crap,” will be driven out of business.
That's easy to stop. Disallow warranty disclaimers.
The EU has something like this, the Cyber Resilience Act, and it has an exception for FOSS.
It works out okay to require compliance from paid vendors - if it costs more to comply with security regulations, that will be included in the price. But it's not reasonable to impose requirements on volunteers that way. After the modification, the EU regulation puts the burden on paid vendors who use free software (who can afford to pay for it), not on free software volunteers.
Don't be silly.
Paid software == a requirement to be of merchantable quality: to do what it claims, do it reliably, and keep doing it, so long as clearly stated requirements are met.
If this means all paid apps become tiny simplistic things in weird restrictive type-safe languages like Ada -- win!
Free software == you break it, you get to keep both halves. Our liability is what you paid, nothing more.
Think about what happens when two people video call each other from opposite sides of the world. How many layers of hardware and software abstraction are they passing through, to have a reliable encrypted video conversation on a handheld computer where the majority of the lag is due to the speed of light? How much of that would you like to remove?
I would venture an alternative bogeyman - "move fast and break things" AKA the drive for profits. It's perfectly possible (as illustrated above) to reliably extract great performance from a tower of abstractions, while building those abstractions in a way that empowers developers; what's usually missing is the will to spend time and money doing it.
And then the video player crashes trying to play an ad
You might say "I don't need to be able to propose a solution in order to point out problems", and sure, but that's missing the point. Because by pointing out a problem, you are still implicitly asserting that some solution exists. And the counter to that is: no, no solution exists, and if you have no evidence in favor of your assertion that a solution exists, then I am allowed to counter with exactly as much evidence asserting that no solution exists.
Propose a solution if you want complaints to be taken seriously. More people pointing out the problems at this point contributes nothing; we all know everything is shit, what are you proposing we do about it?
You give an engineer ownership and let them learn from their own mistakes, rise to the occasion when the stakes are high. This presumes they will have the last word on changes to that sand box that they own. If they want to rewrite it — that's their call. I'm the end they'll create a codebase they're happy to maintain and we will all win.
(And I think they'll be a happier engineer too.)
It happens. People step in and take over. The company continues.
Um, no you're not.
> And the counter to that is: no, no solution exists,
If that's the case, it's probably helpful to know it.
> we all know everything is shit, what are you proposing we do about it?
Give up on whatever doomed goal you were trying to reach, instead of continuing to waste time on it?
Weird take.
FOSS doesn't have these problems.
FOSS is certainly guilty too.
It fosters a culture where everyone can hack something together, and where everyone is knowledgeable enough to make responsible use of technology.
Working as a for-hire developer doesn't let you experience all of that because you're building a product that someone else wants you to build. No wonder one does not give a shit about writing good software at that point! You've taken all the fun and personal fulfillment out of it!
We can build anything we put our mind to -- but most of us are too busy churning out CRUD boilerplate like factory workers. That's depressing.
When done correctly it absolutely adds business value and should not make it harder to adapt or change, that's the point of good engineering. The problem is that you need years, if not decades, of technical experience to see this, and it's also a hard sell when there is no immediate "impact". It's basically something that happens because consumers don't know any better, so then it becomes low priority for making profit... at least until a competitor shows up that has better software and then it has a competitive edge, but that's a different matter.
Its called “being out of touch” I believe
Bad software keeps happening because businesses can afford it, as well as hardware improvements. It's a combination of consumers not knowing what they are missing and hardware advancements allowing for bad software to exist.
If you aren’t writing software with the customer and business in mind, why are you doing it? That’s what you are getting paid for.
That said, AI is about to change all this... but ironically this justifies my position. If software can be so powerful such that you can have a general intelligence that can do almost any cognitive task... then it follows that all this time engineers can also engineer clever systems that can add a lot of value to the company if engineered correctly.
There is no ceiling of how much performance and value can be squeezed out of a software system, but this never happens because businesses are not investing in actual engineering but rather in technical entrepreneurs that can bring something to market quickly enough.
There is no “implicit value” of software to a company. The only value of a software to a company is whether it makes the company money or saves the company money. That’s it, there is no other reason for a company to pay anyone except to bring more value to the company than they cost to employ them.
If software can be so powerful such that you can have a general intelligence that can do almost any cognitive task... then it follows that all this time engineers can also engineer clever systems that can add a lot of value to the company if engineered correctly.
It’s just the opposite. If AI can do any of the coding that a software engineer can do and I am not saying that’s possible or ever will be, what’s becomes even more important are the people who know how to derive business value out of AI.
> There is no ceiling of how much performance and value can be squeezed out of a software system
That may be true. But what’s the cost benefit analysis? Should we all be programming in assembly? Should game developers write a custom bespoke game engines and optimize them for each platform?
The implicit part is that if you engineer a good system then it saves money with less bugs and breaks less, and also makes money by allowing faster development and iterations.
There are plenty of examples here. I could point at how the PlayStation network just went down for 24 hours, or how UIs are often still very laggy and buggy, or I can also point at companies like Vercel that are (I assume) very valuable by providing a convenient and easy way to deploy applications... the fact that there are many SaaS out there providing convenience of development proves that this adds value. Despite this businesses are not having their engineers do this in-house because somehow they don't see the immediate ROI for their own business. I would just call that lack of vision or creativity at the business level, where you can't see the value of a well engineered system.
Businesses are free to run their company in whichever way they please, and they can create crappy software if it makes them money, but the point is that when this is industry-wide it cripples the evolution of software and this is then felt by everyone with downtimes and bad experiences, even though hardware is unbelievably fast and performant.
> Should game developers write a custom bespoke game engines and optimize them for each platform?
This is a good example actually. Most companies that want full creative control are making their own engines. The only exception here is Unreal (other smaller engines are not used by large companies), and from what I can tell the Unreal engine is an example of great software. This is one of those exceptions where engineers are actually doing engineering and the company probably can't afford to have them do something else. Many companies could benefit from this, but it's just not as straight line from the engineering to profit and that's kind of the root of why there is so much bad software out there.
Part of “meeting requirements” is always RTO, RPO, the availability requirements, latency, responsiveness, etc.
An alternative but dangerous approach is to make it known you’re looking elsewhere for work. Don’t do that if it’s relatively easy to replace you, and definitely assume the management thinks it’s easy to replace you, especialy if you haven’t been talking to your boss. ;) But there is the chance that they know you’re valuable and haven’t given you a raise because you seem content and they believe they have the upper hand - which may or may not be true.
When I left a company for increased compensation, which funny enough has only been 3x in almost 30 years across 10 jobs, it’s been between a 25%-60% raise. It’s almost impossible for any manager to push that kind of raise for anyone without a promotion.
Even at BigTech promotions usually come with lower raises than if you came in at that level.
Don’t do that if it’s relatively easy to replace you, and definitely assume the management thinks it’s easy to replace you, especialy if you haven’t been talking to your boss.
Everyone is replaceable. If you are at a company where everyone isn’t replaceable, it’s a poorly run company with a key man risk. I never saw any company outside of a one man company or professional practice where one person leaving was the end of the company.
Very true! Though isn’t it also normal in that case for HR to be recommending inflation raises at least? The exception might be if you came in at a salary that’s higher than your peer group and/or high for the title range. Parent’s problem could be that - either peer group correction or not possible for manager to raise at all without a promotion by company rules. There’s lots of reasons I can imagine, but in any case I wouldn’t expect a change with status quo, right? If you haven’t been talking to your boss, continuing to not talk to your boss is unlikely to change anything.
But yeah, the company should be doing at least inflation raises. A company only has at most two full review cycles to not get me somewhere in the range I think I should be making before I start looking for another job.
But I do understand that it is a shit show out here right now. I was looking for bog standard enterprise dev jobs with AWS experience as a “Plan B” while waiting for the “Plan A” interviews to work their way through. I have never seen anything like this in almost 30 years.
It was not this hard for me to get software developer job interviews in either 2000-2001 or 2008-2010. Admittedly, that’s partially because I was only looking for remote jobs and there the competition is fierce.
There's really way too many developers that care way more about the code than the product, let alone the business.
It ends up like fine dining, where 99% of the times a big Mac would've been tastier and made the customer happier, but wouldn't justify the effort and price.
I keep hearing how 10x engineers make their companies millions upon millions but they only get paid a fraction of that. How does that even make sense as a fair exchange? Not to mention is completely unfeasible for most people to have this kind of impact... it is only possible for those in key positions, yet every engineer is tasked with this same objective to increase company value as much as possible. There's just something very wrong with that.
Because they feel they can extract more value from them then they are paying them.
There are no real boundaries of when work starts and stops or when your responsibilities end because the task is to increase business value and that is technically endless
I work 40 hours a week and they pay me the agreed upon amount. There was nowhere in our agreement the expectation of my working more than that. I also knew that they could put me on a plane anytime during the week.
The company keeps piling more and more work, keeps growing and growing,
That’s completely on the employee to communicate trade offs between time, cost and requirements. My time is fixed at 40 hours a week. They can choose to use my 40 hours a week to work with sales and the customer to close a deal, be a project manager, lead an implementation, be a hands on keyboard developer, or be a “cloud engineer”. It’s on them how to best use my talents for the amount of money they are paying me. But seeing that they pay my level of employee the highest of all of the ICs, they really should choose to have me working with sales and clients to close deals.
That’s not bragging. I make now what I made as a mid level employee at BigTech in 2021.
I keep hearing how 10x engineers make their companies millions upon millions but they only get paid a fraction of that. How does that even make sense as a fair exchange?
The concept of a 10x engineer except in very rare cases is a myth if you think of them as just being on a keyboard everyday. All of the things I listed I could do - project management, backend developer or a cloud engineer - I would say I’m only slightly better than average if that. My multiplier comes because i can work with all of those people and the “business” and they can put me on a plane or zoom call and I can be trusted to have the soft skills necessary and my breadth is wide enough to know what needs to be done as part of a complex implementation and how to derive business value.
If you are making a company millions of dollars and you are only getting a fraction of that - and I doubt someone is doing that on their own without the supporting organizational infrastructure - it’s on you to leverage that to meet your priority stack.
Not to mention is completely unfeasible for most people to have this kind of impact... it is only possible for those in key positions, yet every engineer is tasked with this same objective to increase company value as much as possible. There's just something very wrong with that.
If you are a junior developer, there isn’t much expected of you, you are told what to do and how to do it. You aren’t expected to know the business value.
If you are a mid level developer, you are generally told the business objective and expected to know best practices of how to get there and understand trade offs on the epic/work stream level.
If you are a “senior” developer, now you are expected to understand business value, work with the stakeholders or their proxy, understand risks, navigate XYProblems on the project implementation level and deal with ambiguity.
As you move up the more “scope”, “impact” and “dealing with ambiguity” you have to be comfortable with.
“Codez real gud” only comes into play in getting from junior to mid.
> All of the things I listed I could do - project management, backend developer or a cloud engineer - I would say I’m only slightly better than average if that
I completely acknowledge this is a valid way to run a business, but the context here is how this sort of career progression is preventing the specialization of engineers in their domain and contributing to the widespread of software problems. Instead of investing in good engineers that specialize in their domain, companies move them away from engineering into more of an entrepreneur mindset by tasking them with adding value to the business directly, which is not something that you do as an engineer (it's nowhere in a CS degree, aside from say some electives).
A good metaphor here is a football/soccer team. What companies are doing is telling the goal keeper that he needs to score goals because more goals means winning. The team wants to win so everyone on the field has to score a goal. That obviously doesn't make sense even though the premise is true. You want a team composed of specialists and the more they specialize in their domain and work together the more you win. Even though there are only two or three offensive players that are scoring the goals, everyone is contributing to the success of the team if they specialize in their domain. Similarly, just because talking to clients and selling the product is directly contributing to the revenue of a business it doesn't mean that engineering at a higher level has no value.
And once again to stress the context here, companies can do whatever they want, but having engineers progress through their careers by moving AWAY from engineering is precisely why there is so much bad software out there. Letting engineers create better software should result in more profit in the long term, just probably not in the short term, and it's also hard for non-technical people to manage. So it is what it is.
Engineering is not only the physical labor. Aircraft engineers and building engineers don’t spend most of their time doing hands on work.
https://www.careerexplorer.com/careers/engineer/
Designing and Planning: Engineers are responsible for designing and planning systems, structures, processes, or technologies. They analyze requirements, gather data, and create detailed plans and specifications to meet project objectives. This involves considering factors such as functionality, safety, efficiency, and cost-effectiveness.
When doing construction work, who adds more value?
The general contractor? (Staff software engineer),
The owners of the plumbing, electrical, and HVAC companies assuming they have the actual skills (senior level developers). The owners of the plumbing companies could very well be making more than the general contractors. This is where you can specialize and the sky is the limit.
The actual certified workers (mid level developers). This is the level that the completely head down people are. No matter how good they become at being a hands on plumber , there is a hard ceiling they are going to hit at this level.
The apprentices (juniors)?
I work in consulting. I am currently a staff architect (true - IC5) over a hypothetical project (not a real project). I know the project is going to need a cloud architect, a data architect , and a software architect. They are all specialists at their jobs and are all going to lead their “work streams”. They are all IC4s
I expect each architect to take the high level business objectives and work with the relevant technical people on both sides and lead their work along with some hands on keyboard work.
They will each have people under them that are not customer facing at all. While I know all of the domains at some level, I’m going to defer to their technical judgement as long as it meets the business objectives. I did my high level designs before they came on to the project. Was my design work, figuring out priorities, risks, making sure it met the clients needs, discussing trade offs, etc not “engineering”?
Each level down from myself IC5 to junior engineers (IC1) is dealing with less scope, impact and ambiguity. There is no reason that the architects shouldn’t get paid as much as I do. They bring to the table technical expertise and depth. I bring to the table delivery experience, being able to disambiguate, and breadth.
No, but software is inherently different because you can leverage existing software to create more software. Every airplane has to be created individually, but software that already exists can be extended or reused by just calling functions or in the worst case copy/pasting.
> The actual certified workers (mid level developers). This is the level that the completely head down people are. No matter how good they become at being a hands on plumber , there is a hard ceiling they are going to hit at this level.
Yes, with hardware this can be the case as there is a small number of ways to build something. With software there is no ceiling, and the proof here is AI. We might soon see general intelligence that just codes anything you want. This means software can be designed to automate virtually anything in anyway shape or form, but it requires more and more expertise.
> I did my high level designs before they came on to the project. Was my design work, figuring out priorities, risks, making sure it met the clients needs, discussing trade offs, etc not “engineering”?
I agree what you're outlining is how the industry works. Perhaps the core of the issue here is how software engineering was modeled after other kinds of engineering with physical limitations. Software is closer to mathematics (arguably it's just mathematics). You can of course still design and plan and delegate, but once the role starts dealing with the high level planning, scheduling, managing, etc., there is less of a requirement for the technical details.
I've worked with architects that didn't know the specifics of a language or design patterns, not because they're bad at engineering but because they had no more time to spend on those details. These details are crucial for good software that is reliable, robust, extensible, etc. Junior and even mid level engineers also don't know these details. Only someone that has been hands on for a long time within a domain can hone these skills, but I have seen so many good engineers become senior or tech leads and then forget these details only to then create software that needs constant fixing and eventually rewriting.
I'm a senior myself and have no choice but to engage in these activities of planning, scheduling, etc., when I can clearly see they do not require technical expertise. You just need some basic general knowledge, and they just are time consuming. My time would be better spent writing advanced code that mid-level and junior level can then expand on (which has happened before with pretty good success, accelerating development and eliminating huge categories of bugs). Instead I have to resort to mediocre solutions that can be delegated. As a result I can see all kinds of problems accumulating with the codebase. It's also really hard to convince the leadership to invest in "high level" engineering because they think that you create more impact by managing an army of low to mid-level engineers instead of leveraging properly written software. I'm convinced that it does add value in the long term, it's just a hard sell. Ultimately I guess it comes down to the type of org and the business needs, which often does not include writing software that will not break. Most companies can afford to write bad software if it means they get to scale by adding more people.
That’s true. But when I put my “software engineering”, “cloud engineer”, or “data engineer” (theoretically) hat on, I can only do work of one person. No matter how good I am at any of it, I won’t be producing more output than someone equally qualified at my own company. Distributing software does have zero marginal cost more or less and that’s why we get paid more than most industries.
but I have seen so many good engineers become senior or tech leads and then forget these details only to then create software that needs constant fixing and eventually rewriting.
This is just like both my general contractor analogy and my real world scenario. As an “staff architect”, I come up with the initial design and get stakeholder buy in. But I defer to the SMEs the cloud architect, the data architect and the software architect who still eats, sleep and breathe the details in their specialty.
Just like the owners of the plumbing company, HVAC company and electrical company are the subject matter experts. The general contractor defers to them.
In the consulting industry at least, there are two ways you can get to the top, by focusing on depth or breadth. But in neither case can you do it by being staff augmentation (the plumber, electrician, or HVAC person), you still have to deal with strategy.
> You just need some basic general knowledge, and they just are time consuming
Knowledge isn’t the issue, it’s wisdom that only comes with experience that I assume you have. Going back to the hypothetical large implementation. It involves someone who does know architecture, development and data. No matter how good your code is, high availability, fault tolerance, redundancy, even throughput comes from the underlying architecture. Code and hands on implementation is usually the least challenging part of a delivery.
Its knowing how to deal with organizational issues, managing dependencies, sussing out requirements, etc
> No matter how good your code is, high availability, fault tolerance, redundancy, even throughput comes from the underlying architecture. Code and hands on implementation is usually the least challenging part of a delivery.
I agree to a certain extent. Unless the solution is short lived, designing the code to scale properly is extremely important, and this happens at the implementation level where details of how code is being added makes all the difference. I can see how contractors might not be thinking about this, but it's a recurring pattern with legacy systems. Additionally with software, implementation gets automated away if done correctly, so there is only an initial overhead that pays back as the codebase grows larger and larger.
I'm not saying there is no need for engineers that deal with org and management issues, but rather that companies underestimate the need for senior level engineers focusing on the software implementation details, only to pay the price as the codebase inevitably accumulates complexity later as it scales in size.
Anyway, great discussion, I appreciate your input and will be considering it more.
Look, I get that HN is startup-focused, but many companies DO drown themselves in tech debt of the sake of the business case.
Then a few years pass and they are unable to deliver features quickly enough - they can't keep Sr. engineers because their stack is a soul-sucking, career-killing tarpit. The Jr. engineers don't know how to fix the code and don't even see anything wrong with it because it's all they've known.
It's totally fine if you're a megacorp - Salesforce and so on can hire JUST ENOUGH high-level folks to keep the system manageable. It's also fine if you're so small that all your work is greenfield.
The type of company that I'm describing is generally Series D/E and probably worth a few 100M dollars.
Your reasoning for tackling tech debt is sound. “We are moving like molasses. Our foundation is wobbly and we can’t build new features until we start tackling $X tech debt. These are the reasons that it will help us in the future.”
But on the other hand, if I achieve x, y, and z or someone else achieve it and we are told it didn’t achieve enough “impact”, yeah I am going to do like Google employees do and decide to work on the 5th messaging app.
I'm sorry but I don't buy it.
I've been way too close way too often with lisp and Haskell or many other niches (I know well both Racket/Scheme and Haskell btw) the people that care that much about this correct and reliable and extensible software care about the code more than they care about the products.
That's why the languages that breed creativity and stress correctness have very little, if any, killer software to show when PHP/Java has tons of it.
First, hardware has improved consistently, outpacing any need for optimal software. Second, the end user does not know what they really want from software until it is provided to them, so most people will easily put up with slow, laggy and buggy software until there is a competitor that can provide a better experience. In other words, if companies can make money from making bad software, they will, and this is often what happens because most people don't know any different and also hardware becomes more and more powerful which alleviates and masks the defects in software.
I think there is also another big factor, which is that businesses prefer to scale by adding more people, rather than by making better software. Because they want to scale along human engineers, they tend to prefer low-tier software that is easy to pick up by the masses, rather than the high-level specialized software that is a hard skill to acquire. This is understandable, but is also the reason why software is so slow in advancing forward compared to hardware (and by the same token, hardware engineers require more specialization).
Why care about quality or maintainability if you are gone in year or two anyway...
Coming up with a neat API that turns out to be difficult to modify in the future, or limiting in ways you didn't imagine would when writing it is a good learning experience.
Or seeing how long a system can survive growing usage -- Maybe a simple hack works better than anyone expected because you can just pay more for RAM/CPU each year rather than rebuild into a distributed fashion. Or the opposite, maybe there's some scaling factor or threshold you didn't know existed and system performance craters earlier than predicted.
He works as a highly-skilled tech, at a major medical/scientific corporation. They have invested years of training in him, he brings them tremendous value, and they know it. He was just telling me how he used that value to negotiate a higher compensation package for himself. Not as good as if he swapped jobs, but he really has a sweet gig.
People who stay, take Responsibility for the code they write. They will need to face the music, if it doesn't work, even if they are not responsible for maintaining it.
They are also worth investing in specialized training, as that training will give great ROI, over time.
But keeping skilled people is something that modern management philosophy (in tech, at least) doesn't seem to care about.
Until corporations improve the quality of their managers; especially their "first-line" managers, and improve their culture, geared towards retaining top talent (which includes paying them more -but there's a lot more that needs doing), I can't, with good conscience, advise folks not to bounce.
I’m a founder for 10 people and this is the first thing we think about. Except for low performers; except that youngsters need a variety of experience to be proficient at life; except that the team is not performing well(1). 25% or 30% increases for half the workforce are frequent.
(1) The biggest remark from management coaches is that giving raises lowers employee performance, which I can fully witness in my company. It’s not even good for morale. I’m just happy that people exit the company fitter and with a girlfriend, even a kid and sometimes a permanent residency, but business-wise I’ve been as good as a bad leader.
I’m reaching the sad conclusion that employees bring it upon themselves.
My friend could double his salary, moving almost anywhere else, but he gets a lot of perks at his work, and is treated extremely well by his managers.
They just gave him a rave review, and that did more to boost his willingness to stay, than a 10% raise. He will still negotiate a better salary, but is more likely to be satisfied with less than he might have, if they tried to treat him badly.
Treating employees with Respect can actually improve the bottom line. They may well be willing to remain in difficult situations, if they feel they are personally valued.
I know this from personal experience. When they rolled up my team, after almost 27 years, the employee with the least tenure had a decade. These were top-shelf C++ image processing engineers, that could have gotten much higher salaries, elsewhere.
The problem is our current form of corporate culture. Employees don't feel like they matter, there efforts are a cog in a wheel. If you get a raise in this type culture, it only matters to the bottom line and there is no incentive produce because the employee is already unhappy in the first place.
Change your business culture and these problems will disappear, IMHO.
We also have a 401K match with an immediate vest.
If they would care then job hopping would not exist. If staying at s company would be more. Beneficial to your salary, why would you ever want to change company, if you are otherwise happy?
If your main motivation for working is to exchange your labor for the maximum amount of money possible, I don’t see how that is the positive outcome you think it is.
I personally wouldn’t leave my current job if another one for $100K more fell into my lap. But the “unlimited PTO” where the custom is to take at least 5-6 weeks off during the year not including paid holidays and it being fully remote is hard to beat.
I mean pretty much exactly what you said.
I apologize for being unclear.
This is the root problem. None of the problems the GP pointed up were created by software developer.
Now, if you want to know the consequences, it causes an entire generation of people that don't really know what they are doing because they never see the long-term consequences of their actions. But again, it's not the software developers that are causing this, nor are they the ones that should fix it.
Frequent job hopping: lack of pay upgrades because software is considered a cost center
I could go on but in reality it’s a disconnect between what business thinks the software is worth as opposed to what the engineer wants to do with it.
You can say software is an art but commodity art doesn’t make much money. In reality, the ad driven software has greatly inflated salaries (not complaining but it’s reality). Now it’s going to be an ai bubble. But your rank and file business doesn’t care what software bubble is happening but unfortunately they are bound by the costs that come with it.
Have you seen the process that happens in defense or medical equipment industries. You probably won’t complain.
If you didnt have degree requirements and certification bodies for:
* accountants
* engineers
* doctors
* lawyers
What do you think hiring might look like?
Do you think they would build a hiring process to validate to the best of their ability your aptitude of the core fundamentals- except worse than certification and education bodies?
I would presume so at least.
Have you ever done any of the various IT certs?
Doctors also go to school for 8 years and then do residencies. Lawyers go to school for 7. Are you proposing that?
Realistically, licensing boards are there to protect their members, and rarely do political things against people in the same body with unpopular opinions. You have to be catastrophic for most boards to do anything about you: Just like a police union will defend a union member that has committed gross negligence unless the evidence is public.
When you hire a doctor for something actually important, you don't look at the certification body: You look at long term reputation, which you also do in software. Only the largest of employers will leetcode everyone as a layer of fairness. In smaller employers, direct, personal references replace everything, which is what I'd go with if I needed an oncologist for a very specific kind of cancer. The baseline of aptitude from the certification body doesn't matter there at all.
So we need the barrier to entry to be even lower for such professions that deal with life-changing outcomes? I don't think so. In such high risk fields: "long term reputation" is totally dependent on hiring extremely qualified individuals.
The barrier to entry MUST be continuously raised with the bare minimum requirement of a degree. Only then the secondary requirements can be considered.
> When you hire a doctor for something actually important, you don't look at the certification body: You look at long term reputation, which you also do in software.
I don't think you can compare the two. Since one deals with high risk to the patient such as life and death and the other in most does not. (Unless the software being written deals with safety critical systems in a heavily regulated setting.)
From what you are saying, maybe you would be OK consulting a surgeon or an oncologist that has never gone to medical school.
Semi-related: Start using profane variable names because apparantly it will cause Copilot to stop analyzing your code.
I think people vastly underestimate "the wheel". The wheel was something recreated independently over and over again for thousands of years across human history. Even your favorite programming language or web framework is not comparable to "the wheel".
To me this is the most sure way to identify the adults from the children in the room. People that can actually program, strangely enough, aren’t bothered by programming.