You'll see capability traps everywhere once you learn about them.
Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing.
Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.
Article published in the Summer 2001 edition of California Management Review, yet it never mentioned Y2K, the first thing I thought of when I read the line "fixing problems that never happened". Perhaps it was actually written in 1999 and took a while to get published, because otherwise that seems a very strange omission. The Y2K problem was very much over-hyped by the American news media at the time (no, at no point would airplanes have been falling out of the sky — I literally heard someone say that would happen once — even if no effort had been put into fixing the bug).
But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years.
The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.
My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course."
Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D
One of those days, however, you come up with another of your elegant, simple solutions and it actually replaces a 150K LoC monstrosity with either a 1K script or, even better, with nothing as a simple shift in perspective or process obsoletes it completely.
In the long run, IME, you'll be recognized either by management or your peers if you keep doing that over and over again.
On the other hand, I don't help people with their computer problems anymore because I've found that the more difficult the problem, the longer it takes me to rescue their data or whatever the less impressed they are. The more miraculous the save the more likely they are to tell me the story about their nephew who solved a trivial issue instantly as if to point out that I didn't.
Yeap. Managers perceive complexity by how personally confused they are. I'm late in my career, and I'm realizing I wasted so many man years trying to make code clean, user friendly, and maintainable, when that code was never read by another person again and forgotten 15 minutes after it was released, then used for years. This is why I think AI is coming for our jobs much sooner than many people think: clean code, separation of concerns, maintainability, etc, all the things we spend the most time on, have never actually been valued. "Good enough" is fine, and keeps management happy. And, if something does pop up, AI can patch it, even if with spaghetti, just like fucking that ass at work.
- Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it.
- Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19.
There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”.
Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash.
For some reason, these stories don’t make the news.
There is something I saw on a reddit post of all places, about how every manager who doesn't predict a baseline of "3 annoying problems every month, 1 awful problem every 3 months" is essentially a bad manager. The reasoning being that, if your number of problems is under that threshold, then someone is doing a 'good job'.
I've been in those companies where "struggling departments" ended up getting all the praises and raise in budgets the following quarter because of the heroic saves they did, and raising awareness on how important they are... For stuff they totally caused on themselves.
Meanwhile, my perfectly purring department was struggling to keep the lights on.
It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing).
<insert IBM story about IT department cost cuts>
I'm not sure how we solve this, other than having management come from engineering.
I believe it's a problem in most industries and even humanity in general. I don't believe it's a business problem at all.
Heroes are lauded even if they solve problems they themselves are the cause of - which is conveniently either forgotten or denied - or they are solving non-issues that are deemed important by the ignorami-class. Politics, for example, is utterly dominated by this dynamic.
It's the first instinct: let the expert run the show. However, one of the (many) ways to let a complex project fall apart completely is to hand over full control to engineers. I'm one myself, but I know what I'm good at and what not. Dunning-Kruger is often mentioned in these discussions, but don't forget it also applies to engineers that often lack any management or leadership experience of any appreciable kind. They vastly overestimate their ability to handle management and organization-wide issues and tend to not only miss the forest for the trees but actually miss the trees for the leaves.
"Unix: A History and a Memoir" by Brian Kernighan actually mentions how proper management was crucial to their success. It's a detail that's frequently conveniently forgotten by the engineers who think themselves better than the "suits". For the record, I don't claim engineers are the primary problem, but it's not just management's either. Quotes like "who holds the company standing" and "who understands how to double click" are enormous smells and IMO make quite clear what's happening here.
I don't have ready-made solutions unfortunately, but I do wish we would look further than "it's the suits". It's a systemic, human problem that I believe is a natural result of operating under informational constraints and, very human, cognitive biases on all sides.
I guess the point of view is that if a department is well running, it means it is overressourced. So you reduce the ressources until it's breaking point, just enough for it to not fail. A jaded service manager told me it was part of its official training: if the clients was too satisfied that meant that human ressources were wasted on them, so he had to spin plates between clients. I guess it was economically optimal.
At a previous job the CEO/owner had the idea that you'd get some percentage of any cost saving your could find as a bonus. Something like 20% of the savings for the first year.
My colleague in the IT department had one idea, replace our commercial certificates with Let's Encrypt and drop the EV requirement. In total he'd stand to get a bonus of a little over €2000. He never got the money, because things like that was part of his job apparently.
> I'm not sure how we solve this, other than having management come from engineering.
I disagree with the implied idea here that "engineers are better managers". The solution is to have good management, not to assume that "engineers are better managers". I have seen good and bad managers, and in both groups there were engineers and non-engineers.
In my 35+ years in IT, the "hero attitude" was the one in the top three I most hated traits in a person working with or for me. And talking about traits, I considered crucial to always have in my teams a "saboteur" engineer - the one who thoght, found, come up with all the way we could break a design, service, infra components, app, etc., when all the others were designing or operating for perfect or normal conditions.
Econometricians can solve it, bc we can create rigorous models that map causal inputs to output.
It’s extremely advanced technology, though, and most CEOs would rather rent seek / camp than give up some decision-making power (and very few are even aware it’s possible).
Isn’t this a universal problem though, not just software industry? Even at home, if one kid just does his thing quietly but another kid is difficult, what do we say? “John has his problems but he is trying, we gotta encourage him”. While his brother gets no praise or attention for just doing his thing quietly without fuss.
When things run smoothly, very few people notice. When things break, everyone notices
> I've been in those companies where "struggling departments" ended up getting all the praises and raise in budgets the following quarter because of the heroic saves they did, and raising awareness on how important they are... For stuff they totally caused on themselves.
This is a very game-able system, and I'd wager a decent amount that any senior engineers on those teams know exactly what they are doing. In a lot of (broken, but aren't they all) management structures, it's better to be seen to swoop in with the save than to quietly fix it ahead of time.
And if your management is structuring rewards like this, it leads to your seniors anticipating a bunch of these failures, lining up 90% of the fix before hand, so that they can jump on the oncall escalation with a 100% "Hail Mary" of a fix...
Which they should. I've been lucky enough to work at places that had great non-technical managers that promoted based on great execution, as well as highly technical managers that also promoted based on great execution.
Now I'm at the other kind of place and it sucks. They'll fire the performative engineers though during layoff season. It's almost like they like playing politics until it really matters.
Same. I watched a manager fail upwards till he practically ran the entirety of engineering. All his projects failed, got expanded and restarted with more budget and more devs until he ran everything. Meanwhile the teams that actually wrote working services got their budgets frozen and lost headcount.
Making critical decisions without oversight is just as bad, or maybe worse.
If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.
I'm looking for some data -- if anyone has it -- on the fraction of companies that are led (CEO) by a technical person, over the years/decades. I have the (anecdotal) impression that this fraction has been falling (stories like Boeing), but it would be cool to support or refute this with hard data. Anyone know where to find/assemble something like this? Also, if this trend is true, then why?
We all learned this back in first grade. The kids that behaved in class and did their homework did not command most of the teacher's time and effort. It was the problem children who refused to follow the rules and needed constant praise for every bit of actual effort that they put into their studies; that got the teacher's attention.
This is the real problem with performance reviews in companies, which then feeds into opportunities, promotions and compensation. It's just a popularity contest. And this is particularly harmful to people who are neurodivergent, particularly if they're on the autism spectrum, because neurotypical people, who end up making all these decisions, view such people negatively for literally no reason.
You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts?
1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and
2. They failed to ship and get subpar performance ratings for having no impact.
The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it.
You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own.
Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success.
Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything.
Its really hard to measure effectiveness, problem becomes even harder when a non-engineering person has the job to measure effectiveness of a engineering person.
On other hand. for software engineering some of the signals that can be used to measure such a management itself can be
1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team
2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign
94 comments
[ 3.5 ms ] story [ 81.9 ms ] threaderhm, if this figure is close to true i can see what market ai companies is after.
https://news.ycombinator.com/item?id=8940820 - 24 Jan 2015, 50 comments
https://news.ycombinator.com/item?id=39472693 - 22 Feb 2024, 434 comments
Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing.
Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.
But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years.
The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.
My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course."
Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D
In the long run, IME, you'll be recognized either by management or your peers if you keep doing that over and over again.
- Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it.
- Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19.
There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”.
Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash.
For some reason, these stories don’t make the news.
Meanwhile, my perfectly purring department was struggling to keep the lights on.
It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing).
<insert IBM story about IT department cost cuts>
I'm not sure how we solve this, other than having management come from engineering.
It’s not a problem in this industry, it’s a problem everywhere.
> I'm not sure how we solve this, other than having management come from engineering.
You mean the engineers who are causing the chaos you’re complaining about?
Engineers aren’t some magic group of people who know better than others - we’re just as fallible as other people.
Heroes are lauded even if they solve problems they themselves are the cause of - which is conveniently either forgotten or denied - or they are solving non-issues that are deemed important by the ignorami-class. Politics, for example, is utterly dominated by this dynamic.
It's the first instinct: let the expert run the show. However, one of the (many) ways to let a complex project fall apart completely is to hand over full control to engineers. I'm one myself, but I know what I'm good at and what not. Dunning-Kruger is often mentioned in these discussions, but don't forget it also applies to engineers that often lack any management or leadership experience of any appreciable kind. They vastly overestimate their ability to handle management and organization-wide issues and tend to not only miss the forest for the trees but actually miss the trees for the leaves.
"Unix: A History and a Memoir" by Brian Kernighan actually mentions how proper management was crucial to their success. It's a detail that's frequently conveniently forgotten by the engineers who think themselves better than the "suits". For the record, I don't claim engineers are the primary problem, but it's not just management's either. Quotes like "who holds the company standing" and "who understands how to double click" are enormous smells and IMO make quite clear what's happening here.
I don't have ready-made solutions unfortunately, but I do wish we would look further than "it's the suits". It's a systemic, human problem that I believe is a natural result of operating under informational constraints and, very human, cognitive biases on all sides.
My colleague in the IT department had one idea, replace our commercial certificates with Let's Encrypt and drop the EV requirement. In total he'd stand to get a bonus of a little over €2000. He never got the money, because things like that was part of his job apparently.
I disagree with the implied idea here that "engineers are better managers". The solution is to have good management, not to assume that "engineers are better managers". I have seen good and bad managers, and in both groups there were engineers and non-engineers.
It’s extremely advanced technology, though, and most CEOs would rather rent seek / camp than give up some decision-making power (and very few are even aware it’s possible).
"Accounted for X situation" "Added gaurdrails to protect against Y"
When working as a business analyst i have to do this sort of thing all hte time or else id get no credit for half my work
When things run smoothly, very few people notice. When things break, everyone notices
"John! Why can't you be more like your brother?!"
comes from.
This is a very game-able system, and I'd wager a decent amount that any senior engineers on those teams know exactly what they are doing. In a lot of (broken, but aren't they all) management structures, it's better to be seen to swoop in with the save than to quietly fix it ahead of time.
And if your management is structuring rewards like this, it leads to your seniors anticipating a bunch of these failures, lining up 90% of the fix before hand, so that they can jump on the oncall escalation with a 100% "Hail Mary" of a fix...
Now I'm at the other kind of place and it sucks. They'll fire the performative engineers though during layoff season. It's almost like they like playing politics until it really matters.
If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.
Nobody ever gets credit for fixing problems that never happened (2001) [pdf] - https://news.ycombinator.com/item?id=39472693 - Feb 2024 (424 comments)
Nobody Ever Gets Credit for Fixing Problems That Never Happened (2001) [pdf] - https://news.ycombinator.com/item?id=8940820 - Jan 2015 (50 comments)
You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts?
1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and
2. They failed to ship and get subpar performance ratings for having no impact.
The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it.
You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own.
Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success.
Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything.
[1]: https://psychsafety.com/googles-project-aristotle/
On other hand. for software engineering some of the signals that can be used to measure such a management itself can be
1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team
2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign
That's what the moneys for ...
* https://en.wikipedia.org/wiki/Preparedness_paradox