I think I'm mostly of the opinion these days that there is no such thing as an "outdated technology". There are technologies that are no longer fit for purpose but that is almost never because of their age. It nearly always because of one of as examples: Needing to run in an environment it can't support, Having bugs that are not getting fixed/no longer maintained, Missing features necessary to solve new problems or add new features, Hitting scale limits.
Outdated may sometimes be a euphemism for one of the above but usually when I see it in a discussion it just means "old" or "out of fashion" instead.
> Most Technical Problems Are Really People Problems
The irony is that this is a classic engineer's take on the root cause of technical debt. Engineers are happy to be heads-down building. But when you get to a team size >1, you actually need to communicate - and ideally not just through a kanban board.
Isn't this generally the case across all sectors and industries? We have the technology today to create a post scarcity utopia, to reverse climate change, to restore the biosphere. The fact that none of that happens is a people problem, a political problem, a spiritual problem, more so than any technological barrier.
> Most technical problems are really people problems. Think about it. Why does technical debt exist? Because requirements weren't properly clarified before work began. Because a salesperson promised an unrealistic deadline to a customer. Because a developer chose an outdated technology because it was comfortable.
I used to be a "stay out of politics" developer. After a few years in the industry and move to a PM role, I have had the benefit of being a bit more detached. What I noticed was that intra-developer politics are sometimes way more entrenched and stubborn than other areas of the business.
Sure, business divisions have infighting and politics but at the end of the day those are tempered by the market. It's far harder to market test Ruby Versus Java in a reasonable manner, especially when you have proponents in both camps singing the praises of their favored technology in a quasi-religious manner. And yes, I have also seen the "Why would I learn anything new, <Technology X> works for me, why would I take the effort to learn a new thing" attitudes in a large number of coworkers, even the younger Gen-Z ones.
100% agree. Sadly, I have realised fewer people actually give an F than you realise; for some, it's just a paycheck. I am not sure what has happened over the decades regarding actually being proud of the work you produce.
I also think they tend to be the older ones among us who have seen what happens when it all goes wrong, and the stack comes tumbling down, and so want to make sure you don't end up in that position again. Covers all areas of IT from Cyber, DR, not just software.
When I have moved between places, I always try to ensure we have a clear set of guidelines in my initial 90-day plan, but it all comes back to the team.
It's been 50/50: some teams are desperate for any change, and others will do everything possible to destroy what you're trying to do. Or you have a leader above who has no idea and goes with the quickest/cheapest option.
The trick is to work this out VERY quickly!
However, when it does go really wrong, I assume most have followed the UK Post Office saga in the UK around the software bug(s) that sent people to prison, suicides, etc. https://en.wikipedia.org/wiki/British_Post_Office_scandal
I am pretty sure there would have been a small group (or at least one) of tech people in there who knew all of this and tried to get it fixed, but were blocked at every level. No idea - but suspect.
This is a classic engineering take on the problem. It changes when you become a CTO. Because now technical debt is your problem and the choice whether to fix it or not is yours to make. The flip side here is that wrong choices (either way) can be expensive and even kill your company.
I've been on both sides. Having to beg a manager to get permission to fix a thing that I thought needed fixing. And now I'm on both sides where as a CTO it's my responsibility to make sure the company delivers working products to customers that are competitive enough that we actually stand a chance to make money. And I build our product too.
Two realities:
1) Broken stuff can actually slow down a lot of critical feature development. My attitude as a CTO is that making hard things easier is when things can move forward the fastest. Unblocking progress by addressing the hardest things is valuable. Not all technical debt is created equally. There's a difference between messing with whatever subjective esthetics I might have and shit getting delayed because technical problems are complicating our lives.
2) We're a small company. And the idiot that caused the technical debt is usually me. That's not because I'm bad at what I do but I simply don't get it right 100% of the time. Any product that survives long enough will have issues. And my company is nearly six years old now. The challenge is not that there are issues but prioritizing and dealing with them in a sane way.
How I deal with this is very simple. I want to work on new stuff that adds value whenever I can. I'm happy when I can do that and it has a high impact. Whenever some technical debt issue is derailing my plans, I get frustrated and annoyed. And then I sit down and analyze what the worst/hardest thing is that is causing that. And then I fix that. It's ultimately my call. But I can't be doing this all the time.
One important CTO level job is to keep the company ready for strategic goals and make sure we are ready for likely future changes. So I look at blocking issues from the point of view of the type of change that they block that I know I will need to do soon. This is hard, nobody will tell me what this is. It's my job to find out and know. But getting this right is the difference between failing or succeeding as a technology company.
Another perspective here is that barring any technical moat, a well funded VC-funded team could probably re-create whatever you do in no time at all. If your tech is blocking you from moving ahead, it can be sobering to consider how long it would take a team unburdened by technical debt to catch up with you and do it better. Because, if the answer is "it wouldn't be that hard" you should probably start thinking about abandoning whatever you are trying to fix and maybe rebuilding it better. Because eventually somebody else might do that and beat you. Sometimes deleting code is better than fixing it.
I couldn't disagree more with this description of why technical debt exists and it's a dangerous line of reasoning. Sure, maybe requirements weren't clarified. But often it's impossible to clarify them and you have to build something and even if the requirements were clear to begin with who is to say they'll still be the same by the time you've finished the project let alone 5 years later. Maybe the develop chose a stable and dependable technology because it's battle worn and proven? Maybe the sales person has to manage an impossible situation between an engineering team which can't commit to the time line needed to win the sale?
There are lots of good reasons tech debt exists, and it's worrying that this person seems to think that they all boil down to "I don't know how but someone, somewhere, fucked up"
Jerry Weinberg, Secrets of Consulting (1985) - "No matter how it looks at first, it's always a people problem." - no matter how technical a problem seems, its root cause always involves people—their choices, communication, management, or skills—making human factors central to any solution, from software development to complex systems
As a data engineer in big tech, the two hardest problems I deal with are:
* Conway's law causing multiple different data science toolchains, different philosophies on model training, data handling, schema and protocol, data retention policies, etc.
* Coming up with tech solutions to try to mitigate the impact of multiple silos insisting on doing things their own way while also insisting that other silos do it their way because they need to access other silos' data.
And the reason standardization won't happen: the feudal lords of each of those branches of the hierarchy strongly believe their way is the only way that can meet their business/tech needs. As someone who gets to see all of those approaches - most of their approaches are both valid and flawed and often not in the way their leaders think. A few are "it's not going to work" levels of flawed as a result of an architect or leadership lacking operating experience.
So yeah, it might look like technical problems on the surface, but it's really people problems.
Which is why when arguing that technology XYZ succeeded, or failed, one needs to look into the larger picture of the human side regarding the related outcome in the market adoption.
The author suggested that if senior leadership had a development background then tech debt would be easier to get support and resources to deal with. Between the lines I'm reading that the risks are just inherently understood by someone with a tech background.
Then the author suggests that senior leadership without a tech background will usually need to be persuaded by a value proposition - the numbers.
I'm seeing these as the same thing - the risks of specific tech debt just needs to be understood before it gets addressed. Senior leaders with a development background might be better predictors of the relationship between tech debt and its impact on company finances. Non technical leaders just require an extra translation step to understand the relationship.
Then considering that some level of risk is tolerated, and some risk is consciously taken on to achieve things, both might ultimately choose to ignore some tech debt while addressing other bits.
This article resonates strongly. I am consulting right now to a group that has enormous struggles technically, but they are all self-inflicted wounds that come down to people and process.
Management claims to want to understand and fix the problem, and their "fixes" reveal the real problems. Fix 1 - schedule a lot of group meetings for twice a week. After week 1, management drops off and fails to show up anymore for most of them. The meetings go off track. The answer? More meetings!
We now have that meeting daily. And have even less attendance.
Fix 2 - we don't know what people are doing, let's create dashboards. A slapdash, highly incorrect and problematic dashboard is created. It doesn't matter, because none of the managers ever checks the dashboard. The big boss hears we are still behind, and commandeers a random product person to be his admin assistant and has her maintain several spreadsheets in semi-secret tracking everyone's progress.
This semi-secret spreadsheet becomes non-secret and people find a million and one problems with it (not surprising as the commandeered admin assistant nee product person was pulling the data from all sorts of random areas with little direction with little coordination with others). We then have the spreadsheet war of various managers having their own spreadsheets.
Fix 3 - we are going to have The Source of Truth for product intake and ongoing development, with a number of characteristics (and these are generally not terrible characteristics). These are handed off to a couple of junior people with no experience to implemented with zero feedback. The net result is we still don't have a Source of Truth, but more of an xkcd situation that now we have 4 or 5 sources of truth strung together with scripts, duct tape, bandaids and prayer.
This continues on and on over years. Ideas are put forth, some good, some bad, some indifferent, but none of them matter because the leaders lack the ability to followup or demonstrate even basic understanding of what our group actually does.
It is truly soul crushing, but in this jobs environment, what are you going to do?
I worked somewhere with a similarly dysfunctional culture.
Peak absurdity, yet you and I have both seen it happen first hand. I wonder how common it is, because it's as ugly as it is mystifying.
When management isn't properly engaged, they need to delegate to someone who is. If neither things happen, it's just chaos and angry, ignorant apes making a lot of noise.
> in this jobs environment, what are you going to do?
I am currently unemployed with a rapidly shrinking cushion, and I'm honestly on the fence as to whether putting up with the above would be better. If there is no hope for improvement, all you're doing is exchanging your mental health for a few more beans.
Say this in an interview and its a perfect way to fail, even though its true. Its sad how interviewers often take pleasure in pointing out that anything said outside their packets is a signal for lack of technical knowledge. I've been in and passed several tech interviews. I've also interviewed plenty of people, if someone points out the human aspect of a problem, I actually award points. Sad how often I have to fight with my colleagues.
"But what about using a message queue.."
"Candidate did not use microservices.."
"Lacks knowledge of graph databases.." (you know, because I took a training last week ergo it must be the solution).
..and most people problems are communication problems.
Calling them 'people problem' is a convenient catch-all that lacks enough nuance to be a useful statement. What constitutes good communication? Are there cross purposes?
> Non-technical people do not intuitively understand the level of effort required or the need for tech debt cleanup; it must be communicated effectively by engineering - in both initial estimates & project updates. Unless leadership has an engineering background, the value of the technical debt work likely needs to be quantified and shown as business value.
The engineer will typically say that the communication needed is technical, but in fact the language that leadership works with is usually non-technical, so the translation into this field is essential. We do not need more engineers, we need engineer who know how to translate the details.
I realise that, here on HN, most will probably take the side of the rational technologist, but this is a self-validating cycle that can identify the issue, but cannot solve it.
IMO, we need more generalists that can speak both languages. I have worked hard to try and be that person, but it turns out that almost no-one wants to hire this cross-discipline communicator, so there's a good chance that I'm wrong about all of this.
And most people problems are communication problems. Engineers aren't engaged with the product vision or the customer base, and are allowed to silo themselves. Product doesn't see the point of engineers being engaged and feed the engineering team like an in-house outsourcing shop. Sales and CS fail to understand the cost of their promises to individual customers to the timelines of features they're hungry for from the product plan. Goals and metrics for success fail to align. And thus everyone rows in their own direction.
The solution usually isn't "better people." It's engaging people on the same goals and making sure each of them knows how their part fits with the others. It's also recognizing when hard stuff is worth doing. Yeah you've got a module with 15 years of tech debt that you didn't create, and no-one on the team is confident in touching anymore. Unlike acne, it won't get better if you don't pick at it. Build out what that tech debt is costing the company and the risk it creates. Balance that against other goals, and find a plan that pays it down at the right time and the right speed.
Product doesn't see the point of engineers being engaged and feed the engineering team like an in-house outsourcing shop.
Because they want to feel superior as the ‘this was my idea and you executed on my idea’ nonsense. Their answers to most ‘why are we doing this ?’ ‘trust me bro’. I am perhaps generalizing and there are outlier product managers who have earned the ‘trust me bro’ adage, but most haven’t.
This PM behaviour will never change. Engineers have said enough is enough and are now taking over product roles, in essence eliminating the communication gap.
Most communication problems are hierarchical problems. Communication and relationships are inherently hierarchical and groups think and social behaviors, fitting in, understanding hierarchy and status runs communication more than we think.
72 comments
[ 3.5 ms ] story [ 70.7 ms ] threadOutdated may sometimes be a euphemism for one of the above but usually when I see it in a discussion it just means "old" or "out of fashion" instead.
The irony is that this is a classic engineer's take on the root cause of technical debt. Engineers are happy to be heads-down building. But when you get to a team size >1, you actually need to communicate - and ideally not just through a kanban board.
It comes as no surprise that a worker unit who makes this conscious decision might have problems interfacing with a Homo sapiens unit.
I used to be a "stay out of politics" developer. After a few years in the industry and move to a PM role, I have had the benefit of being a bit more detached. What I noticed was that intra-developer politics are sometimes way more entrenched and stubborn than other areas of the business.
Sure, business divisions have infighting and politics but at the end of the day those are tempered by the market. It's far harder to market test Ruby Versus Java in a reasonable manner, especially when you have proponents in both camps singing the praises of their favored technology in a quasi-religious manner. And yes, I have also seen the "Why would I learn anything new, <Technology X> works for me, why would I take the effort to learn a new thing" attitudes in a large number of coworkers, even the younger Gen-Z ones.
I also think they tend to be the older ones among us who have seen what happens when it all goes wrong, and the stack comes tumbling down, and so want to make sure you don't end up in that position again. Covers all areas of IT from Cyber, DR, not just software.
When I have moved between places, I always try to ensure we have a clear set of guidelines in my initial 90-day plan, but it all comes back to the team.
It's been 50/50: some teams are desperate for any change, and others will do everything possible to destroy what you're trying to do. Or you have a leader above who has no idea and goes with the quickest/cheapest option.
The trick is to work this out VERY quickly!
However, when it does go really wrong, I assume most have followed the UK Post Office saga in the UK around the software bug(s) that sent people to prison, suicides, etc. https://en.wikipedia.org/wiki/British_Post_Office_scandal
I am pretty sure there would have been a small group (or at least one) of tech people in there who knew all of this and tried to get it fixed, but were blocked at every level. No idea - but suspect.
ignorance against wrongdoers has been a bliss for your generation, curse for ours and deadly for future
I've been on both sides. Having to beg a manager to get permission to fix a thing that I thought needed fixing. And now I'm on both sides where as a CTO it's my responsibility to make sure the company delivers working products to customers that are competitive enough that we actually stand a chance to make money. And I build our product too.
Two realities:
1) Broken stuff can actually slow down a lot of critical feature development. My attitude as a CTO is that making hard things easier is when things can move forward the fastest. Unblocking progress by addressing the hardest things is valuable. Not all technical debt is created equally. There's a difference between messing with whatever subjective esthetics I might have and shit getting delayed because technical problems are complicating our lives.
2) We're a small company. And the idiot that caused the technical debt is usually me. That's not because I'm bad at what I do but I simply don't get it right 100% of the time. Any product that survives long enough will have issues. And my company is nearly six years old now. The challenge is not that there are issues but prioritizing and dealing with them in a sane way.
How I deal with this is very simple. I want to work on new stuff that adds value whenever I can. I'm happy when I can do that and it has a high impact. Whenever some technical debt issue is derailing my plans, I get frustrated and annoyed. And then I sit down and analyze what the worst/hardest thing is that is causing that. And then I fix that. It's ultimately my call. But I can't be doing this all the time.
One important CTO level job is to keep the company ready for strategic goals and make sure we are ready for likely future changes. So I look at blocking issues from the point of view of the type of change that they block that I know I will need to do soon. This is hard, nobody will tell me what this is. It's my job to find out and know. But getting this right is the difference between failing or succeeding as a technology company.
Another perspective here is that barring any technical moat, a well funded VC-funded team could probably re-create whatever you do in no time at all. If your tech is blocking you from moving ahead, it can be sobering to consider how long it would take a team unburdened by technical debt to catch up with you and do it better. Because, if the answer is "it wouldn't be that hard" you should probably start thinking about abandoning whatever you are trying to fix and maybe rebuilding it better. Because eventually somebody else might do that and beat you. Sometimes deleting code is better than fixing it.
There are lots of good reasons tech debt exists, and it's worrying that this person seems to think that they all boil down to "I don't know how but someone, somewhere, fucked up"
https://www.amazon.com/Peopleware-Productive-Projects-Tom-De...
* Conway's law causing multiple different data science toolchains, different philosophies on model training, data handling, schema and protocol, data retention policies, etc.
* Coming up with tech solutions to try to mitigate the impact of multiple silos insisting on doing things their own way while also insisting that other silos do it their way because they need to access other silos' data.
And the reason standardization won't happen: the feudal lords of each of those branches of the hierarchy strongly believe their way is the only way that can meet their business/tech needs. As someone who gets to see all of those approaches - most of their approaches are both valid and flawed and often not in the way their leaders think. A few are "it's not going to work" levels of flawed as a result of an architect or leadership lacking operating experience.
So yeah, it might look like technical problems on the surface, but it's really people problems.
Then the author suggests that senior leadership without a tech background will usually need to be persuaded by a value proposition - the numbers.
I'm seeing these as the same thing - the risks of specific tech debt just needs to be understood before it gets addressed. Senior leaders with a development background might be better predictors of the relationship between tech debt and its impact on company finances. Non technical leaders just require an extra translation step to understand the relationship.
Then considering that some level of risk is tolerated, and some risk is consciously taken on to achieve things, both might ultimately choose to ignore some tech debt while addressing other bits.
Management claims to want to understand and fix the problem, and their "fixes" reveal the real problems. Fix 1 - schedule a lot of group meetings for twice a week. After week 1, management drops off and fails to show up anymore for most of them. The meetings go off track. The answer? More meetings!
We now have that meeting daily. And have even less attendance.
Fix 2 - we don't know what people are doing, let's create dashboards. A slapdash, highly incorrect and problematic dashboard is created. It doesn't matter, because none of the managers ever checks the dashboard. The big boss hears we are still behind, and commandeers a random product person to be his admin assistant and has her maintain several spreadsheets in semi-secret tracking everyone's progress.
This semi-secret spreadsheet becomes non-secret and people find a million and one problems with it (not surprising as the commandeered admin assistant nee product person was pulling the data from all sorts of random areas with little direction with little coordination with others). We then have the spreadsheet war of various managers having their own spreadsheets.
Fix 3 - we are going to have The Source of Truth for product intake and ongoing development, with a number of characteristics (and these are generally not terrible characteristics). These are handed off to a couple of junior people with no experience to implemented with zero feedback. The net result is we still don't have a Source of Truth, but more of an xkcd situation that now we have 4 or 5 sources of truth strung together with scripts, duct tape, bandaids and prayer.
This continues on and on over years. Ideas are put forth, some good, some bad, some indifferent, but none of them matter because the leaders lack the ability to followup or demonstrate even basic understanding of what our group actually does.
It is truly soul crushing, but in this jobs environment, what are you going to do?
Peak absurdity, yet you and I have both seen it happen first hand. I wonder how common it is, because it's as ugly as it is mystifying.
When management isn't properly engaged, they need to delegate to someone who is. If neither things happen, it's just chaos and angry, ignorant apes making a lot of noise.
> in this jobs environment, what are you going to do?
I am currently unemployed with a rapidly shrinking cushion, and I'm honestly on the fence as to whether putting up with the above would be better. If there is no hope for improvement, all you're doing is exchanging your mental health for a few more beans.
"But what about using a message queue.."
"Candidate did not use microservices.."
"Lacks knowledge of graph databases.." (you know, because I took a training last week ergo it must be the solution).
Calling them 'people problem' is a convenient catch-all that lacks enough nuance to be a useful statement. What constitutes good communication? Are there cross purposes?
> Non-technical people do not intuitively understand the level of effort required or the need for tech debt cleanup; it must be communicated effectively by engineering - in both initial estimates & project updates. Unless leadership has an engineering background, the value of the technical debt work likely needs to be quantified and shown as business value.
The engineer will typically say that the communication needed is technical, but in fact the language that leadership works with is usually non-technical, so the translation into this field is essential. We do not need more engineers, we need engineer who know how to translate the details.
I realise that, here on HN, most will probably take the side of the rational technologist, but this is a self-validating cycle that can identify the issue, but cannot solve it.
IMO, we need more generalists that can speak both languages. I have worked hard to try and be that person, but it turns out that almost no-one wants to hire this cross-discipline communicator, so there's a good chance that I'm wrong about all of this.
The solution usually isn't "better people." It's engaging people on the same goals and making sure each of them knows how their part fits with the others. It's also recognizing when hard stuff is worth doing. Yeah you've got a module with 15 years of tech debt that you didn't create, and no-one on the team is confident in touching anymore. Unlike acne, it won't get better if you don't pick at it. Build out what that tech debt is costing the company and the risk it creates. Balance that against other goals, and find a plan that pays it down at the right time and the right speed.
Because they want to feel superior as the ‘this was my idea and you executed on my idea’ nonsense. Their answers to most ‘why are we doing this ?’ ‘trust me bro’. I am perhaps generalizing and there are outlier product managers who have earned the ‘trust me bro’ adage, but most haven’t.
This PM behaviour will never change. Engineers have said enough is enough and are now taking over product roles, in essence eliminating the communication gap.
Remember that most communication is non-verbal?