"Financial debt, however, accrues interest and hurts more regardless of what it is incurred for."
I'd disagree, I think all technical debt accrues interest (similar to financial debt), but the interest rate is different. As the example about a usps api demonstrated, that has a near 0% interest rate. Of course if you were ever to need to update it (maybe usps updated its api) you'd get hit. A bad domain model has a high interest rate. All the ideas are the same though, i'm just being persnickety :D
A problem with comparing the metaphorical debt of technical debt to the financial one is that interest rates can be negative.
In addition, the term "debt" also applies to moral aspects, and not just financial ones. A builder may come back and improve something not because it's good for the employer or has financial benefits, but because of a belief that quality is good for the soul of the builder. That is also an aspect of technical debt.
Regarding the original article, it says tech companies are "successful because they got a product to market fast and at the right time". The "fast" is incorrect. Companies are successful if they are at the right time. If a company (like GO Corporation with pen-based computing) is fast but too early, they they are not successful. If a company (like Microsoft with Excel) is slow but eventually gets there, then they are successful.
"Fast" is therefore irrelevant in the face of "the right time." It can be that the right time is "as soon as possible", but that's not intrinsic to success.
>If a company (like Microsoft with Excel) is slow but eventually gets there, then they are successful.
Excel was successful because Lotus decided 123 had so much technical debt it would be a good idea to "upgrade" the old assembler code to C. This took at least a year longer than planned, and introduced a lot of bugs and incompatibilities.
Meanwhile Windows had arrived, and the planned Windows rewrite of 123 never happened as scheduled.
So it wasn't so much that technical debt killed 123. But poor management of technical debt certainly did.
Effectively the C rewrite was a complete waste of time, and Lotus should have aimed for a Windows release as soon as they could.
It's nice to be able to say this with hindsight, but it was doubtless much less obvious at the time, when Windows was barely considered a mediocre visual DOS shell, and certainly not a serious OS.
Bottom line - you have to be smart, prescient, and a little lucky to manage technical debt. It's not just about getting the code right - it's making decisions with educated guesses about where you may be a year or two from now.
I don't believe that, even in the short run, taking shortcuts actually saves you time. It's much like working 12 hour days - you feel more productive, but you're not actually getting any more done on a larger scale. There's an obsession with 'hurry up and get it done' that contributes little on a larger scale.
I also don't believe it's possible to pick the 'important' areas of code a priori and make only those areas free of technical debt. Doing good work is an attitude, a mind set, and something to practice - if you practice making poor quality code, it's going to affect you across your work, not just in the places that you do it.
I think that, by being professionals and putting aside the 'hack it out' mentality, we can improve not just the code that we're working on, but our mindset towards it. How many codebases have 'that scary part' that adds background anxiety to the people that work on it? Is that something you want to invite into your life on an ongoing basis?
Especially for things as important as integrations into external APIs - sure, they may not change much, but in my experience you really want to make sure they're done right when they're how you deliver your product to customers. Given how error-and-failure-prone many external APIs are, I expect a lot of work there just to robustly handle the other end of your integration being flaky.
I think that quality is something you can't 'bolt on' later. It has to be built in, top to bottom, throughout your organization, code base, and personal skills.
But often the choice is: do I implement this in an hour, or do I spend a week making a nice well-though-out framework for this, complete with test-suite (which may take multiple iterations to reach perfection).
And you may pick the latter, but what if management wants to make a quick prototype and turn that into a product later? At that point you'll still be having the "technical debt" talk with your manager.
I think that's a straw man argument - the choice isn't between an hour and a week, it's a choice between an hour and two or three hours. I don't think that making a framework for every change is a good idea, but I do think that, if you've only got an hour to work on something, you'd better make some quality tests for it, because under time pressure you're even more likely to make the sorts of errors that tests catch.
That said, I'm not implying that you should slow down, just do the best possible work at all times. Don't do a crappy job just because you're time-limited - we all only have 24 hours a day, and there's always more work than can reasonably fit in. The moment you start going "I am going to lower my standards right now", you've started a trend.
I think it's interesting that people are always saying "hire the best possible people you can", and then saying "but don't ask them to do the best possible work they can do".
But a good programmer always sees a number of solutions to a particular problem. Let's say he sees 10 solutions. Solution 1 costs an hour to implement, but gives crappy code and he knows it. Solution 9 gives good quality code, but takes a week to implement, and he then has to sell this to his boss.
And then there's solution number 10, which requires the programmer to invent a new programming language, and requires at least a couple of months to develop. "You hired the best possible person for this job, so let me do the best possible job".
I just bet most people here wished they could always deliver the best possible code.
I think what you hear me saying is "never do anything less than perfect work". I'm saying "given your current constraints, don't choose a worse option when a better one is available". Those constraints include time, your skills and beliefs, the environment you work in, the project you're working on, and the problem you're trying to solve.
>But a good programmer always sees a number of solutions to a particular problem. Let's say he sees 10 solutions. Solution 1 costs an hour to implement, but gives crappy code and he knows it. Solution 9 gives good quality code, but takes a week to implement, and he then has to sell this to his boss.
A good programmer will try and improve code the code incrementally, and won't bother trying to sell anything.
It's rarely that extreme a difference in time, but even if it were, what you are really talking about isn't the whole story, except in truly throw away code (almost never happens). It's more likely to be something like:
1 week today to do it properly, a couple of hours a month from now to tweak it, a couple of more hours 6 months later to add a new tested feature,
vs,
1 hour today to hack it in, 3 hours on friday to rework it, 1/2 a day next week to debug a problem, and hour the next day to fix your fix ... plus the entire following day when that didn't work properly
Followed by 1 day to do a "proper rewrite" in a month, to add the tweak that should have been easy but breaks some of your assumptions plus 2 days to debug why something doesn't work properly anymore interacting with another subsystem.
Followed by someone else spending most of a week debugging a subtle bug you introduced by stepping on some state "nobody was using anyway"
Followed by 6months later when the new feature is needed someone looking at all this and throwing up their hands because they can't understand what it's doing. At which point they say... "screw it, I can re-write this in a couple of hours".
Lather, rinse, repeat.
Most of the time, the "fast way" is pretty reliably going to cost you more time. Just not more time today.
> I think that quality is something you can't 'bolt on' later. It has to be built in, top to bottom, throughout your organization, code base, and personal skills.
I agree with all of your points and just wanted to add one more thing that doesn't seem to be addressed much. The author mentioned technical debt didn't have an "interest rate", but it absolutely does. As soon as you start to add debt to a particular part of the code base, it must be paid back immediately the next time you or someone else has to work on that same section. This may genuinely not matter for a one-off script, but if it's code that will ever be worked on again it can start a snowball effect that causes the interest to compound and usually manifests itself as code that's buggy, unstable, and hard to work on.
What do you think of this extension of your analogy:
Financial debt has an interest rate, but you take it out because you believe the return on the proceeds raised adds more value than the cost.
Technical debt is meant to (ideally) be the same. As long as it's outstanding and compounding, it should be because your return on business value will be lower if you "pay it back", then if you do not and spend time/resources elsewhere.
if your discount rate is 30%+ per annum, this makes for some counter-intuitive conclusions about when it's smart or not to "finance" with technical debt.
It strikes me as a little absurd to say that taking shortcuts is always a bad idea. Let's look at a few shortcuts you might take.
Allow the page to break if the user's browser does not support websockets
If you know that 95% of your target audience uses modern browsers this seems like a reasonable compromise.
Don't write enough unit tests
Maybe you're using a library that is difficult to mock. Maybe most of your logic is happening at the db layer. Eventually you may want to thoroughly test this piece of code, but it's a ton of work to mock a database in a meaningful way.
Write a monolithic service instead of a modular one
While it's bad to let a single service grow too large, when you're starting out, it can be helpful to write it all out, and _then_ see how it can be broken apart. (It's often hard to foresee everything at the design stage.)
Finally, a lot of code ends up being never used. Nobody cares about well-factored well-tested code that isn't used. Sometimes a dirty version is the quickest way to tell if a project makes sense at all.
"Maybe you're using a library that is difficult to mock. Maybe most of your logic is happening at the db layer."
At this point, your design is already screwed and borderline-unmaintainable, so yeah, I guess unit tests won't help much. But when people talk about Doing it Right, usually they mean having a codebase that's loosely coupled enough that tests are relatively simple to write.
While I agree with you on time vs. shortcuts, I do think it's a question of valuing explicit time vs. opportunity cost that's a depper issue here.
My opinion is that whether it's programming or general white-collar work, most cases of doing it "the right way" vs. "the dirty way" tend to be about investing the "certainty" of lost time immediately for the "promise" of savings.
For some things, the tradeoff is pretty decent (taking time to improve core knowledge or relevant skills that are related to goals).
But for a quickly growing company, how many situations are there where a "hack it out" fix that costs time in the long run is a net win in opportunity cost and hence long-term value?
If a shortcut that will cause you headaches in 9-18 months also allows you to add critical features at an accelerated pace for the next 6 months, is it:
(1) inefficient because you net lose man-hours or
(2) efficient
(a) maximizes your near-term growth trajectory,
(b) thereby increasing potential access to resources and hence long-term growth
To be clear, I do agree with your statements about shortcuts usually not saving time especially when you deal with teams/setting standards.
My point is more that while time saved/lost is easier to mentally calculate, it alone is not really what's driving value in many situations.
There's also the inherent problem of using qualitative terms that mean different things to different people.
That's a dangerous attitude that makes sense for some startups, especially those with high burn rates.
"If we don't get traction/product out the door now, we won't be able to raise our next round, so we have to get this product out the door yesterday to de-risk the whole company falling apart when we run out of money in a few months." Then later you raise your round and do the same thing for the next round. Eventually, of course, it will catch up with you and slow key progress. If you're unlucky that leads to a down round and you're all screwed.
I see that as being a larger risk than competitors moving faster in the sort term (them moving faster short term may mean you move faster long term). Higher burn rates mean you have a clearer deadline for fundraising, means you're racing the clock, means you're incentivized to take on technical debt. I'd love to have a look at what really tends to kill companies more often, but it's just anecdotes and speculation from me today.
It's definitely a dangerous attitude, this is also the problem with anecdotes, we all start postulating strawmans.
I would point out though that in the financing-focused case you gave, although you have a definitive slow down from technical debt, it's still an open question whether or not it's actually a net slowdown vs the "status quo" scenario.
In practice of course, every individual company, with different teams and different contexts, has to find the right "cognitive bias" for themselves.
Everyone thinks they can pay after the release. If you cut those corners hard, though, you'll often regret it before your release. Cut harder still, and you won't even make it to the end-of-sprint demo before there's a knock at the door.
Systems take too long to get to market, which causes
marketing efforts to stall or become irrelevant, and
competitors beat out the startup to an offering.
I disagree. Well, maybe I agree in part, but I feel like this misses the forest for the trees. Getting to market fast helps you get in front of users (and prospective users) faster, which—assuming you're listening to them and iterating—helps you achieve product/market fit before you run out of capital or a competitor does the same.
The article applies mostly to short-lived webcrap and appcrap, where time to market matters. There's software where reliability matters more than features - databases, for example. Source code repositories. Banking systems.
Soon, automatic driving. (The coming "Internet of Crappy Things" is a real worry.)
I think that quality matters even in 'short lived webcrap and appcrap' - you only get one chance to convince your customers to trust you, and if your application even so much as behaves subtly wrong they're gone. I think that's the class of error that 'hack it out' most causes - it's not an application error, it's an error of logic or off-by-one or floating-point-math-where-it-should-be-exact.
Time to market always matters. If you created the most reliable and solid automatic car; one that never crashed or made any mistake, you would have an amazing product.
If you released it today, you would own the car market and you would be best pals with Elon.
If you released it in 50 years, people would think you are quaint for playing with that old-tech for a problem that been solved 10x over.
The author seemingly misses an important way in which this can be looked at as "real".
Some subsystems don't change much because they already do what is needed and aren't broken. Sure, the code could be improved, but it's not important enough to do (or at least, to do now).
Other subsystems don't change much because they are scary to change. If the system is brittle and poorly designed, engineers may have very good reason to be hesitant about changing it, and instead it is worked around.
So this latter case behaves very much like "technical debt" . This system costs you engineering time every time you even think about touching it. It's hard to reason about, it's hard to change, and over time these problems get worse if you introduce workarounds in other parts of the system, because now they are worse and harder to work with.
This is a good point, but I think it's very much in line with what the OP is saying. Technical debt hampers your ability to change. Anything you're scared of changing clearly fits that description.
Those workarounds that get created instead of changing the scary thing put me in mind of the "scar tissue" metaphor that SapphireSun proposed elsewhere in this thread.
I'm not disagreeing with the main thrust of the OP's argument, but with the statement that "Technical Debt isn't real".
I think that it often is best thought of as "real debt" in that it costs you time and attention, and (at least some of the time) that cost compounds over time. In other words, some of your engineering spend is just going to service this debt, not produce anything useful. That's a very real opportunity cost.
Even the code for the subsystems that work could be improved and turned into real dollars. I've seen it in so many projects where a commonly used web page loads very slowly. On an ecommerce site, seconds and milliseconds count. Optimizing those pages with caching would have provably increased sales (we saw sales go up on a page that was "accidentally" optimized ;)).
It probably could, but every hour spent on that has an opportunity cost. You might find the same hours spent elsewhere do more for you.
If you're doing your planning and analysis right, you should hope to have identified this... so in your example, hopefully that subsystem went up in priority because it was identified as a place for gain.
There are some things I agree with in this, however I think the context in which it is presented is flawed. This is particular: "Technical Debt is a Positive and Necessary Step in software engineering" is just incorrect. If technical debt was positive and necessary we would call it Technical Credit.
IMO a better context for this would be: real software engineering requires compromises. There are a number of competing factors that need to be juggled in order to be successful: time, money, complexity, quality, etc. The point should be that you need to balance these factors, not focus on one above all else.
From years of experience, I detect in the tone of this piece a certain notion that product managers / non-technical people tend to form when working with developers - namely, that developers don't care about speed-to-market and would rather agonize and winge over perfect architecture and coding standards... What I think you'll find, more often than not, on the business end of this attitude is a developer(s) patiently trying to explain that speed-to-market is, in fact, directly related to the technical debt accrued on the project for all but the most static and inconsequential things. We must think and communicate about technical debt the way we do to keep errant PMs from drifting off into the Ice Cream Sea of Candyland where all of there half-thought-through ideas they demand be implemented immediately by cutting all corners have no impact on the next set of half-thought-through ideas they want implemented.
I don't think the metric 'any code that decreases agility as the project matures' is quite the same thing. I think most code lowers agility, as it is increasing complexity and the number of interactions and entanglements across your codebase. Adding features decreases agility. These are not 'technical debt'.
Specifically, he is NOT talking about 'dirty code' or just badly written code, that's taking shortcuts and I believe it is never a good idea in software. We're talking about building good code but doing so before we fully understand the problem, often with the aim of learning more about the problem, especially from user feedback.
What I take Ward's discussion to mean is that "technical debt" is accrued when writing code for a problem you do not YET completely understand (e.g. almost all 'first pass' code). The debt occurs when your code's existing understanding is static (because it is in source control), and your teams understanding grows. The difference is your debt. You can continue building on the code's not-quite-right understanding for the moment which is carrying the debt forward and paying the costs of fighting a not-quite-right model, but at some point it becomes worth while to re-factor based on what your team has learned about your problem, and thus pay off the debt.
I don't necessarily agree that adding features decreases agility. Sometimes it can massively increase agility - especially if your product is built in a way that a new feature can immediately gel with other features.
But apart form that, absolutely. Technical debt in the article isn't the technical debt that was originally coined.
I still think the common meaning of technical debt is just as important though. Code that can't be taken forward, for whatever reason, costs its owner.
From years of experience, I detect in the tone of this piece a certain notion that product managers / non-technical people tend to form when working with developers - namely, that developers don't care about speed-to-market and would rather agonize and winge over perfect architecture and coding standards... What I think you'll find, more often than not, on the business end of this attitude is a developer(s) patiently trying to explain that speed-to-market is, in fact, directly related to the technical debt accrued on the project for all but the most static and inconsequential things. We must think and communicate about technical debt precisely the way we do to keep errant PMs from drifting off into the Ice Cream Sea of Candyland where all of the half-thought-through ideas they demand be implemented immediately by cutting all corners have no impact on the next set of half-thought-through ideas they want implemented.
New PMs are like children, they tend to only learn the fire is hot by actually getting burnt.
Instead of digging my heels in on not giving such PMs everything they want, I try to offer alternatives and put the choice on them: they can have this feature today and in 6 months we grind to a halt because we can't fit in anything new, or we slow down just half a beat and do it right the first time. "Up to you, buddy," I say.
It happens with jr. devs, too. They want to use a singleton to solve a problem, or they want to forgo using any sort of referential constraints in their schemas.
Of course, the PMs and jr. devs always choose the shortcuts. I help them both pick up the pieces afterwards, but I never work overtime for it. The point is to make it clear that nobody other than themselves is responsible for their own mistakes. They led us into the mess, they can lead us out.
After that, everyone seems to get along a lot better and everyone seems super keen on having discussions about the "right" way to do things and how to fit things into the current state of the project correctly.
|| they can have this feature today and in 6 months we grind to a halt
Buy some daisies and chocolate for the PMs you work with... because here's how that would go over with the ones I've had:
PM: "Yes! Cut all corners and get it done today!"
...6 Months go by...
PM: "I came up with a new idea for that! Today, please!" DEV: "But remember..."
PM:"Today, thanks!"
You do a job. There is a reason you are there, and it's not to write code. It's to fulfill requests. Feel free to provide advice on how best to fulfill those requests, provide feedback, ask for clarification, etc. But at the end of the day, all you do is what you are told. It's not to anticipate what they will want in 6 months, because you'll be wrong just as often as right and you'll be in the same scenario. Just do your job.
So when you're told to cut corners today, then in 6 months you can't make magic happen to grant their every whim, you have to let it blow up. They want to be the leader. They are responsible for the failings of the team.
You have to play chicken with them. Because every time you capitulate, all they know is "I got what I wanted". They don't care how much you complained or how many hours off the clock you worked. And honestly, if you tell them it can't be done overnight, and then you do it overnight, you're a liar and they won't trust your judgement ever again. When you say, "can't do it", you have to not do it. Don't make magic happen.
Or just don't work at such places. I have thankfully been out of that bullshit for three years running now.
That's the everyday struggle; as developers we can produce so much wealth just by typing at a keyboard. Yet we let ourselves be controlled by people who don't know what's what and can't even prove their ideas have any return on investment.
As you say, don't make magic happen. It doesn't matter how cool it would be if, or how you think you'll show them your ideas are better; when you say "it cannot be done" don't go and try and do it. Don't let the manager bully you and don't let them try and get another developer to tackle the impossible.
I've had that happen to me where I said it will take X amount of time to do it so let's not do it till next sprint. Then the manager went around my back when I was gone for a day and gave the task to another dev. The dev, who caused the mess, of course was able to fix it within an hour. Why did I say X amount of time? To cleanup and make sure the bug stays fixed.
When you work with other developers, actually work together. Stand up for them, don't make bad comments about them, and always stick together when you're asked to do the impossible.
If I understand your story correctly, that other dev accomplished the task in some characteristically short amount of time, plus an hour for fixing a bug.
While you didn't specify what your "X" was, it sounds like the manager (and the company) got a better outcome by going around you and it was far from being "asked to do the impossible".
I'm not going to slag on the developer; the manager got a better short term outcome. The longer than 1 week outcome is that I've had to go back and re-visit that code and make sure it's "done done for reals it's done" due to the lack of unit tests and thorough thought given to it.
On a long enough timeline, the better outcome turns into multiple bugs.
This is what I'm talking about with offering the alternatives. You can't hide the distasteful "make the pain go away, but not fix the underlying problem" option. You have to give people enough rope to hang themselves, which is thankfully also the same thing as giving them the opportunity to surprise you with their leadership. When you have a heavy-handed PM over you, you have to give them all the information, force the decision on them. Don't make the decisions for them that option A or option B is best. If they want to lead, they have to take responsibility for this stuff. It's up to you to present the options appropriately. "A) I hide the bug, maybe takes an hour, B) I fix it, probably takes a week". You don't know, you might be surprised, he might tell you "do both, A right now then get right on B." Or he might only be interested in A. Either way, if you don't want responsibility for the failures, you have to abdicate responsibility for the successes.
> That's the everyday struggle; as developers we can produce so much wealth just by typing at a keyboard. Yet we let ourselves be controlled by people who don't know what's what and can't even prove their ideas have any return on investment.
That's the everyday misconception. Nobody produces wealth just by typing at a keyboard. Products don't design and sell themselves and it's arrogant to pretend that product people and salespeople aren't a huge part of the value creation process in tech.
There are of course developers who are capable of identifying a market opportunity, conceptualizing a product to exploit the opportunity, building the actual product and getting it into market successfully.
But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
> That's the everyday misconception. Nobody produces wealth just by typing at a keyboard. Products don't design and sell themselves and it's arrogant to pretend that product people and salespeople aren't a huge part of the value creation process in tech.
Okay you've got a point, it is fairly arrogant; on the other hand, any sort of automation or small script or product is valuable enough that we don't need to be subservient.
> But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
That's what I'm asking, why are talented developers allowing themselves to be subservient? That's why I'm wondering why technical managers all the way up to the CTO level are subservient to other departments when it's the product based on technology that determines the success of the company in the end?
And actually, your idea is optimistic; I wouldn't mind low six figures for 10-12 hours a day ;) maybe that's why we're so subservient, we look at our peers in other industries and they're making much less cash money and it's easy to see that we're lucky to be working in a nice air-conditioned office working with something we're typically passionate about.
> That's what I'm asking, why are talented developers allowing themselves to be subservient?
Answer: trying to get a real product to market successfully requires a lot more than small scripts that automate processes.
It's not that automation and the like doesn't have value, but ask the average developer to get on the phone and sell something, or to have a meaningful conversation with a customer, and you might start to understand why other people in an organization (salespeople, product managers, etc.) have some sway too.
If you simply put a bunch of talented developers in a room, chances are you're not going to end up with a successful business.
It sounds like you have a bad relationship with your PM. I've been in that situation before (seriously, check my post history), and it poisoned my attitude for quite a while.
There are a lot of terrible PMs out there, but there are also a lot of really good ones, and you learn to suss out which is which when you're interviewing for a new team. If you read Rands in Repose and start wishing your PM actually thought and cared about the things he talks about, maybe you need to find a new job.
From years of my experience, both in big corporate tech and startups, smart Engineers will always find business reasons to sell you their tech-debt free perfect rainbow unicorn architecture.
Even worse, if you're then not equally technical as a manager you will never know whether you've made a solid choice by aligning with or against the engineer's view, and even worse your engineers will quickly take on a "see, I told you so" stance should things work out not perfectly under an agile model.
I've seen entire products - established, successful products - sunk by a single decision to let the engineers build a gold-plated replacement for some component that had a messy architecture and eye-watering code on the inside but otherwise wasn't broken and didn't need fixing.
This is a spot where I'm definitely in line with the Scrum position: If it's really causing problems, then the developers should have no problem making a reasonable business case for fixing them. If working on it's really causing that much pain, then the developers will figure out a way to quietly fix it in a way that doesn't (and needn't) vex the product owner. If neither of those is the case then you best let sleeping bears lie.
Just because you don't think anything is wrong in terms of external acceptance criteria, does not mean that there is not something wrong with the code that is of real consequence.
Contrarian anecdotes aside, the reason everyone knows the phrase "technical debt" is because SO MANY of us see projects wrecked by greedily demanding many features and changes as fast as possible in a continuous crunch mode, without considering long-term costs like defect rate or turnaround time on bugfix and feature implementations... or developer burnout.
It's better not to assume that developers are blathering when they ask for formal permission to carry out QA or maintenance tasks or upgrades, if they are the people who know what the work looks like on the ground level. If you are putting lots of time pressure on developers and implying that heads will roll if aggressive deadlines are missed, it doesn't make sense also to leave it up to those developers to magically find the time to fix lurking internal time-bombs.
This makes products worse, it makes developers' lives worse and developers are stuck with the blame for something they can't control. Maybe as a PM or whatever that suits you, to dump all the risk in the laps of your developers. But if you are yourself an engineer you should not make yourself part of this problem.
I wouldn't want to make myself a part of that problem. If I found myself on a project where the business folks didn't care about things like defect rate or supportability, I'd be looking for a new job pronto. Those things aren't invisible to the business, they're a core part of the customer's value perception. A product manager who doesn't understand that is a product manager who doesn't have the slightest idea how to manage a product.
Similarly, living in continuous crunch mode is something I let myself get bamboozled into quite a bit in my younger days. I bake time to leave the camp cleaner than I found it into my effort estimates, and I know how not to let myself be pushed past my limits. If I worked for a place that couldn't respect that, I'd also be looking for a new job pronto.
I hope that you were being hyperbolic for rhetorical effect in your post, but if your work environment is really like the situations you were describing then I would also encourage you to be looking for a new job pronto. Life's too short to waste on that kind of stress.
Words of wisdom for sure. I'll add that voting with your feet is a common thing for developers to do, and poor management, being poor at management, won't notice it for what it is.
Eh, referential transparency is a thing. And that idea can be extended beyond a simple function.
It's totally possible to contain horrible sins behind a clean API. I've done horrible things, but those things don't leak all over the whole project.
On the other hand, i worked at the same place for a long time. I got to see the benefits and consequences of my mistakes and successes over years. I do think i burned out. It took me a long time to learn how to contain nuclear waste.
In the end, it's easier to just work someplace that doesn't put crazy demands on developers. There's a continuum of perfection <-> functionality. unfortunately for new developers, it takes a while to figure out where you lie on that line. Finding a place that's close to your preference is really best.
It's kind of like driving. Everyone slower is a Sunday driver, everyone faster is a lunatic with a death wish. I'm happy with homogeneous drivers - too far from the mark and i get nervous. I'd bet it's the same for pretty much everyone else as well.
Had a startup fail because we tried to do the right thing instead of "Fuck it, ship it". Worked at another place where the codebase finally curled up into a ball and refused to accept any refactorings short of a complete rewrite of some core systems--at which point we discovered some nasty bugs in production that we punted to our support folks to deal with. I've no idea if they ever eventually sorted it out.
Good engineers will recognize when it's "good enough". However, that requires biz folks that can provide useful feedback and actually get things in front of customers and get paid. Because, honestly, if we're like three extra features in, have no validation, and are cutting corners, you're damned right I'm going to put on the brakes and redirect the effort toward paying off technical debt.
Because, and this can't be overstated enough, whether or not the business fails in six months is not something we engineers can control--but I'll be damned if I'm going to come in and keep sinking my dwindling life force on a fucking technical lemon. If your biz folks can't establish trust in their abilities, then you might as well hone your craft.
EDIT:
Large amounts of technical debt are a sign of either rapid growth or bad management. If the company is growing, we can overlook the debt every time we, say, waste 30 minutes waiting on a build or fixing a weird nested CSS bug or some other damn fool thing. If the company isn't, then we lose respect for everyone above us in the technical leadership, because they let--by deliberate neglect!--the situation get so bad.
A simpler way to look at technical debt is to ignore all kinds of dubious notions of "quality" and "correctness" and focus on lines of code.
The more lines of code you have, the more stuff you have to maintain and understand.
Of course this is pretty stupid, but it's stupid in a way that sometimes leads to insight.
If your application consists of 1000 lines of code, you can rewrite it in an afternoon.
Also, cleverness can create its own kind of debt. Even as you believe you're working on reducing debt by creating some clever and sophisticated abstraction, you are actually just making it worse. I think this happens all the time.
That's why my basic, fundamental approach to software development is "throw it away and replace it with a simple shell script."
technical debt is actually worst than financial debt. imagine an interest rate of x%. now imagine that one day the bank comes in and says: now your interest rate is going to be 20x. you don't get a say in it and you cannot refinance.
also another mistake I've seen made over and over again is to say stupidity==technical debt. If you're doing stupid things just to 'get things done' you're doomed in the long run. If you've thought about it and made the decision to cut a corner you should know why you've done it.
Either this was poorly written or the author just doesn't get it. Developers shouldn't do TDD or 100% code coverage because they want to build a perfect system; developers should do TDD because they understand that they can never build a perfect system and the T in TDD protects them from complete failure. This means you never attempt to build a perfect system; you only attempt to do just enough. What is just enough? It is the amount of functionality to make the tests in TDD pass... never the amount of functionality to make others 'fall down on their knees and claim “we’re not worthy!”'
Thought I'd also add that technical debt is unavoidable, but we realize it is not OK to have technical debt (the article concludes that technical debt is OK) and that is why we back it up with tests. These tests are like collateral for a loan (technical debt).
I hate to put it this snarkily and I fully expect karmic blowback, but I'll just go out on a limb and say that what we need more of is not hastily shipped crapware that barely functions and if successful keeps a legion of smart engineers busy, cursing, hacking, grinding teeth and tittering.
No, what _you_ need is more hastily shipped crapware to capture markets from your competitors so you can make more money and disappear. Hastily shipped crapware has enabled lots of new and interesting things. It also keeps 10x as many engineers busy serving it as would moderately better thought out, slightly later shipped software. Babysitting crapware isn't bad money for an engineer, but it's cosmically questionable as to whether it really advances us more than its opportunity cost. And hastily shipped crapware at scale is a real giant energy problem. Take a look at the size of a Facebook datacenter and tell me again how a metric fuckton of PHP is a good thing for the environment.
I agree with the author more or less at least with respect to compromises between software quality and speed of execution. Code does not have to be perfect, especially when it doesn't affect the end user.
On the other hand this is what I miss most about school, or perhaps just doing things out of my own interest.
There was a time when I could appreciate the beauty of code and all I could worry about whether my software was elegant. Then I joined industry and everything changed.
I'm unconvinced that accepting large amounts of technical debt allows startups to react faster.
I've definitely seen startups that have almost found a good idea, but can't tweak it, scale it, or make it stable, because they're overwhelmed by technical debt and even simple changes have become engineering death marches.
Letting your back-end code turn into a complicated, disgusting mess means that adding critical features may become a multi-week nightmare. Which is then followed by a series of additional multi-week nightmares for each successive feature.
Now, this isn't to say that obsessive code-polishing is a good idea, either. But startups require rapid iteration, and you can't iterate especially rapidly when your code is complicated, badly organized, and poorly tested.
One of the most valuable startup skills is to know which kind of debt has a low interest rate and which kind of technical debt has a high interest rate. Certain things - lack of tests, poor build system, poor deploy system, very poorly structured code - have a very high interest rate. Every time you create bugs, you waste time tracking it down and fixing it. Every time you check something in that breaks the dev environment for the rest of the team, you create bugs. On the other hand, messy code that is isolated to one system, or an architecture that has a bit of copy and paste, are usually not too big of a deal. Going on some deep dive to create some "generalized framework solution" is almost always an error.
Some people talk about Reversible Decisions, and I think what you're saying plays into that sentiment.
Unfortunately some people don't realize that in many situations not making a decision is itself a decision, and so they don't always notice it when it happens. That can be anything from the ones you mentioned, like testing and tools, to authentication, auditing, localization, robust error handling, resource/memory leaks, or monolithic designs that prevent scaling.
> I'm unconvinced that accepting large amounts of technical debt allows startups to react faster.
Accepting large amounts of technical debt enables you to deliver faster if your initial assumptions (both about what you want and how to achieve it) are correct, but slows you down if any of those turn out to be wrong.
OTOH, it might offset that slowness in whole or in part that by speeding up your ability to determine that they are wrong.
>For instance, if you’re using a relational database, and your models are improperly structured, this will cause you significant pain later on. If you build a system using NoSQL that later needs to become highly relational, you will incur massive frustration in the future.
I don't think this is what Linus was talking about...
In the past 15 years I've been in a situation twice where shipping something NOW was more important than taking the time to do the needed cleanup. Simply because the competition was breathing down our neck, and taking a break to refactor stuff would just give them an opportunity to bury us. And who's going to care about your code quality then?
And it wasn't just crappy web apps or throwaway MVP stuff. The first one was in medical systems, where the suits figured we had a 6 month lead on our closest competitor, the other was in consumer electronics (navigation devices), in the time that competition was still stiff in that sector.
In both cases it was decided that releasing something was more important, knowing full well that we had to pay the cost later. Of course not everyone agreed with it, since no one likes having to work with legacy crap, and everyone and their mother always wants to refactor all the things, but guess what: code quality is not the goal of your company, making money is.
I'm starting to believe that debt is the wrong metaphor. Imagine you have 1k in debt and 2k in the bank. You're free to pay down that debt, so you do. If you later realize that you actually need that 1k used for the debt, you can simply take out a new loan and effectively undo that step. This is precisely where the tech-debt analogy breaks. When you invest a month into fixing tech debt, you can't get that month back. That decision is final.
IMO This is all you need to know about "technical debt"
1/ Poor code =/= technical debt. Far too often, poor code masquerades as "technical debt"
2/ Good engineers do the right thing which is always a judgment call based on circumstance & culture.
3/ Take guidance from your best engineers, and find a way for the business to adapt. Much easier to change the sales deck or pricing today, much harder to overcome bad technical decisions. No one ever said "I never knew that sales deck or pricing would last this long" but many say this about software. Be super careful & deliberate.
4/ Build things that are simple (not easy), and respond to demand. Focus on movability & robustness, not "perfect" design.
Your #1 is spot on. Every "grind-to-a-halt" scenario I've been in has been because we expected X number of users or Y amount of data and once we released it turned out to be 100 times those numbers. The system you build for 10 internal users is going to look a lot different than the system you build for 1,000 users on the internet, and that's going to be completely different than what you build for 1,000,000 users. That doesn't make the small system poorly designed, it just makes it inappropriate.
Interesting. Not my experience. Every grid to halt scenario I see or hear of is just bad software decisions made under haste without deliberation, or foresight beyond initial launch (no attention paid to supporting it, let alone scaling)
Ergo, in your scenario even with 10 users, the application basically died under its own weight under nominal use.
113 comments
[ 2.6 ms ] story [ 202 ms ] threadI'd disagree, I think all technical debt accrues interest (similar to financial debt), but the interest rate is different. As the example about a usps api demonstrated, that has a near 0% interest rate. Of course if you were ever to need to update it (maybe usps updated its api) you'd get hit. A bad domain model has a high interest rate. All the ideas are the same though, i'm just being persnickety :D
In addition, the term "debt" also applies to moral aspects, and not just financial ones. A builder may come back and improve something not because it's good for the employer or has financial benefits, but because of a belief that quality is good for the soul of the builder. That is also an aspect of technical debt.
Regarding the original article, it says tech companies are "successful because they got a product to market fast and at the right time". The "fast" is incorrect. Companies are successful if they are at the right time. If a company (like GO Corporation with pen-based computing) is fast but too early, they they are not successful. If a company (like Microsoft with Excel) is slow but eventually gets there, then they are successful.
"Fast" is therefore irrelevant in the face of "the right time." It can be that the right time is "as soon as possible", but that's not intrinsic to success.
Excel was successful because Lotus decided 123 had so much technical debt it would be a good idea to "upgrade" the old assembler code to C. This took at least a year longer than planned, and introduced a lot of bugs and incompatibilities.
Meanwhile Windows had arrived, and the planned Windows rewrite of 123 never happened as scheduled.
So it wasn't so much that technical debt killed 123. But poor management of technical debt certainly did.
Effectively the C rewrite was a complete waste of time, and Lotus should have aimed for a Windows release as soon as they could.
It's nice to be able to say this with hindsight, but it was doubtless much less obvious at the time, when Windows was barely considered a mediocre visual DOS shell, and certainly not a serious OS.
Bottom line - you have to be smart, prescient, and a little lucky to manage technical debt. It's not just about getting the code right - it's making decisions with educated guesses about where you may be a year or two from now.
And that's hard.
I also don't believe it's possible to pick the 'important' areas of code a priori and make only those areas free of technical debt. Doing good work is an attitude, a mind set, and something to practice - if you practice making poor quality code, it's going to affect you across your work, not just in the places that you do it.
I think that, by being professionals and putting aside the 'hack it out' mentality, we can improve not just the code that we're working on, but our mindset towards it. How many codebases have 'that scary part' that adds background anxiety to the people that work on it? Is that something you want to invite into your life on an ongoing basis?
Especially for things as important as integrations into external APIs - sure, they may not change much, but in my experience you really want to make sure they're done right when they're how you deliver your product to customers. Given how error-and-failure-prone many external APIs are, I expect a lot of work there just to robustly handle the other end of your integration being flaky.
I think that quality is something you can't 'bolt on' later. It has to be built in, top to bottom, throughout your organization, code base, and personal skills.
And you may pick the latter, but what if management wants to make a quick prototype and turn that into a product later? At that point you'll still be having the "technical debt" talk with your manager.
That said, I'm not implying that you should slow down, just do the best possible work at all times. Don't do a crappy job just because you're time-limited - we all only have 24 hours a day, and there's always more work than can reasonably fit in. The moment you start going "I am going to lower my standards right now", you've started a trend.
I think it's interesting that people are always saying "hire the best possible people you can", and then saying "but don't ask them to do the best possible work they can do".
And then there's solution number 10, which requires the programmer to invent a new programming language, and requires at least a couple of months to develop. "You hired the best possible person for this job, so let me do the best possible job".
I just bet most people here wished they could always deliver the best possible code.
A good programmer will try and improve code the code incrementally, and won't bother trying to sell anything.
I am sure that most product/project managers want feature as soon as they can think of it
1 week today to do it properly, a couple of hours a month from now to tweak it, a couple of more hours 6 months later to add a new tested feature,
vs,
1 hour today to hack it in, 3 hours on friday to rework it, 1/2 a day next week to debug a problem, and hour the next day to fix your fix ... plus the entire following day when that didn't work properly
Followed by 1 day to do a "proper rewrite" in a month, to add the tweak that should have been easy but breaks some of your assumptions plus 2 days to debug why something doesn't work properly anymore interacting with another subsystem.
Followed by someone else spending most of a week debugging a subtle bug you introduced by stepping on some state "nobody was using anyway"
Followed by 6months later when the new feature is needed someone looking at all this and throwing up their hands because they can't understand what it's doing. At which point they say... "screw it, I can re-write this in a couple of hours".
Lather, rinse, repeat.
Most of the time, the "fast way" is pretty reliably going to cost you more time. Just not more time today.
B) You can spend an hour and a half instead of an hour and include one test and make some incremental improvements as you go.
I agree with all of your points and just wanted to add one more thing that doesn't seem to be addressed much. The author mentioned technical debt didn't have an "interest rate", but it absolutely does. As soon as you start to add debt to a particular part of the code base, it must be paid back immediately the next time you or someone else has to work on that same section. This may genuinely not matter for a one-off script, but if it's code that will ever be worked on again it can start a snowball effect that causes the interest to compound and usually manifests itself as code that's buggy, unstable, and hard to work on.
Financial debt has an interest rate, but you take it out because you believe the return on the proceeds raised adds more value than the cost.
Technical debt is meant to (ideally) be the same. As long as it's outstanding and compounding, it should be because your return on business value will be lower if you "pay it back", then if you do not and spend time/resources elsewhere.
if your discount rate is 30%+ per annum, this makes for some counter-intuitive conclusions about when it's smart or not to "finance" with technical debt.
Allow the page to break if the user's browser does not support websockets If you know that 95% of your target audience uses modern browsers this seems like a reasonable compromise.
Don't write enough unit tests Maybe you're using a library that is difficult to mock. Maybe most of your logic is happening at the db layer. Eventually you may want to thoroughly test this piece of code, but it's a ton of work to mock a database in a meaningful way.
Write a monolithic service instead of a modular one While it's bad to let a single service grow too large, when you're starting out, it can be helpful to write it all out, and _then_ see how it can be broken apart. (It's often hard to foresee everything at the design stage.)
Finally, a lot of code ends up being never used. Nobody cares about well-factored well-tested code that isn't used. Sometimes a dirty version is the quickest way to tell if a project makes sense at all.
At this point, your design is already screwed and borderline-unmaintainable, so yeah, I guess unit tests won't help much. But when people talk about Doing it Right, usually they mean having a codebase that's loosely coupled enough that tests are relatively simple to write.
My opinion is that whether it's programming or general white-collar work, most cases of doing it "the right way" vs. "the dirty way" tend to be about investing the "certainty" of lost time immediately for the "promise" of savings.
For some things, the tradeoff is pretty decent (taking time to improve core knowledge or relevant skills that are related to goals).
But for a quickly growing company, how many situations are there where a "hack it out" fix that costs time in the long run is a net win in opportunity cost and hence long-term value?
If a shortcut that will cause you headaches in 9-18 months also allows you to add critical features at an accelerated pace for the next 6 months, is it: (1) inefficient because you net lose man-hours or (2) efficient (a) maximizes your near-term growth trajectory, (b) thereby increasing potential access to resources and hence long-term growth
To be clear, I do agree with your statements about shortcuts usually not saving time especially when you deal with teams/setting standards.
My point is more that while time saved/lost is easier to mentally calculate, it alone is not really what's driving value in many situations.
There's also the inherent problem of using qualitative terms that mean different things to different people.
But hey, that's part of the fun on the internet.
"If we don't get traction/product out the door now, we won't be able to raise our next round, so we have to get this product out the door yesterday to de-risk the whole company falling apart when we run out of money in a few months." Then later you raise your round and do the same thing for the next round. Eventually, of course, it will catch up with you and slow key progress. If you're unlucky that leads to a down round and you're all screwed.
I see that as being a larger risk than competitors moving faster in the sort term (them moving faster short term may mean you move faster long term). Higher burn rates mean you have a clearer deadline for fundraising, means you're racing the clock, means you're incentivized to take on technical debt. I'd love to have a look at what really tends to kill companies more often, but it's just anecdotes and speculation from me today.
I would point out though that in the financing-focused case you gave, although you have a definitive slow down from technical debt, it's still an open question whether or not it's actually a net slowdown vs the "status quo" scenario.
In practice of course, every individual company, with different teams and different contexts, has to find the right "cognitive bias" for themselves.
Everyone thinks they can pay after the release. If you cut those corners hard, though, you'll often regret it before your release. Cut harder still, and you won't even make it to the end-of-sprint demo before there's a knock at the door.
Wikipedia has a bunch of great links to more on the topic if you're unfamiliar with it: http://en.wikipedia.org/wiki/Product/market_fit
If you released it today, you would own the car market and you would be best pals with Elon.
If you released it in 50 years, people would think you are quaint for playing with that old-tech for a problem that been solved 10x over.
Some subsystems don't change much because they already do what is needed and aren't broken. Sure, the code could be improved, but it's not important enough to do (or at least, to do now).
Other subsystems don't change much because they are scary to change. If the system is brittle and poorly designed, engineers may have very good reason to be hesitant about changing it, and instead it is worked around.
So this latter case behaves very much like "technical debt" . This system costs you engineering time every time you even think about touching it. It's hard to reason about, it's hard to change, and over time these problems get worse if you introduce workarounds in other parts of the system, because now they are worse and harder to work with.
This is a very real cost.
Those workarounds that get created instead of changing the scary thing put me in mind of the "scar tissue" metaphor that SapphireSun proposed elsewhere in this thread.
I think that it often is best thought of as "real debt" in that it costs you time and attention, and (at least some of the time) that cost compounds over time. In other words, some of your engineering spend is just going to service this debt, not produce anything useful. That's a very real opportunity cost.
Speed follows quality.
If you're doing your planning and analysis right, you should hope to have identified this... so in your example, hopefully that subsystem went up in priority because it was identified as a place for gain.
The trouble is, I don’t know which half is which until it’s too late.
IMO a better context for this would be: real software engineering requires compromises. There are a number of competing factors that need to be juggled in order to be successful: time, money, complexity, quality, etc. The point should be that you need to balance these factors, not focus on one above all else.
I always enjoy Ward Cunningham's discussion of this: http://c2.com/cgi/wiki?WardExplainsDebtMetaphor (Ward first used the metaphor)
Specifically, he is NOT talking about 'dirty code' or just badly written code, that's taking shortcuts and I believe it is never a good idea in software. We're talking about building good code but doing so before we fully understand the problem, often with the aim of learning more about the problem, especially from user feedback.
What I take Ward's discussion to mean is that "technical debt" is accrued when writing code for a problem you do not YET completely understand (e.g. almost all 'first pass' code). The debt occurs when your code's existing understanding is static (because it is in source control), and your teams understanding grows. The difference is your debt. You can continue building on the code's not-quite-right understanding for the moment which is carrying the debt forward and paying the costs of fighting a not-quite-right model, but at some point it becomes worth while to re-factor based on what your team has learned about your problem, and thus pay off the debt.
But apart form that, absolutely. Technical debt in the article isn't the technical debt that was originally coined.
I still think the common meaning of technical debt is just as important though. Code that can't be taken forward, for whatever reason, costs its owner.
Instead of digging my heels in on not giving such PMs everything they want, I try to offer alternatives and put the choice on them: they can have this feature today and in 6 months we grind to a halt because we can't fit in anything new, or we slow down just half a beat and do it right the first time. "Up to you, buddy," I say.
It happens with jr. devs, too. They want to use a singleton to solve a problem, or they want to forgo using any sort of referential constraints in their schemas.
Of course, the PMs and jr. devs always choose the shortcuts. I help them both pick up the pieces afterwards, but I never work overtime for it. The point is to make it clear that nobody other than themselves is responsible for their own mistakes. They led us into the mess, they can lead us out.
After that, everyone seems to get along a lot better and everyone seems super keen on having discussions about the "right" way to do things and how to fit things into the current state of the project correctly.
Buy some daisies and chocolate for the PMs you work with... because here's how that would go over with the ones I've had:
PM: "Yes! Cut all corners and get it done today!" ...6 Months go by... PM: "I came up with a new idea for that! Today, please!" DEV: "But remember..." PM:"Today, thanks!"
So when you're told to cut corners today, then in 6 months you can't make magic happen to grant their every whim, you have to let it blow up. They want to be the leader. They are responsible for the failings of the team.
You have to play chicken with them. Because every time you capitulate, all they know is "I got what I wanted". They don't care how much you complained or how many hours off the clock you worked. And honestly, if you tell them it can't be done overnight, and then you do it overnight, you're a liar and they won't trust your judgement ever again. When you say, "can't do it", you have to not do it. Don't make magic happen.
Or just don't work at such places. I have thankfully been out of that bullshit for three years running now.
As you say, don't make magic happen. It doesn't matter how cool it would be if, or how you think you'll show them your ideas are better; when you say "it cannot be done" don't go and try and do it. Don't let the manager bully you and don't let them try and get another developer to tackle the impossible.
I've had that happen to me where I said it will take X amount of time to do it so let's not do it till next sprint. Then the manager went around my back when I was gone for a day and gave the task to another dev. The dev, who caused the mess, of course was able to fix it within an hour. Why did I say X amount of time? To cleanup and make sure the bug stays fixed.
When you work with other developers, actually work together. Stand up for them, don't make bad comments about them, and always stick together when you're asked to do the impossible.
While you didn't specify what your "X" was, it sounds like the manager (and the company) got a better outcome by going around you and it was far from being "asked to do the impossible".
On a long enough timeline, the better outcome turns into multiple bugs.
That's the everyday misconception. Nobody produces wealth just by typing at a keyboard. Products don't design and sell themselves and it's arrogant to pretend that product people and salespeople aren't a huge part of the value creation process in tech.
There are of course developers who are capable of identifying a market opportunity, conceptualizing a product to exploit the opportunity, building the actual product and getting it into market successfully.
But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
Okay you've got a point, it is fairly arrogant; on the other hand, any sort of automation or small script or product is valuable enough that we don't need to be subservient.
> But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
That's what I'm asking, why are talented developers allowing themselves to be subservient? That's why I'm wondering why technical managers all the way up to the CTO level are subservient to other departments when it's the product based on technology that determines the success of the company in the end?
And actually, your idea is optimistic; I wouldn't mind low six figures for 10-12 hours a day ;) maybe that's why we're so subservient, we look at our peers in other industries and they're making much less cash money and it's easy to see that we're lucky to be working in a nice air-conditioned office working with something we're typically passionate about.
Answer: trying to get a real product to market successfully requires a lot more than small scripts that automate processes.
It's not that automation and the like doesn't have value, but ask the average developer to get on the phone and sell something, or to have a meaningful conversation with a customer, and you might start to understand why other people in an organization (salespeople, product managers, etc.) have some sway too.
If you simply put a bunch of talented developers in a room, chances are you're not going to end up with a successful business.
Quite, yet this seems to be how many first-time startups operate.
There are a lot of terrible PMs out there, but there are also a lot of really good ones, and you learn to suss out which is which when you're interviewing for a new team. If you read Rands in Repose and start wishing your PM actually thought and cared about the things he talks about, maybe you need to find a new job.
>PM: "Yes! Cut all corners and get it done today!"
You: Can I get that in writing, please?
Generally I agree with you on the principle of using communication to help people learn from experience. I wrote about it here: https://medium.com/@brlewis/fighting-technical-debt-in-an-ag...
Even worse, if you're then not equally technical as a manager you will never know whether you've made a solid choice by aligning with or against the engineer's view, and even worse your engineers will quickly take on a "see, I told you so" stance should things work out not perfectly under an agile model.
Edit: I should add that I am an engineer myself.
This is a spot where I'm definitely in line with the Scrum position: If it's really causing problems, then the developers should have no problem making a reasonable business case for fixing them. If working on it's really causing that much pain, then the developers will figure out a way to quietly fix it in a way that doesn't (and needn't) vex the product owner. If neither of those is the case then you best let sleeping bears lie.
(I'm an engineer too.)
Contrarian anecdotes aside, the reason everyone knows the phrase "technical debt" is because SO MANY of us see projects wrecked by greedily demanding many features and changes as fast as possible in a continuous crunch mode, without considering long-term costs like defect rate or turnaround time on bugfix and feature implementations... or developer burnout.
It's better not to assume that developers are blathering when they ask for formal permission to carry out QA or maintenance tasks or upgrades, if they are the people who know what the work looks like on the ground level. If you are putting lots of time pressure on developers and implying that heads will roll if aggressive deadlines are missed, it doesn't make sense also to leave it up to those developers to magically find the time to fix lurking internal time-bombs.
This makes products worse, it makes developers' lives worse and developers are stuck with the blame for something they can't control. Maybe as a PM or whatever that suits you, to dump all the risk in the laps of your developers. But if you are yourself an engineer you should not make yourself part of this problem.
Similarly, living in continuous crunch mode is something I let myself get bamboozled into quite a bit in my younger days. I bake time to leave the camp cleaner than I found it into my effort estimates, and I know how not to let myself be pushed past my limits. If I worked for a place that couldn't respect that, I'd also be looking for a new job pronto.
I hope that you were being hyperbolic for rhetorical effect in your post, but if your work environment is really like the situations you were describing then I would also encourage you to be looking for a new job pronto. Life's too short to waste on that kind of stress.
It's totally possible to contain horrible sins behind a clean API. I've done horrible things, but those things don't leak all over the whole project.
On the other hand, i worked at the same place for a long time. I got to see the benefits and consequences of my mistakes and successes over years. I do think i burned out. It took me a long time to learn how to contain nuclear waste.
In the end, it's easier to just work someplace that doesn't put crazy demands on developers. There's a continuum of perfection <-> functionality. unfortunately for new developers, it takes a while to figure out where you lie on that line. Finding a place that's close to your preference is really best.
It's kind of like driving. Everyone slower is a Sunday driver, everyone faster is a lunatic with a death wish. I'm happy with homogeneous drivers - too far from the mark and i get nervous. I'd bet it's the same for pretty much everyone else as well.
Good engineers will recognize when it's "good enough". However, that requires biz folks that can provide useful feedback and actually get things in front of customers and get paid. Because, honestly, if we're like three extra features in, have no validation, and are cutting corners, you're damned right I'm going to put on the brakes and redirect the effort toward paying off technical debt.
Because, and this can't be overstated enough, whether or not the business fails in six months is not something we engineers can control--but I'll be damned if I'm going to come in and keep sinking my dwindling life force on a fucking technical lemon. If your biz folks can't establish trust in their abilities, then you might as well hone your craft.
EDIT:
Large amounts of technical debt are a sign of either rapid growth or bad management. If the company is growing, we can overlook the debt every time we, say, waste 30 minutes waiting on a build or fixing a weird nested CSS bug or some other damn fool thing. If the company isn't, then we lose respect for everyone above us in the technical leadership, because they let--by deliberate neglect!--the situation get so bad.
The more lines of code you have, the more stuff you have to maintain and understand.
Of course this is pretty stupid, but it's stupid in a way that sometimes leads to insight.
If your application consists of 1000 lines of code, you can rewrite it in an afternoon.
Also, cleverness can create its own kind of debt. Even as you believe you're working on reducing debt by creating some clever and sophisticated abstraction, you are actually just making it worse. I think this happens all the time.
That's why my basic, fundamental approach to software development is "throw it away and replace it with a simple shell script."
also another mistake I've seen made over and over again is to say stupidity==technical debt. If you're doing stupid things just to 'get things done' you're doomed in the long run. If you've thought about it and made the decision to cut a corner you should know why you've done it.
No, what _you_ need is more hastily shipped crapware to capture markets from your competitors so you can make more money and disappear. Hastily shipped crapware has enabled lots of new and interesting things. It also keeps 10x as many engineers busy serving it as would moderately better thought out, slightly later shipped software. Babysitting crapware isn't bad money for an engineer, but it's cosmically questionable as to whether it really advances us more than its opportunity cost. And hastily shipped crapware at scale is a real giant energy problem. Take a look at the size of a Facebook datacenter and tell me again how a metric fuckton of PHP is a good thing for the environment.
On the other hand this is what I miss most about school, or perhaps just doing things out of my own interest.
There was a time when I could appreciate the beauty of code and all I could worry about whether my software was elegant. Then I joined industry and everything changed.
I've definitely seen startups that have almost found a good idea, but can't tweak it, scale it, or make it stable, because they're overwhelmed by technical debt and even simple changes have become engineering death marches.
Letting your back-end code turn into a complicated, disgusting mess means that adding critical features may become a multi-week nightmare. Which is then followed by a series of additional multi-week nightmares for each successive feature.
Now, this isn't to say that obsessive code-polishing is a good idea, either. But startups require rapid iteration, and you can't iterate especially rapidly when your code is complicated, badly organized, and poorly tested.
Unfortunately some people don't realize that in many situations not making a decision is itself a decision, and so they don't always notice it when it happens. That can be anything from the ones you mentioned, like testing and tools, to authentication, auditing, localization, robust error handling, resource/memory leaks, or monolithic designs that prevent scaling.
Accepting large amounts of technical debt enables you to deliver faster if your initial assumptions (both about what you want and how to achieve it) are correct, but slows you down if any of those turn out to be wrong.
OTOH, it might offset that slowness in whole or in part that by speeding up your ability to determine that they are wrong.
I don't think this is what Linus was talking about...
And it wasn't just crappy web apps or throwaway MVP stuff. The first one was in medical systems, where the suits figured we had a 6 month lead on our closest competitor, the other was in consumer electronics (navigation devices), in the time that competition was still stiff in that sector.
In both cases it was decided that releasing something was more important, knowing full well that we had to pay the cost later. Of course not everyone agreed with it, since no one likes having to work with legacy crap, and everyone and their mother always wants to refactor all the things, but guess what: code quality is not the goal of your company, making money is.
> medical systems
Oh no.
Most software, especially middleware and EHRs, are the worst sort of enterprise garbage.
1/ Poor code =/= technical debt. Far too often, poor code masquerades as "technical debt"
2/ Good engineers do the right thing which is always a judgment call based on circumstance & culture.
3/ Take guidance from your best engineers, and find a way for the business to adapt. Much easier to change the sales deck or pricing today, much harder to overcome bad technical decisions. No one ever said "I never knew that sales deck or pricing would last this long" but many say this about software. Be super careful & deliberate.
4/ Build things that are simple (not easy), and respond to demand. Focus on movability & robustness, not "perfect" design.
Ergo, in your scenario even with 10 users, the application basically died under its own weight under nominal use.
With, of course, the realization that simple is hard and that it is almost always orthogonal to easy.