I like to phrase it as acknowledged technical debt. And I think about it like this: when you buy a house, you don't use cash. You borrow. Debt is good as long as you pay it off, and don't get smothered in accumulated interest. Similarly, when you're starting to prototype a complex system with limited time and people resources, it's better to put in working kludges than to attempt a "really good job" on every single component. It's like "perfect is the enemy of good", but crappier. More like "good is the enemy of meh-it'll-do".
The difference between a rough prototype and a minefield is that with the former, you record technical debt in the ledger You regularly pay it off, and watch out for interest (kludges on top of kludges, bad architectural decisions because of forced workarounds, or actual financial cost due to inefficiencies).
There's a very interesting parallel to your technical debt, and an implicit choice you're not making explicit: One can instead opt to accrue feature debt. Instead of "finishing" the complex system with meh-it'll-do quality, maybe one should focus on finishing the quality even if that only results in a meh-it'll-do system.
Sure, the system will be underfeatured to begin with. But it'll be high-performing, rock-solid reliable, easy to work with, quick to iterate on. So by time, you can start paying off the feature debt and add more and more things to create the complex system you're going for in the end.
I'd even argue that paying off feature debt taken out on a simple system is much cheaper than trying to pay off technical debt on a complicated system.
(This is the premise of "worse-is-better" software development.)
I think that it also depends on funding, for example:
- you have great seed funding and decent runway, you can afford not to have all the required features to make the product viable for longer z and can do a better job.
- your potential investors want to see AI being used in your stack before they close on a funding round, so you cludge a basic autoML model or basic tensorflow network and put it somewhere it won't break stuff and scream "we're using AI". Then actually do it when it makes sense properly later.
- you need a massive client to show board you're capable of revenue, they demand a feature in order to close, so you build it ASAP.
I think what I'm trying to say is the lean startup art is to carefully leverage technical debt and feature debt in the appropriate times, and have a culture of cleaning little bits as you work o other features.
Having a good eye for architecture and clean code is the secret sauce. As long as there are people reviewing code who understand this, you'll hopefully end up with isolated technical debt, which can be kept under control and won't sink your startup when you expand and accelerate growth.
This is a really great example that I wish more engineers understood. There are an enormous amount of external factors that play into why that software project must have features a, b, and c and must be done by next friday. Good code costs time and money which are luxuries you don't always have.
> You borrow. Debt is good as long as you pay it off
I think this is where software and other sectors come together to demonstrate why something that seems good ends up evil.
What is to stop someone from borrowing slightly further beyond their means than you? What does that mean for the market? Housing is unaffordable, turning us into slaves and also in a wired crisis, software is either free, vaporware or impossible to discover under all the vapourware and the website needed to compete with vaporware..
Without standards, ratings, limits, strong unions and trade associations, everything becomes a market for lemons since someone will take the risk of faking it all when the penalties are only partial revocation of the profits.
Long I've been pondering how our notions of technical debt (managing risk, complexity) might apply to government, regulations, bureaucracy, etc.
There's so many analogs. eg René Descartes said our cities would be simple and orderly if designed by a single mind.
Now your comment makes me realize I ignored the flip. What are our analogs (in software, design) to regulatory capture, rent seeking, and other pathologies.
> What is to stop someone from borrowing slightly further beyond their means than you?
There's always a spectrum of risk-taking. Some people will borrow every last dollar their bank will let them. Others will not. I was shocked at the size of the loan that the bank pre-approved me for back in 2007 (fairly crazy times in US housing market). I borrowed substantially less than that because I wasn't comfortable taking out a loan at that high of a debt-to-income ratio. On the flip-side, had I taken out a maximum-sized loan, I'd have made more paper gains in housing in the last 13 years, so there is often reward for risk-taking.
If it was just based on the title and the term "half-assed" then I would agree about technical debt. But having read the article, I'm not sure that is quite what he was saying.
He is talking about prioritization of effort and how thorough the solutions are. If you properly judge the importance of things or the level of sophistication actually needed to solve the problem adequately (but not necessarily "fully"), you may not ever need to revisit that part. Especially since requirements evolve. But also because bottlenecks (in terms of performance or requirements churn or bugs etc.) are often clustered in particular areas. So even though there is a more complicated solution that could be more elegant or flexible or performant, it may actually never make sense to implement because the relative importance just never increases.
What he seems to be more or less talking about is similar to the 80/20 Rule (ala Pareto Principle).
I think a less negative term could have been less confusing.
The good thing with technical debt is inflation in some way. The code will likely get rewritten every few years with new features in mind, or will change significantly anyway so it's almost as if it's inflation. If technical debt allows you to get great gains in short time through product (just like any useful debt), technical debt is warranted.
I spent 6.5 years in google and seen an sdk get rewritten 2 times, 3rd was about to happen after I left but product director asked me if it was warranted, I said it sounds more like all these problems you are mentioning are monitoring/detection/testing problems and rewriting it won't make sense. The code did have technical debt, but it didn't matter
Linked to perfectionism, I think the Fear of Missing Out (FOMO) plays an deterring role in half-arsing things. By reading a book from cover to cover, you have access to all the information available. By skipping chapters, you risk missing important information that (you think) could change your life.
To counter this fear, it's a good exercise to express your expectations about the book and to judge what it can bring to you: is it really life-changing ? Or would you just want it to be ?
In my opinion, it demands greater reading/evaluation skills to be able to skip pages to extract the most data, and a good resistance to FOMO.
I can't really get on board - for most situations, risk of 'broken window' effect is strong unless your team all operate on the same page with a common understanding of what tradeoffs are acceptable.
Perhaps reasonable in projects/tasks that you have full ownership of and intend to continue to do so for the foreseeable future. In general though, I think there's good reason to strongly prefer the 'if a thing is worth doing, it's worth doing well' side of the spectrum.
What if you don't know whether a thing is worth doing, and the cheapest way to find out is to half-arse it and then with the additional evidence evaluate whether it'd be worth doing well?
On reflection my gripe with the post is really about vocabulary.
Half-arsing carries negative connotations - to me, at least, it is the 'bad' form of pragmatism, and as such in my book it is never defensible.
IMO we have other words to describe the scenario where an expert consciously makes a 'good' or reasonable trade-off to complete a task in a smaller window of time, and should use those instead.
I would go even further and say that if the result is good enough for the requirements, it's not half-arsing, even if the result could be made better with more time and effort. In other words an outcome will be judged based on expectations and consequences, and not compared to "the best it could be" (unless someone expects it to be the best it could be).
This highlights the importance of naming and terminology: if I say I will "prepare a report" people will expect one thing versus "throw together a few visualizations". Of course if it's the CEO it's inappropriate to just throw together a few visualizations. This shifts the issue from execution to alignment of expectations, which for me has the benefit of not feeling I'm doing a half-arsed job.
> I would go even further and say that if the result is good enough for the requirements, it's not half-arsing, even if the result could be made better with more time and effort.
Ooh, both I and professor Deming would disagree with you there. It is not enough to just meet requirements.
The thing you're building is going to become a component in a very complex system, and all the nuances that matter there are impossible to capture in a specification.
If it just conforms to specification, someone will need to file it down to be able to slot it into the right place, and it'll still only go with excess friction and thorny edges.
To create something that actually works and makes users happy, one has to go beyond the specification, understand and optimise for the system it is going into.
The one thing I remember from my ill-fated vocational training as a toolmaker (industrial tools, not software tools) is that one sentence our master hammered into our heads:
"Not as precise as possible, only as precise as necessary."
This wisdom from the physical world also came in handy in software development where I eventually ended up ;)
One situation that lends itself quite well to half-arsing is working to maximise the performance of a system that consists of a number of components, where the performance of the system is bottlenecked by the weakest performing component. You can get pretty far by (i) identifying what the bottleneck of the whole system is, (ii) figure out the cheapest/easiest move to eliminate the bottleneck, and then (iii) iterating. There's little value in full-arsing the performance of a ccomponent if it is not the system bottleneck and unlikely to become the bottleneck any time soon.
E.g. suppose we've got a system with three components A, B and C, so that perf(system) := min { perf(A) , perf(B) , perf(C) } . Currently perf(A) is 0.3, perf(B) is 0.5 and perf(C) is 0.4, so the performance of the system is 0.3 since it is bottlenecked by A. You have an elegant idea to reengineer C completely which should lift perf(C) to the region of 0.9 -- 0.95 but it will take a few months to prototype and sort out the details. You've also got a lamentable idea for a hack to increase perf(A) from 0.3 to 0.5 -- it's a dead end and the hack cannot be extended in future, but can be cobbled together quickly. What's your move?
This is why, before setting out to improve performance, you have to define a performance metric, goal, and do some profiling. Otherwise people end up flailing around changing things based on what they've heard has better performance.
Reading books has to be one of the most overvalued activities known to man.
The problem is that book authors are not incentivized to deliver "the gist". If all the insights of a book, properly condensed, amount to no more than a pamphlet, people will give it a bad review.
If instead a book has hundreds of pages, there mere act of giving it a "proper" critique becomes costly. Judging a book that you haven't read is frowned upon. Those who could be bothered to work their way through it, on the other hand, are less likely to be critical.
Meh. I think it's self-evident that sometimes you have no choice but to half-arse things, but my experience is that it's better to full-arse one thing at the expense of not being arsed at all with something else. A hidden advantage is that once full-arsed the first time, the second time a task is done, it doesn't take much more than half-arse effort.
I wouldn't call that half-arsing, more dumb guessing.
You have no insight into the internal model, you cannot check or improve it. A single training mistake renders the whole model afoul.
Most of AI as I learned it in school needs internal verfification steps, you print out the inferred rules, so you can learn from them. The black-art model really is just black art, but how should e.g. a self-driving car be allowed to drive on the streets, when you cannot explain the most basic rules, in case this or that.
It turned out that some things were just too hard to really understand. The weird thing is when you realize that actually we never understood how to do these things. People doing them are as competent as they need to be AND good storytellers/presenters about how they "know" what they're doing. Trying to tell a computer how to unambiguously do these tasks only revealed that nobody actually knows how to do them.
This is also why a lot of business logic software is a lot more crappy than it seems like it ought to be. For sure some of it is written by people who don't know how to design a good software product. But there's also a lot of it that can't be made well because the people asking for it to be made don't understand (on like a logical or specification level) what their job actually is.
If machine learning is a half-arsing industry, then it's only there because every industry that machine learning is attempting to augment is also an industry that is half-arsing.
EDIT:
And I think it's worth mentioning that I very well may owe my life to one of these "half-arsers" that I'm talking about. He found and removed a precancerous mole. The tests came back showing pre-basal cells (iirc). I asked a lot of questions about how he was able to identify the mole as problematic (and he saw it right off the bat even though my general practitioner and myself ignored it) and he really wasn't able to give a satisfying answer that didn't also identify other moles he said were (and have been) safe. He just knew it was bad and he was right.
For some reason I kind of see it as a different application of the 80/20 rule.
I can say with certainty, once you have a second child, you get really good at half-arsing. There just is not enough time to do everything you want to do.
Because it worked for me in the earlier stages of the pandemic (at home with kids during the day, call in to meetings when possible, try to catch up at night) - I've posed this suggestion to the teachers in my family who are freaking out about the new school year:
I know you're saying this with a bit of jest in it, but it's a great point and there's a lot of value to asking "have you considered lowering your expectations for how this is going to go, given the situation at hand?"
I meant to say this much earlier, but this is the best place to leave his comment.
It comes down to tolerancing and doing the right amount of anything to get a goal and not much more. And even then, sometimes a 10% effort job is good enough. Systems should be efficient and degrade gracefully. Doing a shitty job is part of that resilience.
Too bad there's no good examples in the blog post. With regards to the color of the bike shed, if any color is good enough (even though there might be some perfect color that is very visible yet also soothing and pretty), then picking a color at random is not doing a bad job, it's doing an adequate job.
Perhaps a better example would be a restaurant that half-asses their website because most customers find them on Facebook anyway. In that scenario, visitors to the website might think "ugh, this is useless crap" – ugly, hard to navigate, doesn't have up-to-date information – but it doesn't hurt the business enough for it to warrant paying attention to.
I think half-arsing is fine if you're able to make a reasonable judgement of what you can and can't half-arse. It's certainly better than relentless perfectionism since this tends to mean you leave a lot of things undone - even things you should have done.
Still, and this is where it's going to be hard to avoid sounding like a snob, the implicit assumption here is that everybody is qualified to decide what they can and can't half-arse. Manifestly this isn't so:
- If you half-arse redecorating your bathroom by painting it with plain old matt emulsion paint rather than dedicated bathroom paint, and refloor it with laminate flooring that isn't water resistent, it's not going to last: the paint will peel and the laminate will, well, delaminate
- If you half-arse building a concrete block wall by not putting in footings it will one day blow over in a strong wind
- If you half-arse electrical wiring you create a serious hazard and risk injury both to yourself and others, as well as the possibility you may burn your house down
The problem is that plenty of people half-arse all of the above along with many more things that are immensely stupid to half-arse.
Hence schools drum into people that it's important to do things "properly" because the world is rammed full of people who can't reliably decide when it makes sense (and is safe) to cut corners.
40 comments
[ 3.4 ms ] story [ 74.5 ms ] threadThe difference between a rough prototype and a minefield is that with the former, you record technical debt in the ledger You regularly pay it off, and watch out for interest (kludges on top of kludges, bad architectural decisions because of forced workarounds, or actual financial cost due to inefficiencies).
Sure, the system will be underfeatured to begin with. But it'll be high-performing, rock-solid reliable, easy to work with, quick to iterate on. So by time, you can start paying off the feature debt and add more and more things to create the complex system you're going for in the end.
I'd even argue that paying off feature debt taken out on a simple system is much cheaper than trying to pay off technical debt on a complicated system.
(This is the premise of "worse-is-better" software development.)
- you have great seed funding and decent runway, you can afford not to have all the required features to make the product viable for longer z and can do a better job.
- your potential investors want to see AI being used in your stack before they close on a funding round, so you cludge a basic autoML model or basic tensorflow network and put it somewhere it won't break stuff and scream "we're using AI". Then actually do it when it makes sense properly later.
- you need a massive client to show board you're capable of revenue, they demand a feature in order to close, so you build it ASAP.
I think what I'm trying to say is the lean startup art is to carefully leverage technical debt and feature debt in the appropriate times, and have a culture of cleaning little bits as you work o other features.
Having a good eye for architecture and clean code is the secret sauce. As long as there are people reviewing code who understand this, you'll hopefully end up with isolated technical debt, which can be kept under control and won't sink your startup when you expand and accelerate growth.
I think this is where software and other sectors come together to demonstrate why something that seems good ends up evil.
What is to stop someone from borrowing slightly further beyond their means than you? What does that mean for the market? Housing is unaffordable, turning us into slaves and also in a wired crisis, software is either free, vaporware or impossible to discover under all the vapourware and the website needed to compete with vaporware..
Without standards, ratings, limits, strong unions and trade associations, everything becomes a market for lemons since someone will take the risk of faking it all when the penalties are only partial revocation of the profits.
In Europe, banking laws.
"everything becomes a market for lemons..."
Long I've been pondering how our notions of technical debt (managing risk, complexity) might apply to government, regulations, bureaucracy, etc.
There's so many analogs. eg René Descartes said our cities would be simple and orderly if designed by a single mind.
Now your comment makes me realize I ignored the flip. What are our analogs (in software, design) to regulatory capture, rent seeking, and other pathologies.
Thank you.
Evolution over time, successfully addressing challenges, accumulating wisdom, avoiding classic blunders, works pretty well.
There's always a spectrum of risk-taking. Some people will borrow every last dollar their bank will let them. Others will not. I was shocked at the size of the loan that the bank pre-approved me for back in 2007 (fairly crazy times in US housing market). I borrowed substantially less than that because I wasn't comfortable taking out a loan at that high of a debt-to-income ratio. On the flip-side, had I taken out a maximum-sized loan, I'd have made more paper gains in housing in the last 13 years, so there is often reward for risk-taking.
He is talking about prioritization of effort and how thorough the solutions are. If you properly judge the importance of things or the level of sophistication actually needed to solve the problem adequately (but not necessarily "fully"), you may not ever need to revisit that part. Especially since requirements evolve. But also because bottlenecks (in terms of performance or requirements churn or bugs etc.) are often clustered in particular areas. So even though there is a more complicated solution that could be more elegant or flexible or performant, it may actually never make sense to implement because the relative importance just never increases.
What he seems to be more or less talking about is similar to the 80/20 Rule (ala Pareto Principle).
I think a less negative term could have been less confusing.
I spent 6.5 years in google and seen an sdk get rewritten 2 times, 3rd was about to happen after I left but product director asked me if it was warranted, I said it sounds more like all these problems you are mentioning are monitoring/detection/testing problems and rewriting it won't make sense. The code did have technical debt, but it didn't matter
To counter this fear, it's a good exercise to express your expectations about the book and to judge what it can bring to you: is it really life-changing ? Or would you just want it to be ?
In my opinion, it demands greater reading/evaluation skills to be able to skip pages to extract the most data, and a good resistance to FOMO.
Perhaps reasonable in projects/tasks that you have full ownership of and intend to continue to do so for the foreseeable future. In general though, I think there's good reason to strongly prefer the 'if a thing is worth doing, it's worth doing well' side of the spectrum.
Half-arsing carries negative connotations - to me, at least, it is the 'bad' form of pragmatism, and as such in my book it is never defensible.
IMO we have other words to describe the scenario where an expert consciously makes a 'good' or reasonable trade-off to complete a task in a smaller window of time, and should use those instead.
This highlights the importance of naming and terminology: if I say I will "prepare a report" people will expect one thing versus "throw together a few visualizations". Of course if it's the CEO it's inappropriate to just throw together a few visualizations. This shifts the issue from execution to alignment of expectations, which for me has the benefit of not feeling I'm doing a half-arsed job.
Ooh, both I and professor Deming would disagree with you there. It is not enough to just meet requirements.
The thing you're building is going to become a component in a very complex system, and all the nuances that matter there are impossible to capture in a specification.
If it just conforms to specification, someone will need to file it down to be able to slot it into the right place, and it'll still only go with excess friction and thorny edges.
To create something that actually works and makes users happy, one has to go beyond the specification, understand and optimise for the system it is going into.
"Not as precise as possible, only as precise as necessary."
This wisdom from the physical world also came in handy in software development where I eventually ended up ;)
E.g. suppose we've got a system with three components A, B and C, so that perf(system) := min { perf(A) , perf(B) , perf(C) } . Currently perf(A) is 0.3, perf(B) is 0.5 and perf(C) is 0.4, so the performance of the system is 0.3 since it is bottlenecked by A. You have an elegant idea to reengineer C completely which should lift perf(C) to the region of 0.9 -- 0.95 but it will take a few months to prototype and sort out the details. You've also got a lamentable idea for a hack to increase perf(A) from 0.3 to 0.5 -- it's a dead end and the hack cannot be extended in future, but can be cobbled together quickly. What's your move?
https://en.wikipedia.org/wiki/Theory_of_constraints
This is why, before setting out to improve performance, you have to define a performance metric, goal, and do some profiling. Otherwise people end up flailing around changing things based on what they've heard has better performance.
The problem is that book authors are not incentivized to deliver "the gist". If all the insights of a book, properly condensed, amount to no more than a pamphlet, people will give it a bad review.
If instead a book has hundreds of pages, there mere act of giving it a "proper" critique becomes costly. Judging a book that you haven't read is frowned upon. Those who could be bothered to work their way through it, on the other hand, are less likely to be critical.
Instead of understanding the system you are trying to control, you train the machine with some data and hope for the best.
With piloting 2 ton vehicles and trading trillions on the stock exchange.
What could possibly go wrong?
You have no insight into the internal model, you cannot check or improve it. A single training mistake renders the whole model afoul.
Most of AI as I learned it in school needs internal verfification steps, you print out the inferred rules, so you can learn from them. The black-art model really is just black art, but how should e.g. a self-driving car be allowed to drive on the streets, when you cannot explain the most basic rules, in case this or that.
This is also why a lot of business logic software is a lot more crappy than it seems like it ought to be. For sure some of it is written by people who don't know how to design a good software product. But there's also a lot of it that can't be made well because the people asking for it to be made don't understand (on like a logical or specification level) what their job actually is.
If machine learning is a half-arsing industry, then it's only there because every industry that machine learning is attempting to augment is also an industry that is half-arsing.
EDIT: And I think it's worth mentioning that I very well may owe my life to one of these "half-arsers" that I'm talking about. He found and removed a precancerous mole. The tests came back showing pre-basal cells (iirc). I asked a lot of questions about how he was able to identify the mole as problematic (and he saw it right off the bat even though my general practitioner and myself ignored it) and he really wasn't able to give a satisfying answer that didn't also identify other moles he said were (and have been) safe. He just knew it was bad and he was right.
- G.K. Chesterton, What's Wrong with the World, 1910
I can say with certainty, once you have a second child, you get really good at half-arsing. There just is not enough time to do everything you want to do.
– Kurt Vonnegut
"Have you considered just doing a bad job?"
It comes down to tolerancing and doing the right amount of anything to get a goal and not much more. And even then, sometimes a 10% effort job is good enough. Systems should be efficient and degrade gracefully. Doing a shitty job is part of that resilience.
Perhaps a better example would be a restaurant that half-asses their website because most customers find them on Facebook anyway. In that scenario, visitors to the website might think "ugh, this is useless crap" – ugly, hard to navigate, doesn't have up-to-date information – but it doesn't hurt the business enough for it to warrant paying attention to.
[0] https://en.wikipedia.org/wiki/Worse_is_better
Still, and this is where it's going to be hard to avoid sounding like a snob, the implicit assumption here is that everybody is qualified to decide what they can and can't half-arse. Manifestly this isn't so:
- If you half-arse redecorating your bathroom by painting it with plain old matt emulsion paint rather than dedicated bathroom paint, and refloor it with laminate flooring that isn't water resistent, it's not going to last: the paint will peel and the laminate will, well, delaminate
- If you half-arse building a concrete block wall by not putting in footings it will one day blow over in a strong wind
- If you half-arse electrical wiring you create a serious hazard and risk injury both to yourself and others, as well as the possibility you may burn your house down
The problem is that plenty of people half-arse all of the above along with many more things that are immensely stupid to half-arse.
Hence schools drum into people that it's important to do things "properly" because the world is rammed full of people who can't reliably decide when it makes sense (and is safe) to cut corners.
> If your fidelity to perfectionism is too high, you never do anything.