Ask HN: How do you overcome decision fatigue in software development?
Name any kind of project in nearly any domain, and you can choose from a dizzying array of possible language choices and tech-stacks. The modern state of software development in nearly any mature ecosystem, and many even that aren't so mature, allows for building just about anything on any platform. Want to build a web app, but don't like javascript? There are so many options in languages that compile to JS, as well as languages with front-end WASM frameworks. Want to write a websocket server? Nearly any language can do that now too. Want to store your data? Phew don't get me started on the infinite ways to do that one, with or without a database, with or without a server... it just seems overwhelming, in part because software development is very time-consuming. You don't want to spend all the time on the inferior choice or the choice that will bite you later and trigger a rewrite and decision fatigue all over again. Or so the psychology seems to go. How do you deal with it?
122 comments
[ 3.6 ms ] story [ 192 ms ] threadThere's time and place for using cutting edge solutions. But if that time is 'always' and that place is 'everywhere', you might be doing it wrong.
Keep things simple.
[0] https://mcfunley.com/choose-boring-technology
But I want to share an alternative to this because it helped me greatly in the past:
Sometimes this kind of decision fatigue is actually clouding something more profound and you might be on to something valuable, some kind of truth that is hidden away and still needs to come out.
Your decisions get blocked (from your subconciousness?), because you know this in some way and you cannot quite grasp why that is yet.
The hard part is to decide whether this is the case at all. I don't have a silver bullet for this, but one thing that helps is to try to be optimistically self-critical.
If you determine that this is actually the case: You have found some deeper idea/connection that you don't understand enough yet. Then you want to explore the idea by writing prototypes and text to figure out whether there is some pragmatic core that you can extract.
Note that sometimes these ideas are positives or rather solutions. But often they are problem statements or criticisms. The latter needs more exploration, often with different technologies.
This can easily stand on its own as a general reason yes. But I meant specifically in combination of having some "rough diamond" (please excuse the pathos) that is nagging you in the back of your mind and blocking decisions because you are not pursuing to find and polish it.
Often the pragmatic or "boring" approach is just right. But sometimes it isn't because of a reason that is hard to articulate and it just doesn't leave you alone. I think this is the nature of the creative process. Being brave and exploring it would be the "right way" in this case to figure out whether the idea is feasible or sound, (which it often isn't, but you can't help it and figuring this out is valuable in of itself).
For situations where there is not a clear popular or technical standout, I timebox the research time. After x (days|hours) I need to have a decision spec'd out. That way if I spin my wheels too long, then at some point I hit a deadline and my time is up!
A clarification here that I think is worthwhile:
Pick things that are "popular" as in widely used, not things that are "popular" as in "trendy" or "in vogue".
If the system is all completely new to you, then pick a very small project (or wait until you have a lot of time on your hands and failure is an option).
You should look a new tech as something to be experimented with and tried out in a casual sense. New JS Framework? I'm going to try and re-create this thing that I know how to do in the Old JS Framework and compare. New database? Make a simple app to explore it and flesh it out.
If you don't know enough about the new product to compare it to what you know, you shouldn't be considering going with something new. New stuff comes out all the time, but that doesn't mean it's good or worth looking into.
For experimentation and prototyping I tend to check out new things, those might end up in the 'boring' stack once I'm familiar with them.
For me it's a personal stack of tools that work for me, and I try not to be too influenced by what's the next new thing on sites like StackOverflow or Hackernews, I use what works for me and what makes my clients happy.
In seriousness, the phrase "boring technology" is, to me, a reference to the talk "Choose Boring Technology" [0], which has been discussed on HN numerous times [1] (deservedly, because it's great).
The author doesn't explicitly say what boring technology is, but uses examples you'd generally expect: postgres, php, python, memcached, mysql, apache web server.
Basically, products that have been doing relatively complicated things (interpreted code, rdbms, key-value stores) for a very long time.
Explicitly, he says that "boring technology" is technology that's defined by its very low maintenance cost. He claims that maintenance costs are mostly driven by uncertainties around failure conditions and design. These kinds of uncertainties are especially present in newer stacks, where the opportunities for footguns and unspecified failure conditions abound, because people are still working them out.
He implicitly references Redis as a fancy new technology, but a lot of people today would probably consider Redis "boring" (unless you're trying to use it as your RDBMS).
[0]: http://boringtechnology.club/
[1]: recently, here: https://news.ycombinator.com/item?id=20323246
99% of the developers here aren't even close to needing to think in terms of that scale, beyond simply not doing dumb things in your code that can happen no matter what tools you use. Hell, there are whole companies that don't need to do these things. A few companies I've worked for or contracted with were bogged down, IMO, because someone couldn't settle for "adequate", making the company suffer because their architecture is so hard to figure out.
I like using "boring" tech(in particular Ember.js, Node.js, Ruby, CouchDB and PostgresSQL/SQLite) because if something gets the job done in a reasonable amount of time, most of which I think comes down to having a competency with a set of tools, then I really don't care very much for drinking from the firehose of frameworks and libraries.
Letting go of ego is tough but, when you do, you're freed from the need to be "like Twitter" or "like Uber" and instead just build things using tools you are genuinely interested in or work for you. It's one thing if you really have the desire to be the kind of wizard to work at Google(and not be relegated to writing shell scripts all day), but if that isn't for you that's fine because the vast majority of jobs out there don't need an archmage.
What most companies need is someone who is going to KISS and not eff things up. If you can do that, you'll have a hard time getting fired even if you sometimes slack off.
There really is no reason to use plain JS for anything!
unless you don't want to compile your code. I can spin up little projects that don't need a build step just fine.
I have a relatively neutral opinion of Typescript, but oh boy does that quote ring true. TS can be a very nice safety net when refactoring (just fix all the type errors, and the app should mostly work as it did before), but you can also very well paint yourself in a corner with similar-but-slightly-different types all over the place that'd just work if it was in JS.
Sometimes newer technologies get hyped, but interestingly the most popular ones like React and Rust are also quite boring in many senses (React introduced a new model of programming web views, but stuck to plain JavaScript concepts, didn't try to reinvent the wheel for every component (e.g. like Angular having it'a own special http library) and has been incredibly reliable with few breaking changes from day 1. Rust introduced the idea of a borrow checker, but otherwise intentional borrows most of it'a ideas from other langauges and doesn't try to invent new ways of doing things, and again is incredobly stable and reliable from an implementation point of view)
https://www.kabdebon.com/cto/2020/06/20/making-boring-techni...
My two cents, if you want to build a business or a project for a company: go with battle-tested, mature technologies, always. Take away the risk of immature technology getting in the way of the only mission: delivering value to your customer
Now for a personal project... go crazy, have fun and try what's new out there.
~~~
https://en.m.wikipedia.org/wiki/Model-driven_engineering
~~~
https://music.youtube.com/watch?v=YMl7TEnpB7M&feature=share
Once the task is checked off, I can mentally free myself from the burden of thinking about the other options. If the decision turns out to cause unforseen problems, I can revisit my notes on why I made the decision and reevaluate with the new information.
* There are no solutions, just tradeoffs. Don't think there's a perfect tech stack.
* Done is better than perfect. Velocity matters. This generally means pick something you know if delivery is important.
* Boring tech tends to be reliable. Things like PHP, Rails, Django, etc. These tools have been around a long time and ironed out the kinks. Lots of documentation and tooling to make your life easier. Odds are very low your new fancy idea has any requirements that boring tech can't deliver on. This counts for every layer of the stack (frontend, DB, OS, etc).
I think the most important thing to keep in mind is you'll never have a perfect solution and that's ok! Don't get fooled by all the hype from new technologies that try to make you feel inferior for using other tech.
Wow I really love this perspective.
The last project I worked on still used JSF for the UI, because they guy who created the core of it knew and loved JSF. Then he retired a couple years back, and it’s been a real struggle to find anyone able and willing to do JSF development.
I completely agree with your point, but the absence of a "perfect" solution doesn't mean there isn't a global maximum. I think people already understand that too -- how many "what's the best X?" searches do you think people perform compared to "what's the perfect X?"?
However, I would also assert that for many somewhat mature technologies the value of determining the global maximum for your particular needs does not outweigh the effort required to do determine what it is - in general, a reasonable local maximum will be almost as good as the global maximum; you want satisficing instead of optimizing.
Especially in early stages of a company, where I don't yet know what the product that sticks will look like, I'd rather have a Postgres that I can pivot on in pretty much any direction and I'm hardly ever the first one to run into a problem, than a hyper specialized novel database that might have been strictly speaking the famous "better tool for the job" when I started.
Where devs make their error is, if they're junior, likely missing the technical tradeoffs (yes, other tech may be better for X, but it's also worse for Y), and if they're senior, likely missing the organizational ones (i.e., do other teams want to learn it, does devops/QA/etc have to do anything, what's the ramp up time, will the proposed productivity gains offset that, etc)
There are some types of tools (such as React, Rust, or Kubernetes) that are new and fancy, and which may often be unneeded vs a boring tech, but are also actually solving a different problem than their competition might be. Using Rust is genuinely not the same category of work as using C++, for example, and writing React is very different than writing JQuery. That doesn't mean Rust or React is always the right solution, but it's worth considering if a "fancy" tool actually does help you work on the problem you are actually interested in.
Of course, tech that actually makes a big step like that is pretty rare and still has costs. There will almost always be some costs to go with benefits. But matching your solution to your problem well can go a long, long way.
Wondering what would be the hottest frontend framework if React is boring?
I don’t have enough experience with it to say (though I do like to see it leveraging compiler tech to optimize rendering flows), but I think that is the question to ask. If it’s basically the same model as React, we really don’t lose much by using React, even if Svelte is nicer. If it’s a new model, it may be worth fighting.
Django will break backwards compatibility with every new minor version. You'll either spend a lot of time porting your code base, or stick with an old and vulnerable version.
At the end of the day however, you just need to pick something and roll with it. If it turns out not to be the best choice, you'll have at least learned something and the next time you have to choose you'll be better informed.
Don't stress out about it too much. It's a natural part of software development. When you are coding (after selecting all your tools), you are _still making tons of choices_. It's just in a different context; how will you structure your code? What coding paradigm will you apply? What algorithms will you use to solve your problems? You learned to cope with this stress too. It just takes time and experience.
https://github.com/andrewt3000/web_development#web-developme...
A harder choice is when you know the tech you known doesn't quite fit. Do you go with what you know, or do you try something new?
Concrete example I'm currently considering: If I want to make a web pages that does some background work, should I use Django and Celery like last time, and spend months tweaking Celery settings I don't understand and still feel like the system's kinda clunky (like last time). Or do I use Elixir where there's little difference between handling a HTTP request and running a background task, and where I can just spin up a few thousand threads to monitor background tasks?
TLDR: What do you do when "what you know best" isn't the right choice?
Unless I'm misunderstanding what you mean by 'background work'.
For example, dramatiq and python-rq are simpler to configure and to work with than the helicopter-level-of-complexity that is Celery.
* Stick with what you know. I personally know Python best, so if the solution can be achieved in Python, I go with that.
* Avoid fads. I don't use a language until at least two years after I've first heard about it.
* If you've gotten this far in the list, you'll have to learn a new language or brush up on a weak one, so look for one with a strong community and a lot of libraries.
* Old usually means stable. If it's an old language but still in active development, they've probably found most of the worst edge cases by now. Also there is a lot of best practices.
* Everything is a tradeoff. Nothing is the perfect solution, so know where your solution might be deficient (or learn).
* Doing cool tech is cool, but make sure you're solving the business problem (or personal problem) you set out to solve.
If you have a big work/production project, go with what you know, whatever that is. When there's a deadline or someone will lose money with a delay, it's irresponsible to use that project to add another line to your resume or play around with $NEW_TECH.
If you want to learn a new technology, do it on a project where you can afford to throw your hands up at some point and scrap the whole thing, because you very well may end up in that position.
Think of it as having two goals for the project: learning $NEW_TECH, and writing a program to do $TASK. Don't use $NEW_TECH unless the goal of learning $NEW_TECH is at least as important (or valuable) as the goal of writing a program to do $TASK.
This implies that you want to experiment with smaller programs. But when you do, recognize that you may not be seeing all the problems and limitations of $NEW_TECH.
I also just haven't gotten around to it yet.
If you don't know any tools, pick one and learn it well.
The inferior choices you need to worry about are things like poor architectural descisions, or bad coding habits. These are the things that will "bite you later and trigger a rewrite". Not the stack itself.
This is the exact same in code, start with the language and the tools that you and your team know.
When you start a project, a new feature, a new anything, you should always start with what has worked for you in the past.
A better approach to solve for this problem is to completely ignore, at least for the moment, any idea of decision fatigue and instead focus on developing a solid plan. A good plan will account for risk acceptance, business requirements, a task list, and costs. The most valuable part of a solid plan is not having some magically complete roadmap or blueprint, but that it puts all your challenging decisions up front. It forces you to examine what you are willing to pay in terms of money, time, and effort.
Once you have a well envisioned plan the only consideration of decision fatigue should be limited to maintenance. If a given maintenance effort is too expensive or requires too much investigation or risk then you need to refactor and break things down into more primitive units.
When it comes to running a team there are only two things to consider heading into planning: a defined performance baseline and scope of effort. Those two things are completely in competition and you have to balance them against a budget of available resources. As the team leader be willing to accept input and suggestions from your team, but set the baseline with confidence and don't let the children run the daycare.
2. Calling your team of highly paid professionals children who don't deserve autonomy? Not a winning strategy.
Contrary to the sibling comment, I actually think this explains a lot of my experience. Thanks for pointing it out.
Here are a few things that I have learned:
# Requirements - List out clear requirements, get an agreement with your co-decision makers, and the necessary people. Make sure you address for at least 2-3 yrs of requirements. It is likely, that new requirements would pop up and your system should be adaptive to this - if you think that your system is not going to address a specific thing, think twice, you may get buy-off & assurance from product, directors. However be prepared mentally that six months down the line, this will change.
# My Dos and Don't (always subject to change :))
- Do keep things simple
- Don't make a career out of a single project.
- Do list tradeoffs in your design
- Do document and perform deep dives.
- Don't ship prototypes.
- Don't be afraid to get critical feedback. If someone gives critical feedback, spend a few days to clearly understand the feedback
- Don't be defensive
- Don't be attached to what you build, it does not represent you.
- Do use new tech in baby steps
- Do experiment - The more you experiment, the more you learn, the more you will fail. Is this something that you have the mindset for? You can also ask how much do you have the time/energy for?
+1 to this. The very first thing I'd teach computer science students is "You are not your code... ego investment will only cloud your thinking" .
When the compiler says it doesnt work, trust it's not a compiler bug.
When the computer does what it's instructed and it doesnt work, trust it's not a JIT bug or that your code "clearly says to do what you think it does..."
When a coworker suggests an alternate solution, it doesn't mean either of you are a "better" engineer. (These kinds of decisions can only be decided across many variables and many data points).
At some point you need to commit. It's like relationships. How do you choose a partner out of billions of options? You browse around, learn what you like and don't like, make judgement calls, play around with some and eventually either make a conscious "this is the one" decision or else make a "we've been together so long, I guess we're a couple now" decision. At some point it becomes formal and you stop actively looking at other options because you're content with the option you chose.