I used to think it was much easier to start coding out a problem before doing any sort of planning. My argument for this was that it gave me a hands-on understanding of the data/workflows/etc involved.
Now, the first thing I do is grab a piece of paper, map out what I do and don’t know, and fill in any knowledge gaps by asking questions. It still isn’t second nature for me, but I think it’s something worth prioritizing.
For instance, I run a hypothesis against the system and fill out those gaps in my understanding (napkin, whatever); implement a vertical slice; adapt for the known unknowns.
"Unknown unknowns" - often about oddball limitations of the codebase or it's dependencies - limit my ability to enumerate what I don't know, and planning is often used as an excuse to get trapped in analysis paralysis (by myself included!)
Which isn't to say there's not plenty of room for inquery, questions, and planning, but don't underestimate the value of bulldozing through problems. (That said, this has caused me to step on the toes of fellow contributors to a project without realizing it.)
> And so, one day, a race of hyper-intelligent pan-dimensional beings built themselves a gigantic supercomputer called Deep Thought, to calculate once and for all the answer to the Ultimate question of Life, the Universe, and Everything. For seven-and-a-half million years Deep Thought computed and calculated and eventually announced that the answer was in fact, “Forty-Two”. And so another, even bigger computer had to be built to find out what the actual question was.
My company has quite a few IT architects (the "senior" ones are the worst). They always accuse you of "solutioning" when you propose something, drag you through weeks of meetings, produce a lot of documents and when they are done, they either do nothing or propose a "beautiful" solution that only works on paper.
After going through this a few times I have concluded it's easier to whip out a few prototypes for different aspects of the project. This allows you to understand the problem better while also exploring workable solutions. Often you don't understand the problem but you also don't understand ways to solve them well either and need to do some learning.
You just shouldn't fall into the trap of thinking that once the prototypes work, they are ready for production. Sometimes I wonder if it should be mandatory to scrap that code and start from scratch to development real solution.
Iterating can take many forms, and different teams have different workflows. For my work, I would rarely be able to make a "quick prototype" even if I wanted to, since most of my projects need to modify other people's code. And they have to be deployed to be validated.
In those cases you better have some consistent plan that everyone agrees on before anybody else lets you touch code that they are responsible for. Yes, you can still make mistakes and iterate, but you need to plan for those too.
But I agree with you if you can prototype with a fresh body of code, not touching shared repositories.
If your quick prototypes as a rule require complex interactions with other team's services, the problem is your architecture. Try to find the smallest step toward an architecture that allows you to prototype in relative isolation. Take that and repeat.
Yes, prototypes have to be deployed. That goes for everyone's prototypes. That's how you get feedback on them. If it's a big deal that you have to deploy your prototypes, find the smallest step you can towards deployments that are less of a big deal and take it. Then repeat.
Paper engineering is never better than just trying out the thing in the real world. Most engineering professions have transaction costs that make this impossible. We have the ability to lower those transaction costs, if we just make the right effort. Come on, do it! It will be fun.
(And "that doesn't work here" has been debunked so many times. It does work also for you.)
> You just shouldn't fall into the trap of thinking that once the prototypes work, they are ready for production.
That is a huge “just” and painfully laughable when many of those architects were young men. The industry was practically incapable of looking at software that looks like it should work and declaring victory. Not just to the level of trope, but to the level of trauma. It was fucking brutal.
But we haven’t solved this problem, we’ve just gotten really good at avoiding that situation (eg, CI/CD). That fence is there for a reason, don’t remove it.
“Solutioning” is just corporate buzz phrase for the XY problem. I threw away a day’s worth of code yesterday because I realized at the last moment that I was solving the wrong problem, literally as I was writing the summary for the PR. We have to do A because of B, and we can’t change B because of… umm, why can’t we change B? Because I didn’t understand all the consequences of B the last time I faced a related problem. But now I do, and it’s fairly arbitrary if I fix the error handling in two methods and a couple of tests, OK.
Every line of code you write has to be maintained forever. Most of the ones you delete do not. A few have to be added back in later. Big whup.
At least in my working history “prototype” is explicitly designated for the trash heap. Everything up to that point in your description sounds really good. Just… file your learnings, put all of the materials that lead to your findings in a graveyard, and make digging graves exactly that much effort.
While working at Google, I came to appreciate design docs. They're a way to help think through the system design, and help communicate the goals, decisions, etc. at a high level. It also provides experts with a place to put in checklists, to encourage me to answer how e.g. privacy is affected.
However, what I missed was a "problem doc". Something for when I just have an idea for a problem to work on, but forcing myself not to share a solution. Something that triggers brainstorming in the reader, and perhaps the right place to also ask "is this the right problem?"
Did the Google design docs follow any guides on how to structure/present the information and how to draw the line between too little and too much detail?
There is no standard for design docs at Google and everyone is free to experiment, but most commonly used templates usually share the same ideas. My favourite is to write a green doc and a blue doc. The green doc is typically a "one pager" (basically always 2 pages at Google) and it typically never discusses implementation details. It's mostly about why we need X and the goals and non-goals of the project. Once your green doc is reviewed by your teammates, if everyone's happy, you write a more in-depth blue doc where you'll probably design the protobufs and interfaces, draw the system architecture, make framework choices and trade-offs, design metrics for monitoring, think about scalability, alternative approaches, and maybe even outline a timeline for the project if you're feeling ambitious. This, too, will go through a few rounds of reviews until everyone is satisfied. The green doc is significantly less investment than the blue doc, but it's still some investment (say ~1 SWE-week of writing the first draft + reviews + future iterations), so this pattern only makes sense if your project is complex enough. If I'm just adding things to an existing project and I still need a design doc reviewed by other people, I'd probably just write one design doc.
29 comments
[ 3.6 ms ] story [ 68.1 ms ] threadedit: apparently this is paraphrased (beyond the obvious). see https://en.wikiquote.org/wiki/Frank_Westheimer
I looked at the HTML Source; I hope that is legal.
Now, the first thing I do is grab a piece of paper, map out what I do and don’t know, and fill in any knowledge gaps by asking questions. It still isn’t second nature for me, but I think it’s something worth prioritizing.
For instance, I run a hypothesis against the system and fill out those gaps in my understanding (napkin, whatever); implement a vertical slice; adapt for the known unknowns.
Which isn't to say there's not plenty of room for inquery, questions, and planning, but don't underestimate the value of bulldozing through problems. (That said, this has caused me to step on the toes of fellow contributors to a project without realizing it.)
After going through this a few times I have concluded it's easier to whip out a few prototypes for different aspects of the project. This allows you to understand the problem better while also exploring workable solutions. Often you don't understand the problem but you also don't understand ways to solve them well either and need to do some learning.
You just shouldn't fall into the trap of thinking that once the prototypes work, they are ready for production. Sometimes I wonder if it should be mandatory to scrap that code and start from scratch to development real solution.
In those cases you better have some consistent plan that everyone agrees on before anybody else lets you touch code that they are responsible for. Yes, you can still make mistakes and iterate, but you need to plan for those too.
But I agree with you if you can prototype with a fresh body of code, not touching shared repositories.
Yes, prototypes have to be deployed. That goes for everyone's prototypes. That's how you get feedback on them. If it's a big deal that you have to deploy your prototypes, find the smallest step you can towards deployments that are less of a big deal and take it. Then repeat.
Paper engineering is never better than just trying out the thing in the real world. Most engineering professions have transaction costs that make this impossible. We have the ability to lower those transaction costs, if we just make the right effort. Come on, do it! It will be fun.
(And "that doesn't work here" has been debunked so many times. It does work also for you.)
That is a huge “just” and painfully laughable when many of those architects were young men. The industry was practically incapable of looking at software that looks like it should work and declaring victory. Not just to the level of trope, but to the level of trauma. It was fucking brutal.
But we haven’t solved this problem, we’ve just gotten really good at avoiding that situation (eg, CI/CD). That fence is there for a reason, don’t remove it.
“Solutioning” is just corporate buzz phrase for the XY problem. I threw away a day’s worth of code yesterday because I realized at the last moment that I was solving the wrong problem, literally as I was writing the summary for the PR. We have to do A because of B, and we can’t change B because of… umm, why can’t we change B? Because I didn’t understand all the consequences of B the last time I faced a related problem. But now I do, and it’s fairly arbitrary if I fix the error handling in two methods and a couple of tests, OK.
Every line of code you write has to be maintained forever. Most of the ones you delete do not. A few have to be added back in later. Big whup.
Looks like author encountered something one time. And they made resolution that it will happen every time. Doesn't sound reasonable.
Sometimes it isn't.
«make sure you both understand the problem, before coming up with solutions»
[1] https://www.agendashift.com [2] https://www.agendashift.com/done
However, what I missed was a "problem doc". Something for when I just have an idea for a problem to work on, but forcing myself not to share a solution. Something that triggers brainstorming in the reader, and perhaps the right place to also ask "is this the right problem?"
(The article itself seems to just be about https://en.wikipedia.org/wiki/XY_problem)