I have huge respect for Mitchell, it's impressive what he achieved.
I agree with all the points of this article and would like to add one: Have a quick feedback loop. For me, it's really motivating to be able to make a change and quickly see the results. Many problems just vanish or become tangible to solve when you playfully modify your source code and observe the effect.
I feel there is a balance to strike with rushing to a demo. ..
(maybe it's an implicit tension between the satisfaction of finishing a component and getting a demo)
I think this is where the choice of language makes a big difference.
In Clojure, the difference between a "component" and a separate library/application is literally just adding a `deps.edn` file and then pointing to the directory from the parent project.
I think breaking the project in to small achievable goals is very sensible. But if you take the extra time to make the component stand on its own as a mini-lib .. it's very satisfying. For instance I had to write a "component" that would read some GeoJSON and segment it (it's took me a couple of days and was mostly a wrapped around GDAL or something). I could have hacked together a solution to rush to a demo - but instead I made a small little library out of it. When I was done with it, I had a sense of "I made a thing". To be clear.. it's still kind of ugly and I would be horrified if someone else tried to use it and submitted PRs.. but it's also not a coupled tangle of code in my larger codebase.
as he says "Build for yourself" - the library/application should only do what you need
By contrast, if I was working in C++ making an API and decoupled library would be such a chore that'd never bother
The most important aspect is that this all ends up not just much more satisfying at every step - but it makes your code incredibly decoupled and refactorable. The more you rush to a demo the more your code is coupled and hard to refactor.
> By contrast, if I was working in C++ making an API and decoupled library would be such a chore that'd never bother
Are you sure? Isn't it common to build convenience libraries, not because you need to have it as a library, but as an internal documentation? To the produced binary it has exactly the same effect.
Granted maintaining a stable API/ABI is work, but that comes down to architecture and is true in every language.
>This is an area where I think experience actually hurts. I've seen senior engineers get bogged down building the perfect thing and by the time they get a demo, they realize it sucks. The implementation doesn't suck, but the product or feature itself actually sucks.
This resonates with me. Sometime i want to "turn off" my brain and write shitty code.
Back in the day, i made a lot of toy project. Sometime all the source code is in single file. No respect to modularity. But it was fun, and it worked. Now just try to finish a toy project seem much harder than ever.
Great read, but I was expecting something different based on the title.
This sounds like his approach working on personal projects. I'm really curious about large technical team projects though. What's the best approach to getting stuff done and making sure everyone is working towards the same goal etc.
After 15 years I have yet to see a technical project that hasn't run over budget, over time, under delivered or burnt people out.
I'm sure there are people out there with counter examples who know exactly how to deliver projects at a massive scale. Any links/suggested reading would be appreciated!
Maybe I’ll get maligned for saying this, but as someone who’s managed and successfully delivered software projects of all sizes, I’ve become somewhat of a convert to the Scaled Agile Framework. I treat it exactly as intended, as a framework. It’s a strawman to adapt based on context, and what I value most is how it led me to explore deeper source material and form my own conclusions.
Over the years, I’ve learned that true success starts with clearly understanding why you’re building something. Without clear goals, it’s impossible to prioritize or even know where to start. That clarity drives better sequencing, and sometimes the wisdom to not build something at all.
The next critical factor is empathy. You have to see through your customer’s eyes and validate that what you’re building actually solves their problem. That doesn’t mean giving them everything they ask for, but rather understanding their pain points well enough to deliver real value.
Ultimately, most projects go over budget or underdeliver because teams spend too much time building the wrong things. If we instead focused on continually steering toward desired, valuable functionality (things people genuinely want or will pay for), more software projects would appear like successes.
> If we instead focused on continually steering toward desired, valuable functionality (things people genuinely want or will pay for), more software projects would appear like successes.
Yes! All overbudget, late to complete, successes of course.
I jest a bit. I think the term "deadline" is completely inappropriate for most projects.
I prefer, "target series", for any significant project.
Each word communicates something very important.
The first target should be set via (1) known knowns/unknowns, (2) some wisdom about unknown unknowns, and (3) mindful optimism.
The last should almost always ensure the project misses its first target. Which sounds backwards, until accounting for the considerable creative impact, and absolute time savings, that a time constraint creates. And how much a team can improve its productivity over time, by continually targeting informed optimistic targets.
Improving a teams rate of return has compounding value, for everyone.
But this is a self-imposed, let's see how well we can beat ourselves, time constraint. Never an, OMG I might lose my job / standing / reputation / customer credibility time constraint.
If the first target is missed, or as soon as it is clear it is wildly unachievable, a second target is set the same way, with the new information.
Each target in a series should require less padding. If more padding is added it is a sign to step back and consider what is not being understood.
Over time, a team should develop a pretty good pattern of completion at target 2 or 3, or something like that. In a way that becomes fairly predictable. It doesn't really matter what the number is, just that the team develops a rhythm that actually does provide some predictability based on current knowledge, even if it isn't in absolute time terms.
This is the best path for optimizing both rate of progress, fast recognition of problems with progress, and realistic levels of project arc predictability. In my experience anyway.
I remember reading this when it was published, and the milestone breakdown Mitchell describes was exactly what got us from a theoretical solution to a production platform.
That combined with launching an MVP first rather than the "complete" vision. Shipping early meant we avoided the trap of spending months perfecting features no one actually used.
> My goal with the early sub-projects isn't to build a *finished* sub-component, it is to build a good enough sub-component so I can move on to the next thing on the path to a demo.
This is so enlightening. And I realized that to do this, one has to "skip" something. Other folks mention they ignore code modularity when doing this, I don't think I will do that, keeping code clean and reading/working in such a codebase actually make me satisfied and motivated. For me, I am going to "skip" algorithms, data strucuture and performance.
So the point here is probably, we should skip things, but if a thing motivates you, it should not skipped?
An interesting consideration is how the chosen modularization approach can impact onboarding time for new contributors. A well structured breakdown might not just aid initial development speed, but also reduce ramp up friction for future team members or external collaborators. This is an impartant factor often under-estimated in solo-driven projects.
//Decompose a large problem into smaller problems.
Only solve the smaller problem enough to progress on a demo...
...then continue to iterate on more functionality.
Make demos as frequently as you can.
Prioritize functionality that enables you to adopt your own software
Go back and iterate on each component as needed for future improvements//
So in essence we have: Empirical Process Control, Self-Organization, Collaboration, Value-Based Prioritization, Time-Boxing, and Iterative Development.
Is this just solo-SCRUM or am I missing something here?
For me the best demo is a test module lighting up all green.
How do make this into a sexy image for management. Sure, business logic is stubbed, but my carefully crafted strongly typed interfaces all mesh together! Imagine the future dividends!
This isn't an "everyone who struggles to finish projects or is a perfectionist has it" kind of post, but if this has been a long-term pattern for you, one that's caused real suffering, lost jobs or contracts, or kept you from finishing important work - please consider getting checked for ADHD. It took me 42 years to get diagnosed, and starting on stimulants genuinely changed my life and pretty much solved those issues for me.
Interesting, especially the "Build for Yourself" section!
Building software for yourself can help you solve a problem you have. If you're also using the software yourself, you can fix bugs in the software. I found some bugs in a web server I'm building by trying to use it myself.
Building demos as a key part of development is really key imo. Demos act as a half-step between working on the software (programming) and writing about the software (writing). Demos act more as a way to continually validate your own theses about what a project SHOULD do, and act as nice feedback mechanism as you continue to work. They are also long-living, so when you break something you can see you broke it and continue the feedback system again.
I do this as part of the work on my own game engine:
I start with the standards. What file types and protocols and external tools do I want to support? How do I make Git and SyncThing understand my files?
How do I build this on an unmodified system with unmodified popular and trusted libraries?
How do I make sure that users don't have to modify my project to use it, unless they are going to be contributors?
If I'm truly just building for myself, which almost never happens, I look at resuability. If I have to learn a new tool, what's the most general one, that I'm likely to be glad I learned when I'm doing some other project?
27 comments
[ 4.9 ms ] story [ 69.1 ms ] threadMy approach to building large technical projects - https://news.ycombinator.com/item?id=36161397 - June 2023 (27 comments)
I agree with all the points of this article and would like to add one: Have a quick feedback loop. For me, it's really motivating to be able to make a change and quickly see the results. Many problems just vanish or become tangible to solve when you playfully modify your source code and observe the effect.
When I want to try/fix something, if the setup itself takes hours, I lose heart and move on.
Thats why I love lisp (or anything with a decent Repl). Instant gratification.
The second you lose motivation the whole thing poofs into non-existence, so making it enjoyable is almost the most important facet.
I think this is where the choice of language makes a big difference. In Clojure, the difference between a "component" and a separate library/application is literally just adding a `deps.edn` file and then pointing to the directory from the parent project.
I think breaking the project in to small achievable goals is very sensible. But if you take the extra time to make the component stand on its own as a mini-lib .. it's very satisfying. For instance I had to write a "component" that would read some GeoJSON and segment it (it's took me a couple of days and was mostly a wrapped around GDAL or something). I could have hacked together a solution to rush to a demo - but instead I made a small little library out of it. When I was done with it, I had a sense of "I made a thing". To be clear.. it's still kind of ugly and I would be horrified if someone else tried to use it and submitted PRs.. but it's also not a coupled tangle of code in my larger codebase.
as he says "Build for yourself" - the library/application should only do what you need
By contrast, if I was working in C++ making an API and decoupled library would be such a chore that'd never bother
The most important aspect is that this all ends up not just much more satisfying at every step - but it makes your code incredibly decoupled and refactorable. The more you rush to a demo the more your code is coupled and hard to refactor.
Are you sure? Isn't it common to build convenience libraries, not because you need to have it as a library, but as an internal documentation? To the produced binary it has exactly the same effect.
Granted maintaining a stable API/ABI is work, but that comes down to architecture and is true in every language.
This resonates with me. Sometime i want to "turn off" my brain and write shitty code.
Back in the day, i made a lot of toy project. Sometime all the source code is in single file. No respect to modularity. But it was fun, and it worked. Now just try to finish a toy project seem much harder than ever.
This sounds like his approach working on personal projects. I'm really curious about large technical team projects though. What's the best approach to getting stuff done and making sure everyone is working towards the same goal etc.
After 15 years I have yet to see a technical project that hasn't run over budget, over time, under delivered or burnt people out.
I'm sure there are people out there with counter examples who know exactly how to deliver projects at a massive scale. Any links/suggested reading would be appreciated!
Over the years, I’ve learned that true success starts with clearly understanding why you’re building something. Without clear goals, it’s impossible to prioritize or even know where to start. That clarity drives better sequencing, and sometimes the wisdom to not build something at all.
The next critical factor is empathy. You have to see through your customer’s eyes and validate that what you’re building actually solves their problem. That doesn’t mean giving them everything they ask for, but rather understanding their pain points well enough to deliver real value.
Ultimately, most projects go over budget or underdeliver because teams spend too much time building the wrong things. If we instead focused on continually steering toward desired, valuable functionality (things people genuinely want or will pay for), more software projects would appear like successes.
Yes! All overbudget, late to complete, successes of course.
I jest a bit. I think the term "deadline" is completely inappropriate for most projects.
I prefer, "target series", for any significant project.
Each word communicates something very important.
The first target should be set via (1) known knowns/unknowns, (2) some wisdom about unknown unknowns, and (3) mindful optimism.
The last should almost always ensure the project misses its first target. Which sounds backwards, until accounting for the considerable creative impact, and absolute time savings, that a time constraint creates. And how much a team can improve its productivity over time, by continually targeting informed optimistic targets.
Improving a teams rate of return has compounding value, for everyone.
But this is a self-imposed, let's see how well we can beat ourselves, time constraint. Never an, OMG I might lose my job / standing / reputation / customer credibility time constraint.
If the first target is missed, or as soon as it is clear it is wildly unachievable, a second target is set the same way, with the new information.
Each target in a series should require less padding. If more padding is added it is a sign to step back and consider what is not being understood.
Over time, a team should develop a pretty good pattern of completion at target 2 or 3, or something like that. In a way that becomes fairly predictable. It doesn't really matter what the number is, just that the team develops a rhythm that actually does provide some predictability based on current knowledge, even if it isn't in absolute time terms.
This is the best path for optimizing both rate of progress, fast recognition of problems with progress, and realistic levels of project arc predictability. In my experience anyway.
That combined with launching an MVP first rather than the "complete" vision. Shipping early meant we avoided the trap of spending months perfecting features no one actually used.
> My goal with the early sub-projects isn't to build a *finished* sub-component, it is to build a good enough sub-component so I can move on to the next thing on the path to a demo.
This is so enlightening. And I realized that to do this, one has to "skip" something. Other folks mention they ignore code modularity when doing this, I don't think I will do that, keeping code clean and reading/working in such a codebase actually make me satisfied and motivated. For me, I am going to "skip" algorithms, data strucuture and performance.
So the point here is probably, we should skip things, but if a thing motivates you, it should not skipped?
So in essence we have: Empirical Process Control, Self-Organization, Collaboration, Value-Based Prioritization, Time-Boxing, and Iterative Development.
Is this just solo-SCRUM or am I missing something here?
How do make this into a sexy image for management. Sure, business logic is stubbed, but my carefully crafted strongly typed interfaces all mesh together! Imagine the future dividends!
Building software for yourself can help you solve a problem you have. If you're also using the software yourself, you can fix bugs in the software. I found some bugs in a web server I'm building by trying to use it myself.
I do this as part of the work on my own game engine:
https://github.com/zinc-framework/Zinc.Demos/tree/main/Zinc....
How do I build this on an unmodified system with unmodified popular and trusted libraries?
How do I make sure that users don't have to modify my project to use it, unless they are going to be contributors?
If I'm truly just building for myself, which almost never happens, I look at resuability. If I have to learn a new tool, what's the most general one, that I'm likely to be glad I learned when I'm doing some other project?