A major factor supporting evolution over big up-front design is the drift in system requirements over time. Even on large military like projects, apparently there's "discovery"--and the more years that pass, the more requirements change.
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Gall’s Law
Some systems require a total commitment to the complexity because it is intrinsic. There is no "simple" form that also works, even if poorly. In many contexts, "systems thinking" is explicitly about the design of systems that are not reducible to simpler subsystems, which does come up in many types of engineering. Sometimes you have to eat the whole elephant.
There is a related phenomenon in some types of software where the cost of building an operational prototype asymptotically converges on the cost of just writing the production code. (This is always a fun one to explain to management that think building a prototype massively reduces delivery risk.)
Software cannot be built like skyscrapers because the sponsors know about the malleability of the medium and treat it like a lump of clay that by adding water can be shaped to something else.
The Evolution method outlined also seems born from the Continuous Delivery paradigm that was required for subscription business models. I would argue Engineering is the superior approach as the Lean/Agile methods of production were born from physical engineering projects whose end result was complete. Evolution seems to be even more chaotic because an improper paradigm of 'dev ops' was used instead of organically emerged as one would expect with an evolving method.
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
I think that a comparison with Engineering is not that helpful for software.
Software has 0 construction cost but that it does have is extremely complicated behavior.
Take a bridge for example: the use case is being able to walk or drive or ride a train across it. It essentially proves a surface to travel on. The complications of providing this depend on the terrain, length etc etc and are not to be dismissed but there's relatively little doubt about what a bridge is expected to do. We don't iterate bridge design because we don't need to know much from the users of the bridge: does it fulfill their needs, is it "easy to use" etc AND because construction of a bridge is extremely expensive so iteration is also incredibly costly. We do, however, not build all bridges the same and people develop styles over time which they repeat for successive bridges and we iterate that way.
In essence, cycling is about discovering more accurately what is wanted because it is so often the case that we don't know precisely at the start. It allows one to be far more efficient because one changes the requirements as one learns.
> There are two main schools of thought in software development about how to build really big, complicated stuff.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
> There should be some balanced path in the middle somewhere, but I haven’t stumbled across a formal version of it after all these decades.
That's very simple. The balanced path depends directly on how much of the requirements and assumptions are going to change during the life time of the thing you are building.
Engineering is helpful only to the extent you can forsee the future changes. Anything beyond that requires evolution.
You are able to comment on the complexity of that large company only because you are standing in the future into 50 years from when those things started take shape. If you were designing it 50 years back, you would end up with same complexity.
The nature's answer to it is, consolidate and compact. Everything that falls onto earth gets compacted into a solid rock over time, by a huge pressure of weight. All complexity and features are flattened out. Companies undergo similar dynamics driven by pressures over time, not by big-bang engineering design upfront.
> Also, the other side of it is that evolutionary projects are just more fun. I’ve preferred them. You’re not loaded down with all those messy dependencies. Way fewer meetings, so you can just get into the work and see how it goes. Endlessly arguing about fiddly details in a giant spec is draining, made worse if the experience around you is weak.
IMO the problem isn't discussing the spec per se. It's that the spec doesn't talk back the way actual working code does. On a "big upfront design" project, there is a high chance you're spending a lot of time on moot issues and irrelevant features.
Making a good spec is much harder than making working software, because the spec may not be right AND the spec may not describe the right thing.
> There are two main schools of thought in software development about how to build really big, complicated stuff.
That feels like a straw man to me. This is not a binary question. For each small design decision you have a choice about how much uncertainty you accept.
There are no "two schools". There is at least a spectrum between two extremes and no real project was ever at either of the very ends of it. Actually, I don't think spectrum is a proper word even because this is not just a single dimension. For example, speed and risk often correlate but they are also somewhat independent and sometimes they anti-correlate.
100%. I came here to find something new from a field that I don't know but imagine has some good lessons for software. Instead I found someone commenting on small iterations vs big design which is quite ho-hum by comparison.
Lots of wisdom in this post about some of the realities of software development.
The core point they're trying to make is that agile (or similar) practices are the incorrect way to approach consolidation of smaller systems into bigger ones when the overall system already works and is very large.
I agree with their assertion that being forced to address difficult problems earlier on in the process results in ultimately better outcomes, but I think it ignores the reality that properly planning a re-write of monumentally sized and already in use system is practically impossible.
It takes a long time (years?) to understand and plan all the essential details, but in the interim the systems you're wanting to rewrite are evolving and some parts of the plan you thought you had completed are no longer correct. In essence, the goal posts keep shifting.
In this light, strangler fig pattern is probably the pragmatic approach for many of these re-writes. It's impossible to understand everything up front, so understand what you reasonably can for now, act on that, deliver something that works and adds value, then rinse and repeat. The problem is that for sufficiently large system, this will take decades and few software architects stick around at a single company long enough to see it through.
A final remark I want to make is that, after only a few years of being a full-time software developer, "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written, this requires skills in effective communication with various people, including other software developers and (probably more importantly) non-technical people who understand how the business processes actually need to work. If you want to be a great software developer, learn how to be good at this.
This missed the point that they are ignoring evolution is literally the way you build things. There is no other way. You don't know what is actually needed or what might work really. You try things and then compress later. If you can try bigger things, bigger leaps great.
>In a sense, it is the difference between the way an entrepreneur might approach doing a startup versus how we build modern skyscrapers. Evolution versus Engineering.
There are core differences in software engineering that, unlike in construction work:
- making changes is often cheaper
- we might not know beforehand everything that is needed to be built, especially unknown unknowns
I would still agree that the truth is somewhere in between, but I would argue that, for software, it's closer to the evolutionary approach.
> you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you.
But I do still think there's a lot of value into coming up with a good plan before jumping in. A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first. (I wonder if we reinforce the jumping in head first mentality because people figure out you can't plan everything) A good plan helps you prevent changing specs and prepares you for hiccups. It helps by having others but basically all you do is try to think of all the things that could go wrong. Write them down. Triage. If needed, elevate questions to the decision makers. Try a few small scale tests. Then build out. But building out you're always going to find things you didn't see. You can't plan forever because you'll never solve the unknown unknowns until you build, but also good prep makes for smoother processes. It's the reason engineers do the math before they build a bridge. Not because the math is a perfect representation and things won't change (despite common belief, it's not static) but because the plan is cheaper than the build and having a plan allows you to better track changes and helps you determine how off the rails you've gone.
It is also perplexing to me that people think they can just plan everything out and give it to a LLMs. Do you really believe your manager knows everything that needs to be done when they assign jobs to you? Of course not, they couldn't. Half the job is figuring out what the actual requirements are.
Nah I’m good. I’ve watched system architecture framework views be developed. Years of prep and planning. System is released and half the employees that had requirements no longer work there and the business already pivoted to a new industry focus.
There’s a reason we went this way in software development a quarter century ago.
"It’s not that you could cut the combined complexity in half, but more likely that you could bring it down to at least one-tenth of what it is today, if not even better. It would function better, be more reliable, and would be far more resilient to change. It would likely cost far less and require fewer employees as well. All sorts of ugly problems that they have now would just not exist."
Incidentally this highlights a problem when using chatbots to build large software projects that are intended to be used for a long period of time.
The key is not how much code you can add but how little you can get away with.
Chatbots only solution ever is to ADD code. They're not good at NOT writing code or even deleting it because after all the training set for the lines of code that do not exist is an empty set. Therefore it's impossible to train a robot to not write code.
What's better than generating 10kloc really fast? Not having it in the first place.
Big upfront designs are obviously based on big upfront knowledge which nobody has.
When they turn out to be based on false assumptions of simplicity the fallout is that the whole thing can't go forward because of one of the details.
Evolutionary systems at least always work to some degree even if you can look after the fact and decide that there's a lot of redundancy. Ideally you would then refactor the most troublesome pieces.
Borrowing from mechanical/electrical etc. Limit the number of things you can build with. An example in the comments here was a gear. You make a new gear based on examples of gears that work. So whats the software equivalent of a gear? an axle, a bearing, etc.. Using OO or some ABI, you specify an object is a gear and behaves like a gear and magically you know how it does or doesnt fit together with other objects. I know this idea has been used before but im wondering if theres a well known software framework or library. We have things like the stl in cpp or built in libraries in python but im thinking of a higher level abstraction.
I disagree with most of this article, but this part stood out:
> the size of the iterations matters, a whole lot. If they are tiny, it is because you are blindly stumbling forward. If you are not blindly stumbling forward, they should be longer, as it is more effective.
You are not blindly stumbling forward, you're moving from (working software + tiny change) to (working software including change). And repeat. If there's a problem, you learn about it immediately. To me that's the opposite of moving blindly.
> you really should stop and take stock after each iteration.
Who is not taking stock after every iteration? This is one of the fundamental principles of agile/lean/devops/XP/scrum. This one sentence drastically lowers my impression of the author's ability to comment on the subject.
> The faster people code, the more cleanup that is required. The longer you avoid cleaning it up, the worse it gets, on basically an exponential scale.
Unsafe tempo is as likely to happen in big-spec design projects as in small iterations. In fact, working in careful small iterations helps us manage a realistic tempo because we know we can't move faster than we can get things into production and evaluate.
The terrible outcomes listed in the same paragraph are linked to unwise practice and have nothing to do with small iteration size.
47 comments
[ 2.5 ms ] story [ 92.3 ms ] threadThere is a related phenomenon in some types of software where the cost of building an operational prototype asymptotically converges on the cost of just writing the production code. (This is always a fun one to explain to management that think building a prototype massively reduces delivery risk.)
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
Software has 0 construction cost but that it does have is extremely complicated behavior.
Take a bridge for example: the use case is being able to walk or drive or ride a train across it. It essentially proves a surface to travel on. The complications of providing this depend on the terrain, length etc etc and are not to be dismissed but there's relatively little doubt about what a bridge is expected to do. We don't iterate bridge design because we don't need to know much from the users of the bridge: does it fulfill their needs, is it "easy to use" etc AND because construction of a bridge is extremely expensive so iteration is also incredibly costly. We do, however, not build all bridges the same and people develop styles over time which they repeat for successive bridges and we iterate that way.
In essence, cycling is about discovering more accurately what is wanted because it is so often the case that we don't know precisely at the start. It allows one to be far more efficient because one changes the requirements as one learns.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Interesting times.
That's very simple. The balanced path depends directly on how much of the requirements and assumptions are going to change during the life time of the thing you are building.
Engineering is helpful only to the extent you can forsee the future changes. Anything beyond that requires evolution.
You are able to comment on the complexity of that large company only because you are standing in the future into 50 years from when those things started take shape. If you were designing it 50 years back, you would end up with same complexity.
The nature's answer to it is, consolidate and compact. Everything that falls onto earth gets compacted into a solid rock over time, by a huge pressure of weight. All complexity and features are flattened out. Companies undergo similar dynamics driven by pressures over time, not by big-bang engineering design upfront.
IMO the problem isn't discussing the spec per se. It's that the spec doesn't talk back the way actual working code does. On a "big upfront design" project, there is a high chance you're spending a lot of time on moot issues and irrelevant features.
Making a good spec is much harder than making working software, because the spec may not be right AND the spec may not describe the right thing.
That feels like a straw man to me. This is not a binary question. For each small design decision you have a choice about how much uncertainty you accept.
There are no "two schools". There is at least a spectrum between two extremes and no real project was ever at either of the very ends of it. Actually, I don't think spectrum is a proper word even because this is not just a single dimension. For example, speed and risk often correlate but they are also somewhat independent and sometimes they anti-correlate.
The core point they're trying to make is that agile (or similar) practices are the incorrect way to approach consolidation of smaller systems into bigger ones when the overall system already works and is very large.
I agree with their assertion that being forced to address difficult problems earlier on in the process results in ultimately better outcomes, but I think it ignores the reality that properly planning a re-write of monumentally sized and already in use system is practically impossible.
It takes a long time (years?) to understand and plan all the essential details, but in the interim the systems you're wanting to rewrite are evolving and some parts of the plan you thought you had completed are no longer correct. In essence, the goal posts keep shifting.
In this light, strangler fig pattern is probably the pragmatic approach for many of these re-writes. It's impossible to understand everything up front, so understand what you reasonably can for now, act on that, deliver something that works and adds value, then rinse and repeat. The problem is that for sufficiently large system, this will take decades and few software architects stick around at a single company long enough to see it through.
A final remark I want to make is that, after only a few years of being a full-time software developer, "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written, this requires skills in effective communication with various people, including other software developers and (probably more importantly) non-technical people who understand how the business processes actually need to work. If you want to be a great software developer, learn how to be good at this.
In short: the tension described in "systems thinking" is the same one as the one between "spec driven" vs. "iterative prompting"
There are core differences in software engineering that, unlike in construction work:
- making changes is often cheaper
- we might not know beforehand everything that is needed to be built, especially unknown unknowns
I would still agree that the truth is somewhere in between, but I would argue that, for software, it's closer to the evolutionary approach.
But I do still think there's a lot of value into coming up with a good plan before jumping in. A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first. (I wonder if we reinforce the jumping in head first mentality because people figure out you can't plan everything) A good plan helps you prevent changing specs and prepares you for hiccups. It helps by having others but basically all you do is try to think of all the things that could go wrong. Write them down. Triage. If needed, elevate questions to the decision makers. Try a few small scale tests. Then build out. But building out you're always going to find things you didn't see. You can't plan forever because you'll never solve the unknown unknowns until you build, but also good prep makes for smoother processes. It's the reason engineers do the math before they build a bridge. Not because the math is a perfect representation and things won't change (despite common belief, it's not static) but because the plan is cheaper than the build and having a plan allows you to better track changes and helps you determine how off the rails you've gone.
It is also perplexing to me that people think they can just plan everything out and give it to a LLMs. Do you really believe your manager knows everything that needs to be done when they assign jobs to you? Of course not, they couldn't. Half the job is figuring out what the actual requirements are.
Nah I’m good. I’ve watched system architecture framework views be developed. Years of prep and planning. System is released and half the employees that had requirements no longer work there and the business already pivoted to a new industry focus.
There’s a reason we went this way in software development a quarter century ago.
Software is not a skyscraper.
Comments on actual blog post: 0
Why are people so afraid to leave replies on the author’s OC
Incidentally this highlights a problem when using chatbots to build large software projects that are intended to be used for a long period of time.
The key is not how much code you can add but how little you can get away with.
Chatbots only solution ever is to ADD code. They're not good at NOT writing code or even deleting it because after all the training set for the lines of code that do not exist is an empty set. Therefore it's impossible to train a robot to not write code.
What's better than generating 10kloc really fast? Not having it in the first place.
When they turn out to be based on false assumptions of simplicity the fallout is that the whole thing can't go forward because of one of the details.
Evolutionary systems at least always work to some degree even if you can look after the fact and decide that there's a lot of redundancy. Ideally you would then refactor the most troublesome pieces.
> the size of the iterations matters, a whole lot. If they are tiny, it is because you are blindly stumbling forward. If you are not blindly stumbling forward, they should be longer, as it is more effective.
You are not blindly stumbling forward, you're moving from (working software + tiny change) to (working software including change). And repeat. If there's a problem, you learn about it immediately. To me that's the opposite of moving blindly.
> you really should stop and take stock after each iteration.
Who is not taking stock after every iteration? This is one of the fundamental principles of agile/lean/devops/XP/scrum. This one sentence drastically lowers my impression of the author's ability to comment on the subject.
> The faster people code, the more cleanup that is required. The longer you avoid cleaning it up, the worse it gets, on basically an exponential scale.
Unsafe tempo is as likely to happen in big-spec design projects as in small iterations. In fact, working in careful small iterations helps us manage a realistic tempo because we know we can't move faster than we can get things into production and evaluate.
The terrible outcomes listed in the same paragraph are linked to unwise practice and have nothing to do with small iteration size.