I know the author's entire argument doesn't stand or fall with the simile, but it still needs to be said: blueprints are more necessary for physical construction than they are for software because things made of wood and concrete are hard to change part-way through.
I could make the argument that source is the blueprint for a program. You use blueprints to know how to build a building, and you use source code to know how to build an executable, or rather, have a tool that automatically makes the program from the blueprint (source code).
Though the idea on specs vs proof systems is good enough , one of the reasons type systems are being studied is to actually make programming languages themselves a good proof system , so much so that DSLs themselves become the spec.
Proof systems have their uses and then there are places where they are just too much.The best is in the mid!
>> "am the creator of the TLA+ specification language."
I think TLA+ is/was being used at Amazon [1]
Safety with Preservation and Progress guarantees of critical systems demand design audits at a larger scale however if the scale of the program is actually quite small , things like type-systems themselves are sufficient proof of safety.
Ofcourse there are cases where even typing is too much!
One of the middlepaths is actually using inspection tools on code/programs as another aspect altogether.
While I do think proofs and planning are both useful, I think that the analogy doesn't work well.
The process of producing code is the process of specifying what the computer should do -- the deliverable is the specification. On the other hand, few civil engineers will start designing plumbing without at least having a sketch of the floor plan and an idea of the building uses.
The process of producing code is the process of specifying what the computer should do
I'm sure coding is "specifying what the computer should do", but that's not what people are talking about when they talk about specifications. A specification defines what the application should do. Producing code is implementing the specification. Without having any specification first you wouldn't have a clue what the code needs to do, so you wouldn't know what to write.
The problem with the analogy is that most people don't understand what a blueprint actually is. A blueprint is an architectural planning document. It defines the building in very broad terms - where the walls go, where there's a staircase, and so on. It doesn't go in to the detail of how the building is constructed. That comes from all the other documentation. As an analogy with software it's a pretty good one - a building's blueprint is equivalent to the high-level planning documentation that any big project needs.
However that doesn't mean I completely disagree with Lamport's arguments. Specifications are necessary in large-scale software because otherwise you can't know if you have actually implemented everything the program is supposed to do. When you release it the end-users get angry and insist that they told you in that email / meeting / watercooler conversation last year how important it was that the program frobnicate the warbles.
Lamport makes much of formal specifications. I agree that in some cases they do add value; trying to prove the correctness of a distributed protocol in any other way is dumb. But in my experience once you get up into application-level requirements a formal specification is basically source code you don't know how to compile; there is no formal correctness criteria against which you can verify it, and the stakeholders can't understand it in order to verify it.
The "source code is the blueprint for software" analogy doesn't work because the source only tells you what the application does, not what it's supposed to do. There is no way to tell, from the source code alone, whether or not what's been built is fit for purpose. You have to have some other document. The "specification is the blueprint" analogy is saying that the specification is a high level document that sets out what the application is meant to do.
As an example, imagine you're paid $1m to assess my super awesome new database. I give you the source code and tell you "the source is the blueprint." After reading through the code you discover there's no DELETE function. Have I built a clever immutable data store, or am I just an idiot who forgot a critical function? You can't tell that from the source.
Technically the source is a blueprint. But this comes down to scale.
Among other things I've worked on banking and corporate management applications. You cannot reason about the codebase at that scale. In these instances it's not possible to have a good outcome without a blueprint.
Using the building metaphor, you can get away with building a bird feeder without a blueprint. You can't get away with building a skyscraper without one. Houses, maybe. Either way, high level blueprints help everyone.
Good points - I think you need some kind of spec or something to reason about. But I don't think it needs to be complete because it is easier to iterate on software than buildings.
Taking the building metaphor, you can't clone a building locally and make changes to it, then ask another builder to review them, suggest some changes then push them out to a test environment before finally putting them into the real world. So for me, the building metaphor only goes so far but there are definitely some useful points in the article.
> you can get away with building a bird feeder without a blueprint
Definitely. But anyway, unless you have built several bird feeders or similar constructions before, your first version will probably be ugly and you'll have to iterate on it (i.e. find the right angles etc.) to make it better. What is of course fun, but costs you time.
I cannot tell that from the database, sure, because a database is simply structural information, not functional. Without a list of things that the database is supposed to do, it might be great or it might suck. Don't know.
But all modern systems have both structure and behavior. For every database there is a MVC model somewhere, for instance. And looking at those controllers, you should absolutely be able to tell whether the database is well-constructed or not.
Now there's another consideration: does the database structure map into some problem domain such that the introduction of new controllers will be supported? That's usually what people mean when they talk about architecture: the mapping of a problem domain over to a solution domain such that changes in the requirements have minimal impact on the solution.
In System-of-Systems work I'd argue yes, such work has some value -- as long as it is continually driving some kind of delivery (and not just "smart guys drawing boxes"). But in regular green-field system development? Not so much. That's because the tendency is to begin focusing on the wrong thing, the model instead of the user. Timeboxes help with this, so much so that I wouldn't advise any stand-alone architectural work in such an environment without team integration and time-boxing.
If I hand you a blueprint for a skyscraper and I realize there is no lock specified for the front doors. Does that mean it's a lockless skyscraper or I'm speaking to a jackass who forgot a critical function related to security? You can't tell from the blueprint.
If a specification is required to be 100% complete for all things that could possibly relate in any way, then even blueprints don't qualify.
Yes this is exactly how my organization works, because my team leader is too lazy to spec anything out.
Its an absolute nightmare when something doesn't work as expected, because rather than checking a specification that will take a couple of seconds to read, you basically have to debug software, which takes a lot longer.
But the article directly says source code can't be the spec because it contains low level details. E.g. you implement something with map/apply instead of loops because that's all your programming language has, or your filter out some characters from a data stream because they often make a third party parser raise Exceptions you don't understand. These are programming details that the source code contains but the spec shouldn't. Therefore I think your argument is debunked by the main article already.
In any case it's a good discussion to have in this thread, which is why I gave you the up.
An Architect's drawing of a building contains every single detail of that house. It's not a high-level sketch of what needs to be built, it's an exhaustive plan for every part of the finished product that has been extensively iterated with the customer to make sure it's correct. Changes to this plan once complete are expensive.
So yes, it's exactly like code.
But I just wish people would stop comparing software development to construction. They're not alike at all, not even vaguely.
Programming is not like bricklaying, because bricklaying is deterministic and uncreative: there is one correct method of laying the bricks, and any two competent bricklayers will lay the bricks in exactly the same way. But there is never one single correct method of writing a program, and no two programmers will implement a feature in the same way.
Any attempt to create a specification so exhausting that it allows no deviation by individual programmers should stop pretending it's a spec and just write the damn code. In fact, the author of the article should probably stop writing specs and just iterate prototypes to achieve the same end: there's no difference between "understanding our programming task" in formal English or understanding it in code, except that the code is more useful and a lot easier to test.
There are many points worth of discussion in your comment. I just pick one for now. The details of the architects work.
I am no architect, so all I think about it can be true or false. But what I remember from an architect friend is that an architect's work is not really that detailed. It's like the photoshop mockup of a website, not like its source code. It doesn't say exactly which materials should be chosen or how many screws should be used to connect two ends. An architects design says "There is a wall x cm thick and y cm long". The source code says "There z bricks and the last one we had to cut to n cm" (sometimes a comment will state why it had to be cut). For the source code it is to some degree by accident the amount of bricks creates something that can be called a wall.
But all the information about the correct numbers of bricks, the slump of the mortar, the number and type of screws - these are all just library functions. Why would you re-write all that perfectly serviceable code for each project?
Getting a library to work for your use case is exactly one of those things that needs quite a few workaround which the bricks are not part of the library. The library is the earth you build your bricks on top and you need to put in some work that your bricks can stand on it.
I'd have to argue that an architectural drawing is not very detailed. And even if it is, the builder takes plenty of latitude in constructing the building. For example, the way that builders pick nails isn't by accounting for humidity levels or type of wood they use (older pressure-treated wood can corrode certain metals faster, same as newer wood can corrode others.) Instead, most of them just grab what they have, so when nails start failing after a few decades that's not their problem. Not like the architect said to use all galvanized nails everywhere.
But I digress. Here is how bricklayers and programmers are similar: they commonly work in the vacuum of their own workspace, without awareness of the environment that impacts their projects. So once the projects are launched, they're impacted by things that weren't accounted for.
If the foundation is too weak, the brick wall will sag. If nearby trees grow too close to the wall, the wall will bow inward or outward. If someone builds on top of the brick wall and doesn't reinforce it over 8 feet, the inspector will condemn the work. The bricklayers are paid to just lay bricks, so they won't care about the nearby trees, they won't care about the foundation, they won't care about what you put on top of the wall.
Programmers will build the project to spec, and come launch time, not account for a particular load signature because that was never their concern. Seen that time and time again. Substitute foundation for baseline server power, trees for amount of interrupting processes and meddling managers altering the original program at random, and you have a pretty reasonable comparison to bricklaying.
I'm sorry but if you employ programmers to build the project to spec without any awareness of the environment it's going to be deployed to, then no wonder you see that problem time and time again.
The problem here is not that you have insufficiently detailed specs, but that you're treating your programmers like bricklayers.
I strongly suggest you try bringing your programmers into the room during the specification process, even get them to write some prototype code to explore the tricky areas (something bricklayers don't get to do either). You'll probably find you get better code as a result because the people writing the code are aware of the environment the code will be deployed to.
What I'm insinuating here is that the environment is always more complicated than meets the eye. Granted, project complexity is as disparate as they're randomly different from each other.
* Would you bring bricklayers to the design meeting? No. Just the foreman.
* Would you bring all programmers to the design meeting? No. Just the lead developer.
Yes, ideally you'd have all the programmers in the room, but more of them you introduce, you increase the communication overhead. And some projects involve dozens to hundreds to thousands of programmers and are complicated enough to not be able to bring everyone to the discussion table.
So what you have to do is use bricklayer programmers- trust them to do a single task right (warehouse uploaded photos, talk to APIs) but recognize their limits. I'm not faulting neither bricklayers nor programmers, but defining their borders more concisely.
yeah, I've worked on some big projects in my time, 100's of programmers. And all of them would have been done better by splitting the project up into smaller chunks and doing them in series.
I've never met a business problem in my 30+ years doing commercial IT that couldn't be split up into smaller problems with smaller solutions, but there seems to be a bias towards large teams and complex solutions in commercial IT.
I don't know what causes this (I suspect management ego or consultancy fees, the cause of most commercial IT problems), but it's a real problem. The vast majority of these huge projects fail, go massively over-budget, precisely because they're being managed wrong.
Programming is fundamentally not like bricklaying, because it's creative not deterministic. Better, more detailed and more complete specs won't make more successful projects.
If you have too many programmers to bring them into the room when you're specifying the problem, then your project team is too big. The answer is not to exclude programmers and create more detailed specs, because programmers working to a spec like that won't write good code.
I know, I've been there. One project in particular stands out. We had detailed specs for a range (a few hundred) of middle-layer business routines. I received a function spec, wrote the function, wrote a few tests that the function did what it should do, ticked the box. Rinse and repeat for months. There were people working in a similar fashion on the front end and database layer. The system went over-budget, never worked, and was finally scrapped. It's not hard to see why, when the entire system was being written in this fashion. Programming is not like bricklaying.
If you think about it for 5 minutes you'll see that "bringing more programmers into the room increases communication overhead" implies that there is some information that is required to be passed from the customer to the programmer. The solution to this problem is obviously not to exclude the programmer and reduce the information passed to them, because that will lead to inferior code because the programmer is missing information. Instead reduce the amount of information the team is dealing with by segmenting the problem better.
I was going to go the other way here and point out that blueprints are actually code for buildings.. the builder is the compiler I guess?
But blueprints are done at various scales and they differ in complexity and format ie. sketches, models, schedules. They also often come with a book of specifications, and these are typically only referred to when ordering parts or if something goes wrong - and they are always open to interpretation..
"TL;DR: Source code is the blueprint for software."
...and the 2x4s are the blueprint of a house and
the steel beams are the blueprint of the skyscraper....
I'll take the TL;DR at face value since it was written by the author. The statement completely obliterates the entire concept of a blueprint or plan. By that reasoning, it is true everything is its own blueprint, and the concept has no meaning (of course this is a standard tautology).
And, simply nodding to CYA documents (for when the customers/end users come back saying that your product doesn't do X), or to docs for when formal proofs are needed, doesn't cover the concept.
The point of planning (or blueprinting) is to think about what you are going to build before you get out the tools, abstracted away from the specific tools that you'll use, and to then record and communicate those thoughts to all the stakeholders. It doesn't have to be a complete formal, engraved-in-stone blueprint, but a modest amount of actual thinking and planning goes a loooong way.
I've often found that with a little thought, we'll use a different tool than originally expected. If you expect to just dive in and sort it out in your source code, then you've already a priori restricted your choice of tool, and probably also your library, algorithm, etc. (of course if you only have or know one tool then, well, 'everything looks like a nail and who needs a blueprint?').
Without planning, you'll get what grows out of the first thing that comes to mind, not the best thing you could build. The mere fact that it manages to run or stand is not a defense of the bad process.
Planning is where the real thinking takes place. Sure, if all you want to do is exercise your programming skills and avoid thinking, then don't plan. But don't pretend that no one else should think or plan, or complain when someone else presents a plan to which you are expected to build (and don't be surprised 10 years on when you don't advance as far as you expected).
Yeah but they are done before you actually start to lay any bricks.
In software you most of the time have the 2 things running parallel because of time constraints and other business needs, which almost always leads to failure.
Somewhat off-topic anecdote on this: my father in law designed a very rough draft of "blueprints" for his parents new house in rural China (Xinjiang) using MS Paint, local labourers built it in a few months and it's still standing 10 years later!
From a European perspective, I like how you say that the house is still standing 10 years later (and someone in another comment above mentioned 25 years) like it's something outstanding.
I would expect a house to last 100 years. A family member once bought a house that is over 400 years old (and still standing).
Then again, construction and houses are way more expensive in Europe/Germany. They're planning to tear down and rebuild my old high school from scratch, current estimate: 85 million euros total.
It would be interesting to see a more rigorous comparison that took into account size and what not, but here's the first Google result for "new high school building", a $65 million project:
It is very important to differentiate between House and Skyscraper. You can't build a skyscraper (currently) without blueprints. But you can, and it has been generally done, build houses without blueprints. It was even common to build cathedrals without formal blueprints, by the way.
I have built a house. You don't necessarily need professional blueprints but it's a good idea to at least have a scale drawing of your floor plan. Otherwise you won't really know how much of various building materials you will need. How many windows? What size? Where will they be positioned? Will they actually fit there?
Also from a practical standpoint you probably won't be able to get a building permit without blueprints or drawings of some sort (though this varies).
Yeah, just weird conversations about when feature X is fully ready in which sprint, features that were done getting new "improvements" and getting sprints replanned in the middle of execution.
Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
I have done both ways and still haven't found a way to getting it working across the board with all parties.
It seems that those places are not taking Scrum seriously (as much as I hate this expression)
> just weird conversations about when feature X is fully ready in which sprint,
What stories compose that feature? How are you going to plan the following sprints? (are there other stories or only those?). Prioritize.
> features that were done getting new "improvements"
Yes, you can split stories into 2, or create a new story for something that you want changed/improved. Create a story, put it through the system and do it
> and getting sprints replanned in the middle of execution.
It happens. But it shouldn't be frequent.
> Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
Well, that's not really an Agile problem, that's scope creep (and shouldn't be allowed)
It seems your problems are due to people implementing "half agile" or just using the buzzword without the important elements. Especially in small teams.
TL;DR: The real problem to solve is the time to iterate.
I think it's a straw man argument. An unfair comparison. Buildings & bridges etc have been designed and improved for literally thousands of years. Software is a half a century old.
Aza Raskin's blog piece about human powered flight which bubbled up on HN has always stuck with me[0]. "The problem is we don't understand the problem".
Once again I must refer to Naur's "Programming as theory building". Effectively programming is implementing a specific model. For the outcome at first release it does not matter how the model is documented - just that it is understood by the team. As time goes by, the more obscure the model is , the harder it is to maintain the understanding within the team.
This probably one of the reasons some agile practitioners scoff at documentation - if one is using the same framework to implement the similar website time and again effectively the model remains the same and understanding of the model is easy to retain and retrain.
"Programmers spend a lot of time thinking about how to code, and many coding methods have been proposed: test-driven development, agile programming, and so on. But if the only sorting algorithm a programmer knows is bubble sort, no such method will produce code that sorts in O(n log n) time. Nor will it turn an overly complex conception of how a program should work into simple, easy to maintain code."
I wouldn't know a bubble sort off the top of my head (I haven't had to implement anything like that since university). I would know to read up on different algorithms and choose an appropriate library based on what I was sorting. (It will probably be better tested than what I would come up with first time).
> But few programmers write even a rough sketch of what their programs will do before they start coding.
deserves a reference to a study. (or, and this is the first time I use this word: a #strawman flag.)
- or am I the only one who sketches up data models and data flows before going to work?
Also: Prototyping comes early in many other designs as well, often way before detailed blueprints are signed off I think (not a mechanical engineer, just outside observations).
In software it just so happens that prototypes are often made by the same people and from the same material as the final product, only reinforced to withstand production load and abuse.
> am I the only one who sketches up data models and data flows before going to work?
Anecdata: Yes, or rather you're one of a vast minority. I work in an office with no notebooks in the supply closet, and one 5x5 whiteboard which is usually full of UX layouts.
What I've seen before, and seems to be the case in comments here, is that beliefs about programming methodology are heavily influenced by what kind of programming a person does. Those who work on small projects (i.e. few parallel streams of development) love their agile/scrum/whatever. Those who work on larger projects are less sanguine. Likewise, front-enders also love the hipper methodologies, while back-enders tend toward the old school, independently of whether their projects are large or small.
The confounding case is the Linux kernel, which is both large and systems-focused but seems to eschew both kinds of methodology in favor of undirected chatter on mailing lists. Then again, maybe that's just part of a two-level process. More than a few patches to the kernel have real specs behind them, though developers don't call them that. They're just "kind of long, kind of detailed" emails. This tendency to embrace a thing while rejecting the term for it is one of the weirdest tendencies many developers seem to have.
I think the main thing missing in the discussion is organization or team size. Given that the author is a CompSci researcher for the past 40 years he may not be in touch with the reality of how organizations are structured and the impact of that on software development.
If it is a start-up with 5 people, the software probably is the specification. If it is a large organization of devs (25, 50 or 100 people), then you will see more documentation, some of that specifications.
There is likely a threshold in the number of people where the number of communication paths between them is too large to rely on informal ways of communication. Maybe its 15-25 developers. The larger an organization is the more bureaucracy gets introduced and you now have people whose role is to write specs and requirements docs, etc.
Off topic: my house was built without blueprints. Talked to the contractor, sketched what we wanted. He built it. Standing 25 years later, great job.
An experienced carpenter/contractor can build a house, without excruciating detailed notes. In fact they can build it faster, adapting to conditions as they change and solving problems efficiently.
66 comments
[ 5.2 ms ] story [ 2519 ms ] thread[1] https://en.wikipedia.org/wiki/Winchester_Mystery_House
https://www.youtube.com/watch?v=BKorP55Aqvg
Proof systems have their uses and then there are places where they are just too much.The best is in the mid!
>> "am the creator of the TLA+ specification language." I think TLA+ is/was being used at Amazon [1]
Safety with Preservation and Progress guarantees of critical systems demand design audits at a larger scale however if the scale of the program is actually quite small , things like type-systems themselves are sufficient proof of safety.
Ofcourse there are cases where even typing is too much!
One of the middlepaths is actually using inspection tools on code/programs as another aspect altogether.
[1]:http://research.microsoft.com/en-us/um/people/lamport/tla/fo...
The process of producing code is the process of specifying what the computer should do -- the deliverable is the specification. On the other hand, few civil engineers will start designing plumbing without at least having a sketch of the floor plan and an idea of the building uses.
I'm sure coding is "specifying what the computer should do", but that's not what people are talking about when they talk about specifications. A specification defines what the application should do. Producing code is implementing the specification. Without having any specification first you wouldn't have a clue what the code needs to do, so you wouldn't know what to write.
The problem with the analogy is that most people don't understand what a blueprint actually is. A blueprint is an architectural planning document. It defines the building in very broad terms - where the walls go, where there's a staircase, and so on. It doesn't go in to the detail of how the building is constructed. That comes from all the other documentation. As an analogy with software it's a pretty good one - a building's blueprint is equivalent to the high-level planning documentation that any big project needs.
http://paulspontifications.blogspot.co.uk/2013/01/software-h...
TL;DR: Source code is the blueprint for software.
However that doesn't mean I completely disagree with Lamport's arguments. Specifications are necessary in large-scale software because otherwise you can't know if you have actually implemented everything the program is supposed to do. When you release it the end-users get angry and insist that they told you in that email / meeting / watercooler conversation last year how important it was that the program frobnicate the warbles.
Lamport makes much of formal specifications. I agree that in some cases they do add value; trying to prove the correctness of a distributed protocol in any other way is dumb. But in my experience once you get up into application-level requirements a formal specification is basically source code you don't know how to compile; there is no formal correctness criteria against which you can verify it, and the stakeholders can't understand it in order to verify it.
As an example, imagine you're paid $1m to assess my super awesome new database. I give you the source code and tell you "the source is the blueprint." After reading through the code you discover there's no DELETE function. Have I built a clever immutable data store, or am I just an idiot who forgot a critical function? You can't tell that from the source.
Technically the source is a blueprint. But this comes down to scale.
Among other things I've worked on banking and corporate management applications. You cannot reason about the codebase at that scale. In these instances it's not possible to have a good outcome without a blueprint.
Using the building metaphor, you can get away with building a bird feeder without a blueprint. You can't get away with building a skyscraper without one. Houses, maybe. Either way, high level blueprints help everyone.
Taking the building metaphor, you can't clone a building locally and make changes to it, then ask another builder to review them, suggest some changes then push them out to a test environment before finally putting them into the real world. So for me, the building metaphor only goes so far but there are definitely some useful points in the article.
Definitely. But anyway, unless you have built several bird feeders or similar constructions before, your first version will probably be ugly and you'll have to iterate on it (i.e. find the right angles etc.) to make it better. What is of course fun, but costs you time.
But all modern systems have both structure and behavior. For every database there is a MVC model somewhere, for instance. And looking at those controllers, you should absolutely be able to tell whether the database is well-constructed or not.
Now there's another consideration: does the database structure map into some problem domain such that the introduction of new controllers will be supported? That's usually what people mean when they talk about architecture: the mapping of a problem domain over to a solution domain such that changes in the requirements have minimal impact on the solution.
In System-of-Systems work I'd argue yes, such work has some value -- as long as it is continually driving some kind of delivery (and not just "smart guys drawing boxes"). But in regular green-field system development? Not so much. That's because the tendency is to begin focusing on the wrong thing, the model instead of the user. Timeboxes help with this, so much so that I wouldn't advise any stand-alone architectural work in such an environment without team integration and time-boxing.
If I hand you a blueprint for a skyscraper and I realize there is no lock specified for the front doors. Does that mean it's a lockless skyscraper or I'm speaking to a jackass who forgot a critical function related to security? You can't tell from the blueprint.
If a specification is required to be 100% complete for all things that could possibly relate in any way, then even blueprints don't qualify.
Its an absolute nightmare when something doesn't work as expected, because rather than checking a specification that will take a couple of seconds to read, you basically have to debug software, which takes a lot longer.
In any case it's a good discussion to have in this thread, which is why I gave you the up.
So yes, it's exactly like code.
But I just wish people would stop comparing software development to construction. They're not alike at all, not even vaguely.
Programming is not like bricklaying, because bricklaying is deterministic and uncreative: there is one correct method of laying the bricks, and any two competent bricklayers will lay the bricks in exactly the same way. But there is never one single correct method of writing a program, and no two programmers will implement a feature in the same way.
Any attempt to create a specification so exhausting that it allows no deviation by individual programmers should stop pretending it's a spec and just write the damn code. In fact, the author of the article should probably stop writing specs and just iterate prototypes to achieve the same end: there's no difference between "understanding our programming task" in formal English or understanding it in code, except that the code is more useful and a lot easier to test.
I am no architect, so all I think about it can be true or false. But what I remember from an architect friend is that an architect's work is not really that detailed. It's like the photoshop mockup of a website, not like its source code. It doesn't say exactly which materials should be chosen or how many screws should be used to connect two ends. An architects design says "There is a wall x cm thick and y cm long". The source code says "There z bricks and the last one we had to cut to n cm" (sometimes a comment will state why it had to be cut). For the source code it is to some degree by accident the amount of bricks creates something that can be called a wall.
But all the information about the correct numbers of bricks, the slump of the mortar, the number and type of screws - these are all just library functions. Why would you re-write all that perfectly serviceable code for each project?
But I digress. Here is how bricklayers and programmers are similar: they commonly work in the vacuum of their own workspace, without awareness of the environment that impacts their projects. So once the projects are launched, they're impacted by things that weren't accounted for.
If the foundation is too weak, the brick wall will sag. If nearby trees grow too close to the wall, the wall will bow inward or outward. If someone builds on top of the brick wall and doesn't reinforce it over 8 feet, the inspector will condemn the work. The bricklayers are paid to just lay bricks, so they won't care about the nearby trees, they won't care about the foundation, they won't care about what you put on top of the wall.
Programmers will build the project to spec, and come launch time, not account for a particular load signature because that was never their concern. Seen that time and time again. Substitute foundation for baseline server power, trees for amount of interrupting processes and meddling managers altering the original program at random, and you have a pretty reasonable comparison to bricklaying.
The problem here is not that you have insufficiently detailed specs, but that you're treating your programmers like bricklayers.
I strongly suggest you try bringing your programmers into the room during the specification process, even get them to write some prototype code to explore the tricky areas (something bricklayers don't get to do either). You'll probably find you get better code as a result because the people writing the code are aware of the environment the code will be deployed to.
* Would you bring bricklayers to the design meeting? No. Just the foreman. * Would you bring all programmers to the design meeting? No. Just the lead developer.
Yes, ideally you'd have all the programmers in the room, but more of them you introduce, you increase the communication overhead. And some projects involve dozens to hundreds to thousands of programmers and are complicated enough to not be able to bring everyone to the discussion table.
So what you have to do is use bricklayer programmers- trust them to do a single task right (warehouse uploaded photos, talk to APIs) but recognize their limits. I'm not faulting neither bricklayers nor programmers, but defining their borders more concisely.
I've never met a business problem in my 30+ years doing commercial IT that couldn't be split up into smaller problems with smaller solutions, but there seems to be a bias towards large teams and complex solutions in commercial IT.
I don't know what causes this (I suspect management ego or consultancy fees, the cause of most commercial IT problems), but it's a real problem. The vast majority of these huge projects fail, go massively over-budget, precisely because they're being managed wrong.
Programming is fundamentally not like bricklaying, because it's creative not deterministic. Better, more detailed and more complete specs won't make more successful projects.
If you have too many programmers to bring them into the room when you're specifying the problem, then your project team is too big. The answer is not to exclude programmers and create more detailed specs, because programmers working to a spec like that won't write good code.
I know, I've been there. One project in particular stands out. We had detailed specs for a range (a few hundred) of middle-layer business routines. I received a function spec, wrote the function, wrote a few tests that the function did what it should do, ticked the box. Rinse and repeat for months. There were people working in a similar fashion on the front end and database layer. The system went over-budget, never worked, and was finally scrapped. It's not hard to see why, when the entire system was being written in this fashion. Programming is not like bricklaying.
If you think about it for 5 minutes you'll see that "bringing more programmers into the room increases communication overhead" implies that there is some information that is required to be passed from the customer to the programmer. The solution to this problem is obviously not to exclude the programmer and reduce the information passed to them, because that will lead to inferior code because the programmer is missing information. Instead reduce the amount of information the team is dealing with by segmenting the problem better.
But blueprints are done at various scales and they differ in complexity and format ie. sketches, models, schedules. They also often come with a book of specifications, and these are typically only referred to when ordering parts or if something goes wrong - and they are always open to interpretation..
...and the 2x4s are the blueprint of a house and the steel beams are the blueprint of the skyscraper....
I'll take the TL;DR at face value since it was written by the author. The statement completely obliterates the entire concept of a blueprint or plan. By that reasoning, it is true everything is its own blueprint, and the concept has no meaning (of course this is a standard tautology).
And, simply nodding to CYA documents (for when the customers/end users come back saying that your product doesn't do X), or to docs for when formal proofs are needed, doesn't cover the concept.
The point of planning (or blueprinting) is to think about what you are going to build before you get out the tools, abstracted away from the specific tools that you'll use, and to then record and communicate those thoughts to all the stakeholders. It doesn't have to be a complete formal, engraved-in-stone blueprint, but a modest amount of actual thinking and planning goes a loooong way.
I've often found that with a little thought, we'll use a different tool than originally expected. If you expect to just dive in and sort it out in your source code, then you've already a priori restricted your choice of tool, and probably also your library, algorithm, etc. (of course if you only have or know one tool then, well, 'everything looks like a nail and who needs a blueprint?').
Without planning, you'll get what grows out of the first thing that comes to mind, not the best thing you could build. The mere fact that it manages to run or stand is not a defense of the bad process.
Planning is where the real thinking takes place. Sure, if all you want to do is exercise your programming skills and avoid thinking, then don't plan. But don't pretend that no one else should think or plan, or complain when someone else presents a plan to which you are expected to build (and don't be surprised 10 years on when you don't advance as far as you expected).
http://delivery.acm.org/10.1145/2740000/2736348/p38-lamport....
http://cacm.acm.org/magazines/2015/4/184705-who-builds-a-hou...
In software you most of the time have the 2 things running parallel because of time constraints and other business needs, which almost always leads to failure.
I would expect a house to last 100 years. A family member once bought a house that is over 400 years old (and still standing).
Then again, construction and houses are way more expensive in Europe/Germany. They're planning to tear down and rebuild my old high school from scratch, current estimate: 85 million euros total.
http://www.monomoy.edu/page.cfm?p=931
(A regional high school school for about 700 students, 168,000 square feet)
You can apply the same metaphors to software ;)
Also from a practical standpoint you probably won't be able to get a building permit without blueprints or drawings of some sort (though this varies).
Thanks but no thanks
I don't love it, but it works (better than the alternative)
And we don't have any more weird conversations about trying to get the entire software, all the tasks, in one sitting.
Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
I have done both ways and still haven't found a way to getting it working across the board with all parties.
> just weird conversations about when feature X is fully ready in which sprint,
What stories compose that feature? How are you going to plan the following sprints? (are there other stories or only those?). Prioritize.
> features that were done getting new "improvements"
Yes, you can split stories into 2, or create a new story for something that you want changed/improved. Create a story, put it through the system and do it
> and getting sprints replanned in the middle of execution.
It happens. But it shouldn't be frequent.
> Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
Well, that's not really an Agile problem, that's scope creep (and shouldn't be allowed)
It seems your problems are due to people implementing "half agile" or just using the buzzword without the important elements. Especially in small teams.
(been there, done that)
My point is that I have never seen any company doing agile as their founders created it.
Trying to push for agile way since 2006.
But there are some places that managed to "get it".
I think it's a straw man argument. An unfair comparison. Buildings & bridges etc have been designed and improved for literally thousands of years. Software is a half a century old.
Aza Raskin's blog piece about human powered flight which bubbled up on HN has always stuck with me[0]. "The problem is we don't understand the problem".
[0]: http://www.azarask.in/blog/post/the-wrong-problem/
This probably one of the reasons some agile practitioners scoff at documentation - if one is using the same framework to implement the similar website time and again effectively the model remains the same and understanding of the model is easy to retain and retrain.
This.
I would take ABNF as used in RFC:s and a few clarifying diagrams over verbose UML vomit any day :)
For anything of scale we really need blueprints.
> But few programmers write even a rough sketch of what their programs will do before they start coding.
deserves a reference to a study. (or, and this is the first time I use this word: a #strawman flag.)
- or am I the only one who sketches up data models and data flows before going to work?
Also: Prototyping comes early in many other designs as well, often way before detailed blueprints are signed off I think (not a mechanical engineer, just outside observations).
In software it just so happens that prototypes are often made by the same people and from the same material as the final product, only reinforced to withstand production load and abuse.
Anecdata: Yes, or rather you're one of a vast minority. I work in an office with no notebooks in the supply closet, and one 5x5 whiteboard which is usually full of UX layouts.
The confounding case is the Linux kernel, which is both large and systems-focused but seems to eschew both kinds of methodology in favor of undirected chatter on mailing lists. Then again, maybe that's just part of a two-level process. More than a few patches to the kernel have real specs behind them, though developers don't call them that. They're just "kind of long, kind of detailed" emails. This tendency to embrace a thing while rejecting the term for it is one of the weirdest tendencies many developers seem to have.
If it is a start-up with 5 people, the software probably is the specification. If it is a large organization of devs (25, 50 or 100 people), then you will see more documentation, some of that specifications.
There is likely a threshold in the number of people where the number of communication paths between them is too large to rely on informal ways of communication. Maybe its 15-25 developers. The larger an organization is the more bureaucracy gets introduced and you now have people whose role is to write specs and requirements docs, etc.
An experienced carpenter/contractor can build a house, without excruciating detailed notes. In fact they can build it faster, adapting to conditions as they change and solving problems efficiently.