"1. The product is only as good as the plan for the product."
Right up there with Steven Covey's "Begin with the end in mind". I've witnessed it again and again - when agile developers/hackers/programmers grow up they end up doing design up front. Fail fast and often has it's place, but that place is finite.
There's probably more to it than that. Most other teams of developers, given the same amount of time and money, could probably not write such complex software with so few bugs in it.
The road to hell is paved with good intentions. The context in which the space shuttle program is developed has almost no relation to what most developers face in the software world at large, a fact that the article admits toward the end.
Lets see:
* 35 million dollar budget a year.
* developing software that only has "one job", regardless of how hard that job is.
* In other words, no evolving software to meet the demands of an evolving world. Fixed requirements 'forever' (as much as forever can be in software terms). No one at NASA is going "hey, I know we developed this for earth orbit but lets alter the design and try use 90% of the same code to go to the moon as well earth orbit. We'll save time and money." (Unlike SpaceX to some extent)
* irreversible decisions. safety critical etc.
* Due to lavish budget and safety critical risks, reduced schedule pressure. No one wants to be on the other end of a independent report concluding that long hours resulted in sloppy code that got astronauts killed.
The above often even doesn't apply to most relatively expensive (in the millions) non-safety critical military and government software. Watch the Pentagon Wars for a hilarious illustration that comes closer to the reality that software engineers face in those environments.
Its a scandal that space shuttle software is being used to push pseudo-scientific process ideology. Its often claimed that SEI developed CMMI process using the space shutte software process as a blueprint. CGI Federal was CMMI Level 5 on paper and QSSI was CMMI Level 3. Meaning that the companies responsible for the most public software scandal in recent US history that is healthcare.gov wasted something like $300 mil at t0 writing the "wrong stuff" while claiming to do it "the right way".
Enforcing abstract processes instead of dealing with case-to-case facts reminds me of this quote:
"We are all capable of believing things which we know to be untrue, and then, when we are finally proved wrong, impudently twisting the facts so as to show that we were right. Intellectually, it is possible to carry on this process for an indefinite time: the only check on it is that sooner or later a false belief bumps up against solid reality, usually on a battlefield." --George Orwell
Contrast all this with Elon Musk's thoughts on Process:
"Now I have to tell you something, and I mean this in the best and most inoffensive way possible: I don’t believe in process. In fact, when I interview a potential employee and he or she says that “it’s all about the process,” I see that as a bad sign.The problem is that at a lot of big companies, process becomes a substitute for thinking. You’re encouraged to behave like a little gear in a complex machine. Frankly, it allows you to keep people who aren’t that smart, who aren’t that creative."
>when agile developers/hackers/programmers grow up they end up doing design up front.
er, what? The agile push came people who were reacting against creating pages of useless design up front documentation that was Dead On Arrival as soon as it was written. People in general become more capable in design up front as their knowledge of the domain grows, but that too depends on context. I'm not sure how valuable design up front would be to even an experienced software engineer who is faced with a project in a programming language he hasn't worked with and a customer domain and platform he doesn't know....
"The agile push came people who were reacting against creating pages of useless design up front documentation"
er, what? The agile push came from people who saw an opportunity to make money [1]. To understand that you need to understand both the advantages of waterfall, it's disadvantages, and how those disadvantages were mitigated. Documentation is but a single (and pretty minor) aspect of it.
The biggest single issue with waterfall is also is biggest advantage - segmentation (what Barry Boehm called segments or stages of a project). Stages follow each other sequentially. Requirements analysis follows inception, design follows requirements analysis, development follows design, test development, and so forth.
The advantage is that it is predictable (ironically, agile's biggest disadvantage). It's disadvantage is that it's crazy expensive to reverse the sequence, i.e. go from test back to dev, from dev back to design, and so on.
The mitigations back in the early 80's were to prototype (fail fast) and to turn a single protracted release into many smaller releases (what became known as the spiral model) - aka fail often. In amongst all of that TDD happened, and someone decided to turf documentation because everyone hates doing it anyway.
eXtreme programming came along and gave protyping and iterative releases a shiny new badge. Pair programming wasn't for everyone, so someone removed that, added daily standups and gave it a shiny old name (scrum was first applied to product development in the early 80's and formally documented in '86[2] - it was nothing like what it is today).
Oh, and modern agile processes contract release cycles even more.
Note that there are other intrinsic (requirements almost always missing/incomplete, stages produce communication gaps, and as you mention, documentation) and extrinsic issues (it's a pm methodology, not a dev methodology) with waterfall.
"The agile push came people who were reacting against creating pages of useless design up front documentation"
er, what? The agile push came from people who saw an opportunity to make money [1].
There are only four items in the Agile Manifesto. The second item specifically highlights documentation cost overhead of traditional methods as a major grievance: "Working software over comprehensive documentation" (see also [1]). I don't think money is a differentiating factor, we're talking about the software business in the end after all. If you're referring to process snakeoil marketing, I'm against that no matter which side is doing it. I was responding to your assertion that software developers go from "agile" to "design up front" as they "grow up". My view is that the people that began popularizing the agile term were in fact "grown up" programmers. Are we talking about different things when we talk about design? What is the output of "design up front" in the traditional sense if not documentation and diagrams? (for example, see the military standards and ISO standards for what's required for detailed design)
I don't see anything objectionable in the descriptions you give, although I don't view the waterfall process as being predictable in any meaningful sense. Its only predictable in so far as its underlying assumptions hold. And in the software world at large such assumptions (fixed unchanging requirements for example) rarely hold. My claim is that circumstances for the space shuttle software is the exception, not the rule.
Seems we're not really disagreeing about the reality. We do seem to have different... interpretations, maybe?
"The second item specifically highlights documentation cost overhead of traditional methods as a major grievance"
Comprehensive documentation may be a major grievance, but it is pales in actual impact on the organisation, team or project when compared with the extortionate cost of discovering a design error during the test phase.
"Are we talking about different things when we talk about design?"
Two answers. First one is yes, I think we're talking about the same thing. Design in the sense that I use it is system design, or architecture. And yes, in a waterfall world that output typically is documentation + diagrams.
Second answer? Design how I do it doesn't involve documentation at epic war-and-peace proportions. A diagram can be very useful when communicating complex designs to external (to the team) stakeholders. A one-pager with VLAN descriptions and IP addresses can be helpful to a pen tester. However, in my world it's about a plan -
- What's the availability target, and how will it be met?
- Is the database relational, document-based, or are we using a lambda architecture? Justification?
- What's the dev plan? I.e. what feature dependencies exist, and how will they be prioritised? Consider for example a client app that communicates with a USB device. Initially a simple web site might be enough, but ultimately you need to choose between a native app; a daemon that bridges USB and browser via HTML websocks, or a daemon that bridges USB and browser via web site.
"although I don't view the waterfall process as being predictable"
It is extremely predictable in that phases are sequential, and you know exactly which skills are required for each phase. You'd use a business analyst to elicit requirements, and you'd not use a tester to write code. That means you can schedule resources across projects horizontally, instead of leaving them on one project until complete.
You mention "in any meaningful sense", and you're very, very correct to include that. The point is that waterfall is just as efficient at delivering quality code quickly as any modern agile methodology is. What's required is an experienced team.
Consider that an inexperienced team will screw up on an agile project just as much as they would on a waterfall project. Here's an analogy (taken from a blog post I'm still working on):
--
Karate is a martial art. It is also a competitive sport, and has strict rules. Krav Maga is street fighting. Anything goes - no rules, no competitions. I've been doing Krav Maga for over four years, and think it's the best self defence system known to man.
Saying that Krav Maga is the best is interesting. Not because it really is the best, but because that statement demonstrates two important things:
- It demonstrates consistency - my desire to be (and appear to be) consistent with what I have already done. Once we've made a choice, we encounter personal and interpersonal pressures to behave consistently with that commitment.
- It ignores reality - my girlfriend, a 3rd Dan black belt, has been doing Karate for over 18 years. Whether Krav Maga is better than Karate on not is immaterial - she'd kick my arse from here into next week. The reality is that she's simply more experienced. No matter how brutal a counter attack in Krav Maga is, she's so fast that there's no way I can even block her attack (I know because we spar).
--
It's that concept of experience that I refer to when talking about devs/hackers/programmers growing up.
As devs/hackers/programmers I think it serves us well to understand both extremes (waterfall and agile), and move the slider between the two as required. The mix should be appropriate for the team's experience. Enough prescriptiveness from waterfall to coach wher...
10 comments
[ 2.8 ms ] story [ 30.7 ms ] threadRight up there with Steven Covey's "Begin with the end in mind". I've witnessed it again and again - when agile developers/hackers/programmers grow up they end up doing design up front. Fail fast and often has it's place, but that place is finite.
For anyone wanting to dig a bit deeper, check out the NASA Software Safety Guidebook [http://www.hq.nasa.gov/office/codeq/doctree/871913.pdf]
No, they write software this good because they have enough time (e.g. money) to write quality software.
Lets see:
* 35 million dollar budget a year.
* developing software that only has "one job", regardless of how hard that job is.
* In other words, no evolving software to meet the demands of an evolving world. Fixed requirements 'forever' (as much as forever can be in software terms). No one at NASA is going "hey, I know we developed this for earth orbit but lets alter the design and try use 90% of the same code to go to the moon as well earth orbit. We'll save time and money." (Unlike SpaceX to some extent)
* irreversible decisions. safety critical etc.
* Due to lavish budget and safety critical risks, reduced schedule pressure. No one wants to be on the other end of a independent report concluding that long hours resulted in sloppy code that got astronauts killed.
The above often even doesn't apply to most relatively expensive (in the millions) non-safety critical military and government software. Watch the Pentagon Wars for a hilarious illustration that comes closer to the reality that software engineers face in those environments.
Its a scandal that space shuttle software is being used to push pseudo-scientific process ideology. Its often claimed that SEI developed CMMI process using the space shutte software process as a blueprint. CGI Federal was CMMI Level 5 on paper and QSSI was CMMI Level 3. Meaning that the companies responsible for the most public software scandal in recent US history that is healthcare.gov wasted something like $300 mil at t0 writing the "wrong stuff" while claiming to do it "the right way".
Enforcing abstract processes instead of dealing with case-to-case facts reminds me of this quote:
"We are all capable of believing things which we know to be untrue, and then, when we are finally proved wrong, impudently twisting the facts so as to show that we were right. Intellectually, it is possible to carry on this process for an indefinite time: the only check on it is that sooner or later a false belief bumps up against solid reality, usually on a battlefield." --George Orwell
Contrast all this with Elon Musk's thoughts on Process:
"Now I have to tell you something, and I mean this in the best and most inoffensive way possible: I don’t believe in process. In fact, when I interview a potential employee and he or she says that “it’s all about the process,” I see that as a bad sign.The problem is that at a lot of big companies, process becomes a substitute for thinking. You’re encouraged to behave like a little gear in a complex machine. Frankly, it allows you to keep people who aren’t that smart, who aren’t that creative."
>when agile developers/hackers/programmers grow up they end up doing design up front.
er, what? The agile push came people who were reacting against creating pages of useless design up front documentation that was Dead On Arrival as soon as it was written. People in general become more capable in design up front as their knowledge of the domain grows, but that too depends on context. I'm not sure how valuable design up front would be to even an experienced software engineer who is faced with a project in a programming language he hasn't worked with and a customer domain and platform he doesn't know....
related, SpaceX Lessons Learned: http://lwn.net/Articles/540368/
er, what? The agile push came from people who saw an opportunity to make money [1]. To understand that you need to understand both the advantages of waterfall, it's disadvantages, and how those disadvantages were mitigated. Documentation is but a single (and pretty minor) aspect of it.
The biggest single issue with waterfall is also is biggest advantage - segmentation (what Barry Boehm called segments or stages of a project). Stages follow each other sequentially. Requirements analysis follows inception, design follows requirements analysis, development follows design, test development, and so forth.
The advantage is that it is predictable (ironically, agile's biggest disadvantage). It's disadvantage is that it's crazy expensive to reverse the sequence, i.e. go from test back to dev, from dev back to design, and so on.
The mitigations back in the early 80's were to prototype (fail fast) and to turn a single protracted release into many smaller releases (what became known as the spiral model) - aka fail often. In amongst all of that TDD happened, and someone decided to turf documentation because everyone hates doing it anyway.
eXtreme programming came along and gave protyping and iterative releases a shiny new badge. Pair programming wasn't for everyone, so someone removed that, added daily standups and gave it a shiny old name (scrum was first applied to product development in the early 80's and formally documented in '86[2] - it was nothing like what it is today).
Oh, and modern agile processes contract release cycles even more.
Note that there are other intrinsic (requirements almost always missing/incomplete, stages produce communication gaps, and as you mention, documentation) and extrinsic issues (it's a pm methodology, not a dev methodology) with waterfall.
[1] http://www.infoq.com/articles/agile-teenage-crisis
[2] https://www.wittenburg.co.uk/Entry.aspx?id=dce9dde8-d770-47c...
er, what? The agile push came from people who saw an opportunity to make money [1].
There are only four items in the Agile Manifesto. The second item specifically highlights documentation cost overhead of traditional methods as a major grievance: "Working software over comprehensive documentation" (see also [1]). I don't think money is a differentiating factor, we're talking about the software business in the end after all. If you're referring to process snakeoil marketing, I'm against that no matter which side is doing it. I was responding to your assertion that software developers go from "agile" to "design up front" as they "grow up". My view is that the people that began popularizing the agile term were in fact "grown up" programmers. Are we talking about different things when we talk about design? What is the output of "design up front" in the traditional sense if not documentation and diagrams? (for example, see the military standards and ISO standards for what's required for detailed design)
I don't see anything objectionable in the descriptions you give, although I don't view the waterfall process as being predictable in any meaningful sense. Its only predictable in so far as its underlying assumptions hold. And in the software world at large such assumptions (fixed unchanging requirements for example) rarely hold. My claim is that circumstances for the space shuttle software is the exception, not the rule.
[1] http://davidfrico.com/rico08e.pdf
"The second item specifically highlights documentation cost overhead of traditional methods as a major grievance"
Comprehensive documentation may be a major grievance, but it is pales in actual impact on the organisation, team or project when compared with the extortionate cost of discovering a design error during the test phase.
"Are we talking about different things when we talk about design?"
Two answers. First one is yes, I think we're talking about the same thing. Design in the sense that I use it is system design, or architecture. And yes, in a waterfall world that output typically is documentation + diagrams.
Second answer? Design how I do it doesn't involve documentation at epic war-and-peace proportions. A diagram can be very useful when communicating complex designs to external (to the team) stakeholders. A one-pager with VLAN descriptions and IP addresses can be helpful to a pen tester. However, in my world it's about a plan -
- What's the availability target, and how will it be met?
- Is the database relational, document-based, or are we using a lambda architecture? Justification?
- What's the dev plan? I.e. what feature dependencies exist, and how will they be prioritised? Consider for example a client app that communicates with a USB device. Initially a simple web site might be enough, but ultimately you need to choose between a native app; a daemon that bridges USB and browser via HTML websocks, or a daemon that bridges USB and browser via web site.
"although I don't view the waterfall process as being predictable"
It is extremely predictable in that phases are sequential, and you know exactly which skills are required for each phase. You'd use a business analyst to elicit requirements, and you'd not use a tester to write code. That means you can schedule resources across projects horizontally, instead of leaving them on one project until complete.
You mention "in any meaningful sense", and you're very, very correct to include that. The point is that waterfall is just as efficient at delivering quality code quickly as any modern agile methodology is. What's required is an experienced team.
Consider that an inexperienced team will screw up on an agile project just as much as they would on a waterfall project. Here's an analogy (taken from a blog post I'm still working on):
--
Karate is a martial art. It is also a competitive sport, and has strict rules. Krav Maga is street fighting. Anything goes - no rules, no competitions. I've been doing Krav Maga for over four years, and think it's the best self defence system known to man.
Saying that Krav Maga is the best is interesting. Not because it really is the best, but because that statement demonstrates two important things:
- It demonstrates consistency - my desire to be (and appear to be) consistent with what I have already done. Once we've made a choice, we encounter personal and interpersonal pressures to behave consistently with that commitment.
- It ignores reality - my girlfriend, a 3rd Dan black belt, has been doing Karate for over 18 years. Whether Krav Maga is better than Karate on not is immaterial - she'd kick my arse from here into next week. The reality is that she's simply more experienced. No matter how brutal a counter attack in Krav Maga is, she's so fast that there's no way I can even block her attack (I know because we spar).
--
It's that concept of experience that I refer to when talking about devs/hackers/programmers growing up.
As devs/hackers/programmers I think it serves us well to understand both extremes (waterfall and agile), and move the slider between the two as required. The mix should be appropriate for the team's experience. Enough prescriptiveness from waterfall to coach wher...