Agile involves getting the customer deeply involved from the start, so that after one or two spins of cycle, they have a more sober view of what the magicians are doing, and how little they, the customer, get to ask for in the real world, if they want the project to be done this century (and they do.) Done well, it's the equivalent of sticking your dog's nose into the mess they made on your rug; where dog=client.
The issue is one of buy-in. For it to work, everyone involved, client, management, development, qa, design, everyone has to be on board with the agile way of doing things. Without that, you have development working in the agile manner, believing in good faith that everyone else is too, and then the second a snag gets hit, then that all goes out the window, and development is usually the one left holding the bag and punished with overtime.
Right. Sturgeon's law, I guess. Agile Theatre is more common than Agile, just as security theatre is more common than security. So I'd say both above comments are accurate.
Now, each iteration offers not only the chance for project management to be exercised, but more specifically, to ensure that it is exercised by capturing the appropriate, timely information from all the appropriate stakeholders.
The magic of “agile” is that partnership with the customer and shared accountability.
That’s difficult to codify, especially in contracts for delivery. “Agile methods” more often than not translate into hiding a complete lack of planning in a bunch of mumbo jumbo.
The point is that there are always stakeholders and product owners - it just makes that explicit. If the PO is acting as a proxy for the stakeholder (customer in this case) that's ok.
Of course the PO might be wrong... but with small iterations that should be found out quickly and adjusted for.
It would be odd if "business people" just meant the developers' upper management within their firm; supervising costs and parking spaces; but not those without a stake in using the software that results, say.
Business people doesn’t mean customers however. Which was my point.
In fact the only time the customer is mentioned is in principle one: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
Business people absolutely should be involved - product owners & stakeholders (or a proxy for them).
That’s fundamental but again: business people != customers.
You seem to think stakeholders/product owners are devs direct managers too. That’s rarely the case, if ever, on any functioning agile team I’ve seen. They’re orthogonal to each other.
I think I answered this possible objection, but you've ignored that. Also, I made a distinction between stakeholders and devs direct managers, rather than conflating them.
I don't believe I can be of further assistance to you.
HN title is clickbait. Article title is much better.
The article makes a very reasonable argument, however I wish he wouldn't try to introduce some new term, Agile "governance". He's just describing what we already know -- that Agile isn't Agile and care must be taken to institute it correctly. via "governance" as he calls it but this isn't a new revelation.
The entire thing is a thought experiment; of course it's easy to think up nightmare scenarios for Agile development. It would be much more interesting to read about actual disasters.
Its a hypothetical that I have experienced on several projects now, project managers can have a totally warped understanding of agile in that they think they can change course at any point and it won't have any resulting costs in either time or complexity.
One thing I've learned over the years about process is that a "bad" process that is followed consistently and with buy-in from everyone involved will always out perform the "good" process that is followed haphazardly or with only lip service buy-in.
I liken this to diets. Paleo suggests not eating bread (among other things). However, not eating bread doesn't mean you are doing Paleo. The benefits of Paleo are intended to be realized via a systemic application.
So whether it is agile or diets, make rules that work for you (and yours). Then stick to them. Periodically review if they are working. If they aren't change the part that isn't into a new rule and then repeat. There is no one size fits all in these things.
Some smart, well-meaning people write down a couple of brief, broad suggestions on how to do better, and a few years later it's been twisted into an industry of people who want me to pay them for telling me that I'm living my life wrong.
Yes, and much like a religion, there are people who refuse to see any positive elements to it, despite those elements being numerous and obvious.
Also, there are larger truths that live at its heart which make discussions about the topic difficult because if you're seen as disagreeing with some of it, you're seen as disagreeing with all of it, even if you don't.
If you ask the home run king how to hit more home runs, you'll get a lot of advice. Some of it may actually be accurate.
But I believe that if you really want to know how to hit home runs, you want to talk to the king's trainer, not the king. The king has no perspective. He has mostly only worked with himself. To put it bluntly: he's too wound up in his own bullshit to be objective.
Software Developers don't have trainers. Ask a successful person how they got that way and they have a sample size of one, or a handful. All you'll get is speculation.
I liked this article until it got into the charts and didn't really outline what the governance points actually entail in terms of work. I'm also missing how the bottom part actually gets you closer to the goal at the end unless you explicitly make setting that goal part of your governance / planning / project start which wasn't really mentioned.
I gather from the article's initial paragraphs that these "Governance" point is really the team getting input from all the normal stakeholders and creating the backlog of work themselves? This makes all the sense in the world - getting technical folks involved in discussions is key.
What I would like to see added are concrete steps / workflows that these technical teams have found that help make these governance points effective without eating up a lot of time, and what practices they used to ensure that the project goal / direction / outcomes were decided upon up front.
I personally think Agile Roadmapping / Storymapping with the whole group can go a long way toward ensuring whatever you build is going in the right direction. From there, you'll want the team to work with the product owner folks and keep a tighter grip on the backlog toward that direction and good things can happen.
More generally : for any given X and Y, there will be enough instances of X being better than Y and Y being better than X to feed both sides of internet debates
> "The worst thing about estimates is that they push a company in the direction of doing work that’s estimable. This causes programmers to favor the low-yield, easy stuff that the business doesn’t actually need but that is 'safe'".
> "Good engineers want to work in engineer-driven firms where they will be calling shots regarding what gets worked on, without having to justify themselves to 'scrum masters' and 'product owners' and layers of non-technical management"
> "Scrum [is] tailored toward the body shops where client relationships are so mismanaged"
> "Open-plan offices are the most egregious example. They aren’t productive"
> "[Scrum is] Like a failed communist state that equalizes by spreading poverty"
There is more. In "pure scrum" it's not obvious how to account for refactoring, developer experimentation, improving infrastructure (or creating one from scratch!), making tests better. Product driven things - sure, can be driven by scrum. But what to do when work is driven by engineering?
Right, Agile is for outsourced/offshore teams cranking out intranets or shopping carts or adding another minor feature to an existing codebase written by someone else. Like maintenance on a legacy app.
The kind of work that good engineers thrive on - a hard problem starting with en empty Emacs buffer - is something that the methodology merchants have never experienced themselves. The only product they ever shipped is the methodology itself!
I say this as a one-time “certified scrum master”.
This is a pretty tough statement to defend. Take a look at the myriad of open source projects, or just limit it to Apache’s http://apache.org/index.html#projects-list and you’ll see a bunch of “empty emacs buffer” projects with a long audit trail of agile development. Not necessarily scrum.
The article foundation is fair: "Agile tools provide a steering mechanism, not a homing device". But I'm not sure what point it's trying to make after that? "Use an Agile framework"? Just "be aware of it"?
"At least in Waterfall if you did a good job of designing the project, and if you stuck to the spirit of the method, you were to some extent insulated from management noise..."
I think it's an interesting article, with good points, but fundamentally I have to disagree with this statement. Empirically, it just isn't true, in my experience. It just means that all those changes by the customer to what they wanted, that the devs object to and management (which has to meet the customer face to face) always in the end agrees to, happen at the end, when they are more difficult, because up until that point the project wasn't "real" to the client, and they really couldn't tell you what they wanted until they saw that what you built wasn't it.
People constantly confuse agile development with software design. Agile is not, and was never intended to be, a way to design software. It is a way to implement software. How you design it is up to you, though agile certainly encourages you to involve all stakeholders from the beginning, and to make design changes incrementally throughout development as circumstances warrant. But it is not a design methodology.
This is the same extreme reaction people had previously to waterfall. Any methodology fails if it gets applied blindly and wrongly. You have to be willing to adapt it to your specific environment until it works for you. It would be a huge mistake to go back to waterfall just because people perverted Agile into a micromanagement method. The same people who can't make Agile work will also fail (and have failed for a long time) with waterfall or any other method because they refuse to adapt to reality and instead want work instructions they can follow without any thought.
The same thing goes on with OOP. It started out as a good addition to other programming styles, got turned into a religion ("you can't do this. It's not OOP.") and now into a pariah (FP is next to go through this cycle).
In the end stupid people will mess up anything they touch and good people will adapt things until they work.
> This is the same extreme reaction people had previously to waterfall
I mean... there's no correct application of waterfall. The whole waterfall process comes from a paper as an example of how not to run an engineering project.
I have seen projects where there was a very good specification upfront and the developers pretty much just had to work through it. This is not always possible and requires a very good understanding of the problem but it can work really well.
It comes down essentially to that understanding. For well-understood problems waterfall might even be the best possible method.
It's only for not well understood problems that it fails so completely. In software, in my opinion, fewer problems are well understood at the moment, and so we tend to avoid waterfall.
However, if someone wanted me to set up a webserver to serve static files that don't change? I'm 99% sure I could set up nginx on a virtual machine "fully waterfall" and have no problems.
The same is true for many other real life "waterfall" processes, like buildings, or bridges.
actually buildings and bridges aren't really waterfall the way they are designed. They have similar issues with changing requirements, new insights etc as the project progresses. The actual construction is not really the equivalent of writing code in this world.
> In software, in my opinion, fewer problems are well understood at the moment, and so we tend to avoid waterfall.
Or if the problem is that well-understood, there's an off-the-shelf solution so good nobody seriously thinks about re-implementing it.
> However, if someone wanted me to set up a webserver to serve static files that don't change? I'm 99% sure I could set up nginx on a virtual machine "fully waterfall" and have no problems.
Exactly. That's barely software development. That's using fully-commoditized components in well-understood fashions to achieve a well-understood goal.
Just because you either lucked out, or didn't have the testing to find what actually broke, doesn't make waterfall a good model.
The most well defined piece of software I know of, sel4, had a full machine checkable formal proof up front, and they still ran into issues during implementation that required them to go back and modify the spec.
You have a very extreme definition of waterfall. Waterfall doesn't mean that the spec is absolutely immutable. You will always find things to change in any effort of some complexity.
It's literally named waterfall in order to reference the idea that each step becomes immutable as it's being passed to the next step. This isn't some unfortunate consequence of an overstretched metaphor; it's literally the core concept.
It's not my fault that it appears you've been using the term wrong. Waterfall with feedback ceases to be waterfall.
I mean... there's no correct application of waterfall. The whole waterfall process comes from a paper as an example of how not to run an engineering project
That is a very strange thing to say considering that the method comes from "real" branches of engineering (mech, civil, electrical, etc) and you can simply look around you at your built environment to know that it has delivered an awful lot of real, actual stuff. They didn't agile up the building you're sitting in or the roads that lead to it or power station that makes it work or any of the things in it.
Have you worked in those fields? I worked as a commercial interiors contractor, then worked in embedded systems doing (among other things) board design and layout. These fields have plenty of iteration that has to occur. Pretty much no one ships a rev A board, chip, or bridge (the bridges get built out in solidworks).
agile is a culture as much as it is a project management methodology, so it only works when the culture is compatible.
one former employer purportedly did agile but it was a top-down autocracy, which is completely incompatible with agile. (i was ostensibly a product owner, but the CEO set our development priorities with only superficial consideration for customer feedback.)
you cannot both dictate and empower at the same time, and empowerment (via agile) is typically the loser in that fight.
that's how i see agile "failing" most often, before it even has a chance.
The perfect development system will be immune to failure before it's had a chance.
There is no perfect, but one that is fragile to external business realities is probably worse than one that is resilient to it.
(In my company, the problem with agile was that the sales people would insist on always adding some extra tool for a fixed price. We had no choice but to abandon agile.)
i don’t know what such a resilient methodology might be, because we’re talking about human systems where the participants continuously adapt to emerging conditions.
agile seeks to benefit from the intrinsic motivation of each team member to create something beyond any single member’s capabilities in exchange for empowerment.
when a manager doesn’t understand this principle, that leads to failure (as in your case as well). i don’t know of any systems that can self-correct for this kind of external mis-judgement.
That's the holy grail of management: To create systems that are immune to mismanagement. The Agile Manifesto tried to make clear that such systems don't exist but instead it got made into another system that's supposed to be immune from clueless people.
> agile is a culture as much as it is a project management methodology,
Agile is a culture about project management methodologies, which unfortunately is extremely resistant to actually adopting methodology that reflects it's overt cultural values. Largely, IMO, this is because the agile community has been very bad at operationalizing the “over” statements in the Manifesto into mechanisms by which the right side items are subordinated to the left side items. This results in (for the first statement in the Manifesto) either dysfunctional interaction and demotivated people from constant conflict resulting from the absence of processes and tools, or the adoption processes and tools in a manner which does not respect people and interactions, merely because someone has labelled the tools “Agile”.
You can maintain a waterfall-style project plan and use that to feed stories into your agile sprints. If something needs to change, then you feed that back up into your master plan, just like you would if you missed a milestone. The difference is that, with agile, you get more continuous feedback and you explicitly model iteration in your plan (since if you're doing it right, you're delivering progressively more capable versions of the entire system and not just components to be integrated at the end). Nothing in agile precludes long-range planning, or taking deadlines into account.
I definitely agree with the author. Agile is increasingly being used to justify having no plan and just getting devs to do whatever seems most important to a "product owner" each sprint without the real customer feedback that is an essential part of agile.
Don't you think the blame here on the methodology is misplaced? It just sounds like a bad team/company, that'll perform just as poorly with "waterfall"
A bad team will always find a way to pay lip service to any process or workflow you throw at them and still fall back on their old dysfunctional habits.
Its never the process at fault, its the gaping disconnect between the specification of that process and the implementation.
Totally agree. Bad management will mess up any methodology and good management will make almost any methodology work. Good people will just adjust things until they are OK.
The key is, if the 'product owner' is not the end user, they are not the customer. They are often beholden to the more senior management and are more focussed on making their bosses happy with progress reports and demonstating how well they are performing. All this, while playing a theatre of customer centric focus and high velocity.
Current HN title is both not the source title and a misrepresentation of the point of the article. Both the title and body criticize “acephalic Agile”, which the author defines as the use of Agile tools without Agile governance. This is not arguing that “Agile is worse than Waterfall”, but at most “a particular mode of quasi-Agile is worse than Waterfall”.
(Given that the Agile Manifesto doesn't say anything directly about tools but entirely lays out a philosophy addressing governance, I'd actually argue that “Agile” tools without Agile governance aren't anything like Agile, and aren't even really Agile tools; Agile tools are tools chosen for a particular tasks through Agile governance, at a level higher than the governance level this price addresses, and applied within an Agile governance process at the level this piece addresses, which is not to endorse the particular governance approaches in this article or it'd horrendous strawmanning of Waterfall, which inasmuch as it is a real model, is an iterative model of exactly the type this piece is advocating.)
The author of the article is missing a negotiation essential to XP, at least: the customer can change his or her mind, but the scheduled tasks are based on developer estimates on how long they will take.
You can't make these estimates with any accuracy without the customer and at least some developers talking. Furthermore, customers aren't likely to pick something that the developers have estimated will take a long time, unless it really is that important to spend that much time on it.
Also, large tasks need to be broken down into smaller tasks to get decent estimates, which means knowing roughly what needs to be done. If the job isn't obvious, something quite like traditional planning needs to happen there.
The planning difference is really for smaller changes that don't have far-reaching consequences.
Classic consultant move. Trash talk current status quo, rebrand old solution and package it as the holy grail and make $$$ selling your services. What he proposes will also always fail because it is a process dictated from the top down - hire expensive consultants to tell management what to do then dictate solution to rest of org. Unless the solution emerges bottom up it won’t make jack of a difference.
I'll never understand why the software development community as a whole seems to have such an NIH attitude to software project management. Meaning, why aren't our people studying what works in other industries and figuring out how to apply it to our industry?
Is it some conceit that software development is somehow truly unique compared to the problems and concerns faced by other industries? That's dumbfounding to me if so.
Our industry is barely 70 years old and hardly anybody seems interested in stealing the best ideas and practices from other industries that in some cases have been around for thousands of years.
Partially. But I think waterfall has (at least) two major weaknesses.
The requirements are textual. They should be visual like a blueprint.
A real custom home is typically visited and inspected regularly by the future home owner during construction. An improvement for waterfall would be a periodic delivery to the user of intermediate builds of the product – no matter how incomplete.
> A real custom home is typically visited and inspected regularly by the future home owner during construction. An improvement for waterfall would be a periodic delivery to the user of intermediate builds of the product – no matter how incomplete.
With the intention of following the blueprint all the way to the end, user wishes be damned? No, that would not qualify.
But it would qualify if you actually delivered partially built product in iterations and incorporate the changes that the client requests as they inspect it, even if you preceded it with extensive blueprinting. Only in that case, it has been found that developing such detailed blueprint is very time consuming and at the same time not very useful. So we don't do it for most kinds of software projects, but not because it's against any agile principles.
The original agile principles (I was just getting into software development when they were articulated) really prohibit very few things, if anything at all. But they do emphasize valuing working, useful software over following strict plan. Use planning to the exact extent that it helps you do create such software.
The only part of the agile approach I like is iterative delivery.
First, I believe software blueprints should be put together by people with 15 to 30 years of expertise. Secondly, blueprints for a custom home are completely indispensable and are referred to regularly during construction. I believe one of the reasons why is because they are visual, not textual. I believe software requirements should be the same and then they would be easier to adjust and would retain their value.
Custom homes can be adjusted in mid-build. But it is a big deal and you have to get the architect involved and potentially have your plans revetted by inspectors and the city. It's a process that has a high cost. And it should because good blueprints like good software designs are highly dependent internally. I believe every change risks degrading their quality.
If you are in a project where there are large numbers of unknowns about the functionality of the project, I highly recommend shrinking the scope to what you know and actually understand. Build that, and then learn from it. Then come back around and start a new "dave's custom home waterfall" (for lack of a better name) process to implement the next set of functionality.
Constantly course correcting is a recipe for never finishing and weakening quality. There should be natural impediments to course correction and scope reduction should be used until the unknowns have been reduced to a reasonable risk level.
> First, I believe software blueprints should be put together by people with 15 to 30 years of expertise.
Well, that is for one simply completely impractical, those people are few compared to the amount of software the world needs and also very expensive. But it's only one of many reasons why there are better approaches so let's move on.
> Secondly, blueprints for a custom home are completely indispensable and are referred to regularly during construction. I believe one of the reasons why is because they are visual, not textual. I believe software requirements should be the same and then they would be easier to adjust and would retain their value.
Many detailed requirements have, as matter of fact, been very visual, with detailed user interface mockups, process diagrams etc. When they are large in scope, they have been found not to be useful compared to the effort required to create them. When they are small in scope, they are actually used in real world agile software projects.
And, I have to ask something about this process where extremely experienced people create detailed visual blueprints for the entire scope of the project and then deviate very little from it. Is it something you've actually seen produce good results (repeatedly) for your run-of-the-mill software projects?
> If you are in a project where there are large numbers of unknowns about the functionality of the project
... that would be vast majority of them.
> I highly recommend shrinking the scope to what you know and actually understand. Build that, and then learn from it. Then come back around and start a new "dave's custom home waterfall" (for lack of a better name) process to implement the next set of functionality.
Well, every time you give concrete proposal about what to do in the face of uncertainty it sounds exactly like agile.
> scope reduction should be used until the unknowns have been reduced to a reasonable risk level.
That would, again, be agile. And I say that as someone who is not even particularly fond of of most standardized agile processes, such as scrum.
> Well, that is for one simply completely impractical, those people are few compared to the amount of software the world needs and also very expensive. But it's only one of many reasons why there are better approaches so let's move on.
How many years of expertise do you think architects of large buildings probably have? Engineers of critical bridges and other public infrastructure? One thing we need to do is stop aging out our best talent.
> Many detailed requirements have, as matter of fact, been very visual, with detailed user interface mockups, process diagrams etc. When they are large in scope, they have been found not to be useful compared to the effort required to create them. When they are small in scope, they are actually used in real world agile software projects.
> And, I have to ask something about this process where extremely experienced people create detailed visual blueprints for the entire scope of the project and then deviate very little from it. Is it something you've actually seen produce good results (repeatedly) for your run-of-the-mill software projects?
Absolutely. I have a 100% success rate using these techniques with over 20 under my belt. Granted, I'm mainly a solo operator but I've used it for larger projects too on occasion.
> Well, every time you give concrete proposal about what to do in the face of uncertainty it sounds exactly like agile.
The difference is time frame and phasing. Consider deciding to enlarge a kitchen in the middle of building a custom house. Now compare that to an addition to an existing home. They are very different situations.
> That would, again, be agile. And I say that as someone who is not even particularly fond of of most standardized agile processes, such as scrum.
The difference again is the time frame and the phasing. Messing with the scope/design every week I think is an anti-pattern and a contributor to project failure.
Look at a building shell, even if you have zero clue about houses you can see if they used rotten wood. It looks like it will become a house. The more it progresses.. well, you can easily tell "that door shouldn't be where it is" and so on.
Now look at the code generated by $framework via "$tool new project $name". To a non-coder (or even just not using that language) it's not really discernible from a 90% finished product. Your house doesn't turn invisible because looked at it from the back.
Also house construction follows certain "easy" patterns a 5 year old can grasp - you need to start at the bottom, you can't put the paint on before the walls are in, etc.pp - give a coder of 10 years a project in another language without documentation and you won't hear anything in the direction of "it's more 10% or 80%" unless spending days. I'm not saying people building houses have an easier job - but it's palpable.
To the first concern - that’s why I recommend you deliver (ideally) weekly intermediate builds of the product to the end user. Typically an underlying issue will begin to surface pretty rapidly.
To the second concern, first, let’s get on the same page. When I use the home analogy it’s in regards to a custom home - not a from-plan/spec home. To your point, I believe well developed software should also have strong and obvious patterns that are reused in many if not all other projects.
From the abstract:
We methodologists and process designers have been designing complex systems without characterizing the active components of our systems, known to be highly non-linear and variable (people). This paper outlines theories and projects I reviewed on the way to making this stupendously obvious but notable discovery and four characteristics of people that most affect methodology design and project outcome. I find these characteristics of people to be better predictors of project behavior and methodology success than other methodological factors.
"In what follows I’m assuming that we’re talking about a project rather than merely a production line for rolling out bug fixes and features. In the latter case you can use Agile tools like Scrum to deliver code incrementally, and come to no great harm. In talking of a ‘project’ however, I’m thinking of a larger program of work in which an entire solution has to be designed and delivered, (more or less) complete. Such projects have distinct beginning, middle and end phases, all of which must complete successfully if the project is to succeed."
This doesn't make any sense and is contradictory.
It seems you admit that Agile works for "Rolling out bug fixes and features" and to "deliver code incrementally" without "coming to great harm".
Then you say that this is somehow different from "an entire solution [that has to be] designed and delivered".
What precludes "an entire solution" from being developed in an Agile fashion?
Presumably, agile would be to build out the skeleton, and incrementally add small working features over time in increments such that at the end, you had working software that was what you wanted.
You'd put your skeleton it into beta use as soon as possible once it provides any value, and get frequent feedback to iterate.
> but at least this way if the developers are working a late shift because of a rash promise they’ve made, they are the authors of their own misfortune, which is easier to bear. It’s a social contract: we give people control, they accept responsibility.
Congratulations, your lead developer is now a middle manager.
I've never been a fan of "committing to a certain amount of stuff done by the end of the week" as in scrum sprints. It's a zero sum game. Sometimes you do 3 days' worth of work in an afternoon and sometimes finding an extra semicolon takes 2 days...
To your first point, I had a very productive project where instead of as in the article the devs become managers, our project manager was brought very much into the dev team, almost as a tester, using the product daily. He became attuned to the pace of development and never over-promised anything. To your second point, I've had projects where I simply assigned a day to every task, and it turned out remarkably accurate.
109 comments
[ 3.4 ms ] story [ 182 ms ] threadAgile involves getting the customer deeply involved from the start, so that after one or two spins of cycle, they have a more sober view of what the magicians are doing, and how little they, the customer, get to ask for in the real world, if they want the project to be done this century (and they do.) Done well, it's the equivalent of sticking your dog's nose into the mess they made on your rug; where dog=client.
He made it quite clear, it's his conclusion:
Now, each iteration offers not only the chance for project management to be exercised, but more specifically, to ensure that it is exercised by capturing the appropriate, timely information from all the appropriate stakeholders.
Emphasis mine.
That’s difficult to codify, especially in contracts for delivery. “Agile methods” more often than not translate into hiding a complete lack of planning in a bunch of mumbo jumbo.
The point is that there are always stakeholders and product owners - it just makes that explicit. If the PO is acting as a proxy for the stakeholder (customer in this case) that's ok.
Of course the PO might be wrong... but with small iterations that should be found out quickly and adjusted for.
https://www.agilealliance.org/agile101/12-principles-behind-...
It would be odd if "business people" just meant the developers' upper management within their firm; supervising costs and parking spaces; but not those without a stake in using the software that results, say.
In fact the only time the customer is mentioned is in principle one: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
Business people absolutely should be involved - product owners & stakeholders (or a proxy for them).
That’s fundamental but again: business people != customers.
You seem to think stakeholders/product owners are devs direct managers too. That’s rarely the case, if ever, on any functioning agile team I’ve seen. They’re orthogonal to each other.
I don't believe I can be of further assistance to you.
You were never any “assistance”. You had nothing of substance to add.
The article makes a very reasonable argument, however I wish he wouldn't try to introduce some new term, Agile "governance". He's just describing what we already know -- that Agile isn't Agile and care must be taken to institute it correctly. via "governance" as he calls it but this isn't a new revelation.
I liken this to diets. Paleo suggests not eating bread (among other things). However, not eating bread doesn't mean you are doing Paleo. The benefits of Paleo are intended to be realized via a systemic application.
So whether it is agile or diets, make rules that work for you (and yours). Then stick to them. Periodically review if they are working. If they aren't change the part that isn't into a new rule and then repeat. There is no one size fits all in these things.
Some smart, well-meaning people write down a couple of brief, broad suggestions on how to do better, and a few years later it's been twisted into an industry of people who want me to pay them for telling me that I'm living my life wrong.
Also, there are larger truths that live at its heart which make discussions about the topic difficult because if you're seen as disagreeing with some of it, you're seen as disagreeing with all of it, even if you don't.
What a good analogy!
I've been meaning to read the original Waterfall article as well, I've heard it's good.
But I believe that if you really want to know how to hit home runs, you want to talk to the king's trainer, not the king. The king has no perspective. He has mostly only worked with himself. To put it bluntly: he's too wound up in his own bullshit to be objective.
Software Developers don't have trainers. Ask a successful person how they got that way and they have a sample size of one, or a handful. All you'll get is speculation.
I gather from the article's initial paragraphs that these "Governance" point is really the team getting input from all the normal stakeholders and creating the backlog of work themselves? This makes all the sense in the world - getting technical folks involved in discussions is key.
What I would like to see added are concrete steps / workflows that these technical teams have found that help make these governance points effective without eating up a lot of time, and what practices they used to ensure that the project goal / direction / outcomes were decided upon up front.
I personally think Agile Roadmapping / Storymapping with the whole group can go a long way toward ensuring whatever you build is going in the right direction. From there, you'll want the team to work with the product owner folks and keep a tighter grip on the backlog toward that direction and good things can happen.
This article scrapes the surface: https://michaelochurch.wordpress.com/2015/06/06/why-agile-an...
> "The worst thing about estimates is that they push a company in the direction of doing work that’s estimable. This causes programmers to favor the low-yield, easy stuff that the business doesn’t actually need but that is 'safe'".
> "Good engineers want to work in engineer-driven firms where they will be calling shots regarding what gets worked on, without having to justify themselves to 'scrum masters' and 'product owners' and layers of non-technical management"
> "Scrum [is] tailored toward the body shops where client relationships are so mismanaged"
> "Open-plan offices are the most egregious example. They aren’t productive"
> "[Scrum is] Like a failed communist state that equalizes by spreading poverty"
There is more. In "pure scrum" it's not obvious how to account for refactoring, developer experimentation, improving infrastructure (or creating one from scratch!), making tests better. Product driven things - sure, can be driven by scrum. But what to do when work is driven by engineering?
The kind of work that good engineers thrive on - a hard problem starting with en empty Emacs buffer - is something that the methodology merchants have never experienced themselves. The only product they ever shipped is the methodology itself!
I say this as a one-time “certified scrum master”.
I think it's an interesting article, with good points, but fundamentally I have to disagree with this statement. Empirically, it just isn't true, in my experience. It just means that all those changes by the customer to what they wanted, that the devs object to and management (which has to meet the customer face to face) always in the end agrees to, happen at the end, when they are more difficult, because up until that point the project wasn't "real" to the client, and they really couldn't tell you what they wanted until they saw that what you built wasn't it.
The same thing goes on with OOP. It started out as a good addition to other programming styles, got turned into a religion ("you can't do this. It's not OOP.") and now into a pariah (FP is next to go through this cycle).
In the end stupid people will mess up anything they touch and good people will adapt things until they work.
I mean... there's no correct application of waterfall. The whole waterfall process comes from a paper as an example of how not to run an engineering project.
It's only for not well understood problems that it fails so completely. In software, in my opinion, fewer problems are well understood at the moment, and so we tend to avoid waterfall.
However, if someone wanted me to set up a webserver to serve static files that don't change? I'm 99% sure I could set up nginx on a virtual machine "fully waterfall" and have no problems.
The same is true for many other real life "waterfall" processes, like buildings, or bridges.
Source: was a commercial interiors contractor before I was a software engineer.
Or if the problem is that well-understood, there's an off-the-shelf solution so good nobody seriously thinks about re-implementing it.
> However, if someone wanted me to set up a webserver to serve static files that don't change? I'm 99% sure I could set up nginx on a virtual machine "fully waterfall" and have no problems.
Exactly. That's barely software development. That's using fully-commoditized components in well-understood fashions to achieve a well-understood goal.
You could script that. I'm sure some places have.
The most well defined piece of software I know of, sel4, had a full machine checkable formal proof up front, and they still ran into issues during implementation that required them to go back and modify the spec.
It's not my fault that it appears you've been using the term wrong. Waterfall with feedback ceases to be waterfall.
Our industry being rather foolish, people read the paper and thought, 'hey, this sounds like a good idea!'
I don't think it really has anything to do with gravity.
That is a very strange thing to say considering that the method comes from "real" branches of engineering (mech, civil, electrical, etc) and you can simply look around you at your built environment to know that it has delivered an awful lot of real, actual stuff. They didn't agile up the building you're sitting in or the roads that lead to it or power station that makes it work or any of the things in it.
Do you know where I'm sitting?
https://en.wikipedia.org/wiki/How_Buildings_Learn
Waterfall as far as software is concerned has its origins in a paper from Winston W. Royce and not an unguided adoption of mech/civil/electrical engineering principles: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/wate...
one former employer purportedly did agile but it was a top-down autocracy, which is completely incompatible with agile. (i was ostensibly a product owner, but the CEO set our development priorities with only superficial consideration for customer feedback.)
you cannot both dictate and empower at the same time, and empowerment (via agile) is typically the loser in that fight.
that's how i see agile "failing" most often, before it even has a chance.
There is no perfect, but one that is fragile to external business realities is probably worse than one that is resilient to it.
(In my company, the problem with agile was that the sales people would insist on always adding some extra tool for a fixed price. We had no choice but to abandon agile.)
agile seeks to benefit from the intrinsic motivation of each team member to create something beyond any single member’s capabilities in exchange for empowerment.
when a manager doesn’t understand this principle, that leads to failure (as in your case as well). i don’t know of any systems that can self-correct for this kind of external mis-judgement.
What does that have to do with agile? That seems to be a problem that would occur, and be problematic, independent of development methodology.
> We had no choice but to abandon agile.
Seems to me you should have abandoned the offending sales people.
Agile is a culture about project management methodologies, which unfortunately is extremely resistant to actually adopting methodology that reflects it's overt cultural values. Largely, IMO, this is because the agile community has been very bad at operationalizing the “over” statements in the Manifesto into mechanisms by which the right side items are subordinated to the left side items. This results in (for the first statement in the Manifesto) either dysfunctional interaction and demotivated people from constant conflict resulting from the absence of processes and tools, or the adoption processes and tools in a manner which does not respect people and interactions, merely because someone has labelled the tools “Agile”.
I blame scrum for commodotising and cargo-culting all the good stuff out of agile. I'm writing a blog series on the problems with Scrum here: https://www.lambdacambridge.com/blog/2018-05-how-scrum-destr...
Interestingly, DSDM, which the author uses, doesn't get much attention, but was there from the beginning of the Agile movement.
Its never the process at fault, its the gaping disconnect between the specification of that process and the implementation.
(Given that the Agile Manifesto doesn't say anything directly about tools but entirely lays out a philosophy addressing governance, I'd actually argue that “Agile” tools without Agile governance aren't anything like Agile, and aren't even really Agile tools; Agile tools are tools chosen for a particular tasks through Agile governance, at a level higher than the governance level this price addresses, and applied within an Agile governance process at the level this piece addresses, which is not to endorse the particular governance approaches in this article or it'd horrendous strawmanning of Waterfall, which inasmuch as it is a real model, is an iterative model of exactly the type this piece is advocating.)
You can't make these estimates with any accuracy without the customer and at least some developers talking. Furthermore, customers aren't likely to pick something that the developers have estimated will take a long time, unless it really is that important to spend that much time on it.
Also, large tasks need to be broken down into smaller tasks to get decent estimates, which means knowing roughly what needs to be done. If the job isn't obvious, something quite like traditional planning needs to happen there.
The planning difference is really for smaller changes that don't have far-reaching consequences.
Is it some conceit that software development is somehow truly unique compared to the problems and concerns faced by other industries? That's dumbfounding to me if so.
Our industry is barely 70 years old and hardly anybody seems interested in stealing the best ideas and practices from other industries that in some cases have been around for thousands of years.
The requirements are textual. They should be visual like a blueprint.
A real custom home is typically visited and inspected regularly by the future home owner during construction. An improvement for waterfall would be a periodic delivery to the user of intermediate builds of the product – no matter how incomplete.
May I suggest you look into this agile thing?
But it would qualify if you actually delivered partially built product in iterations and incorporate the changes that the client requests as they inspect it, even if you preceded it with extensive blueprinting. Only in that case, it has been found that developing such detailed blueprint is very time consuming and at the same time not very useful. So we don't do it for most kinds of software projects, but not because it's against any agile principles.
The original agile principles (I was just getting into software development when they were articulated) really prohibit very few things, if anything at all. But they do emphasize valuing working, useful software over following strict plan. Use planning to the exact extent that it helps you do create such software.
First, I believe software blueprints should be put together by people with 15 to 30 years of expertise. Secondly, blueprints for a custom home are completely indispensable and are referred to regularly during construction. I believe one of the reasons why is because they are visual, not textual. I believe software requirements should be the same and then they would be easier to adjust and would retain their value.
Custom homes can be adjusted in mid-build. But it is a big deal and you have to get the architect involved and potentially have your plans revetted by inspectors and the city. It's a process that has a high cost. And it should because good blueprints like good software designs are highly dependent internally. I believe every change risks degrading their quality.
If you are in a project where there are large numbers of unknowns about the functionality of the project, I highly recommend shrinking the scope to what you know and actually understand. Build that, and then learn from it. Then come back around and start a new "dave's custom home waterfall" (for lack of a better name) process to implement the next set of functionality.
Constantly course correcting is a recipe for never finishing and weakening quality. There should be natural impediments to course correction and scope reduction should be used until the unknowns have been reduced to a reasonable risk level.
Well, that is for one simply completely impractical, those people are few compared to the amount of software the world needs and also very expensive. But it's only one of many reasons why there are better approaches so let's move on.
> Secondly, blueprints for a custom home are completely indispensable and are referred to regularly during construction. I believe one of the reasons why is because they are visual, not textual. I believe software requirements should be the same and then they would be easier to adjust and would retain their value.
Many detailed requirements have, as matter of fact, been very visual, with detailed user interface mockups, process diagrams etc. When they are large in scope, they have been found not to be useful compared to the effort required to create them. When they are small in scope, they are actually used in real world agile software projects.
And, I have to ask something about this process where extremely experienced people create detailed visual blueprints for the entire scope of the project and then deviate very little from it. Is it something you've actually seen produce good results (repeatedly) for your run-of-the-mill software projects?
> If you are in a project where there are large numbers of unknowns about the functionality of the project
... that would be vast majority of them.
> I highly recommend shrinking the scope to what you know and actually understand. Build that, and then learn from it. Then come back around and start a new "dave's custom home waterfall" (for lack of a better name) process to implement the next set of functionality.
Well, every time you give concrete proposal about what to do in the face of uncertainty it sounds exactly like agile.
> scope reduction should be used until the unknowns have been reduced to a reasonable risk level.
That would, again, be agile. And I say that as someone who is not even particularly fond of of most standardized agile processes, such as scrum.
How many years of expertise do you think architects of large buildings probably have? Engineers of critical bridges and other public infrastructure? One thing we need to do is stop aging out our best talent.
> Many detailed requirements have, as matter of fact, been very visual, with detailed user interface mockups, process diagrams etc. When they are large in scope, they have been found not to be useful compared to the effort required to create them. When they are small in scope, they are actually used in real world agile software projects.
> And, I have to ask something about this process where extremely experienced people create detailed visual blueprints for the entire scope of the project and then deviate very little from it. Is it something you've actually seen produce good results (repeatedly) for your run-of-the-mill software projects?
Absolutely. I have a 100% success rate using these techniques with over 20 under my belt. Granted, I'm mainly a solo operator but I've used it for larger projects too on occasion.
> Well, every time you give concrete proposal about what to do in the face of uncertainty it sounds exactly like agile.
The difference is time frame and phasing. Consider deciding to enlarge a kitchen in the middle of building a custom house. Now compare that to an addition to an existing home. They are very different situations.
> That would, again, be agile. And I say that as someone who is not even particularly fond of of most standardized agile processes, such as scrum.
The difference again is the time frame and the phasing. Messing with the scope/design every week I think is an anti-pattern and a contributor to project failure.
Look at a building shell, even if you have zero clue about houses you can see if they used rotten wood. It looks like it will become a house. The more it progresses.. well, you can easily tell "that door shouldn't be where it is" and so on.
Now look at the code generated by $framework via "$tool new project $name". To a non-coder (or even just not using that language) it's not really discernible from a 90% finished product. Your house doesn't turn invisible because looked at it from the back.
Also house construction follows certain "easy" patterns a 5 year old can grasp - you need to start at the bottom, you can't put the paint on before the walls are in, etc.pp - give a coder of 10 years a project in another language without documentation and you won't hear anything in the direction of "it's more 10% or 80%" unless spending days. I'm not saying people building houses have an easier job - but it's palpable.
To the second concern, first, let’s get on the same page. When I use the home analogy it’s in regards to a custom home - not a from-plan/spec home. To your point, I believe well developed software should also have strong and obvious patterns that are reused in many if not all other projects.
Some are: http://alistair.cockburn.us/Characterizing+people+as+non-lin...
From the abstract: We methodologists and process designers have been designing complex systems without characterizing the active components of our systems, known to be highly non-linear and variable (people). This paper outlines theories and projects I reviewed on the way to making this stupendously obvious but notable discovery and four characteristics of people that most affect methodology design and project outcome. I find these characteristics of people to be better predictors of project behavior and methodology success than other methodological factors.
They are and have been. For one example, much of the Lean Software Development literature leans very heavily on that.
Why the things that are less systematic and empirical are more popular is a valid question, though.
This doesn't make any sense and is contradictory.
It seems you admit that Agile works for "Rolling out bug fixes and features" and to "deliver code incrementally" without "coming to great harm".
Then you say that this is somehow different from "an entire solution [that has to be] designed and delivered".
What precludes "an entire solution" from being developed in an Agile fashion?
Presumably, agile would be to build out the skeleton, and incrementally add small working features over time in increments such that at the end, you had working software that was what you wanted.
You'd put your skeleton it into beta use as soon as possible once it provides any value, and get frequent feedback to iterate.
https://en.m.wikipedia.org/wiki/Hype_cycle
Congratulations, your lead developer is now a middle manager. I've never been a fan of "committing to a certain amount of stuff done by the end of the week" as in scrum sprints. It's a zero sum game. Sometimes you do 3 days' worth of work in an afternoon and sometimes finding an extra semicolon takes 2 days...