While I like the basic premise of embracing the uncertainty, I disagree that "nature" (evolutionary model) is the approach to follow: while it has produced marvelous results, "experiments" took way too long to learn from if they were progress/success or not.
I think we should embrace experimentation, but we should improve on natural processes because we are smart, sentient beings, and there are things we can predict and measure much faster, and thus pivot much faster too (as an example, all genetic diseases that are commonly terminal after the early reproductive age in humans are likely to be carried on into descendants because, well, reproduction has already happened — it would take many generations for "nature" to weed out those diseases, but luckily, we've got medicine that helps us not even having to weed them all out, and does stop some altogether).
Whether nature follows a plan is up for debate. I am personally in the camp that it doesn't, but I understand if someone isn't.
I am guessing that was not your point though, but that you simply missed that I was only complaining about strictly adhering to "natural" process which the OP seems to hold in very high regard (rightfully so, but they fail to mention that we can and should improve on them, which was the point I was making).
I 100% agree, it was not my intention on the post that “nature” is the approach to follow completly, we don’t want to take million years randomly guessing for sure, just borrow a part of what nature does, which is facing the real world, letting some parts die and move on. I agree with your completion to the point
I edited the title because agile is one of those words that cause commenters to immediately react reflexively (yay agile or boo agile...but mostly boo) and this is a more thoughtful article that deserves better.
(Side note - when we do this, we try always to use representative language from the article itself)
> Those four sentences are now an unchallenged way to better develop software in general
Pure garbage. Those 4 "values" are incredibly destructive and have led to far more problems than they ever potentially might have solved. You'll get plenty of challenge here, I'll wager.
> after over 20 years they proved their value empirically - but why is it that they make so much sense?
They don't. They certainly haven't proved their value empirically. For every "agile" success, there are countless failures, because "agile" isn't a magic bullet.
> Pure garbage. Those 4 "values" are incredibly destructive and have led to far more problems than they ever potentially might have solved.
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
I'm not sure which of these four you believe has led to far more problems that they potentially may have solved. Maybe the contract negotiation part. In my experience the core values of agile were fine. Where agile often went off the rails is when people implemented agile in a way that contradicts the four values. Definitely not pure garbage.
> You'll get plenty of challenge here, I'll wager.
Please don't speak for me, thank you. I've done well with agile, and continue to do so.
> No true Scotsman, or appeal to purity, is an informal fallacy in which one attempts to protect their universal generalization from a falsifying counterexample by excluding the counterexample improperly. (Wikipedia)
Please explain how the statement you quoted relates to the One True Scotsman fallacy. “It doesn’t work because you’re doing it wrong” is not an example of the fallacy.
(For example: “A: Light bulbs suck because they don’t light up when you smash them into your head. B: That’s not how lightbulbs work. A: No True Scotsman!” Yeah, no.)
That's a very reasonable statement, and is much easier to debate. In my experience, it's been the opposite...attempting to apply those four "value" results in more problems than it ever solves, because those four "values" are prioritizing the wrong things.
> In my experience the core values of agile were fine. Where agile often went off the rails is when people implemented agile in a way that contradicts the four values. Definitely not pure garbage.
In my experience, "agile" went off the rails the day it was conceived. I've yet to see a legitimate implementation of "agile" that wasn't a cheat in some way. Specifically, any organization I've encountered that claimed to be successful as an "agile" organization wasn't actually being "agile"...they were saying they were "agile", but would abandon it where it conflicted with actually getting work done.
> Please don't speak for me, thank you. I've done well with agile, and continue to do so.
How could anything I wrote be construed as "speaking for you"? Pretty egotistical on your part to assume I was. Based on past "agile" posts, I expect that there will be considerable pushback on what OP presents as empirically proven truth.
>.they were saying they were "agile", but would abandon it where it conflicted with actually getting work done.
Abandoning your process to get stuff done is within the manifesto.
However, what I don't ever get about Agile naysayers is any process that would fix their issues with the organizations they encounter. No sociopathic, micromanaging organization will ever enact a process that makes being sociopathic and micromanaging to their engineers harder.
The problem is those organizations, not the processes or frameworks. It makes as much sense as me saying every company I've worked for that made us use Python had an asshole as the CTO. Therefore, python causes asshole CTOs.
But why does it seem to be the case for every organization? Not just ones I personally worked for, but I get the same reports from peers all over the US, in very different businesses.
Its not. I have worked at a bunch. But let me repeat, it's not the process, its the managers that enact the process. Shitty management is everywhere, even within the same company, you can have some managers understand and push to build great software and never look at stats, where other managers just want commits per engineer to be higher. If you ask them, they will say they are being 'fair' and 'objective'.
So, some managers are only interested in making sure engineers aren't slacking off, they'll measure any way they can. LOC, Commits, Story points, it doesn't matter what, they want the measure, they want to be able to put you on a pip if that metric goes too low.
And yet consistently, everywhere I look, I see that "agile" simply doesn't work as described. Every article like this one treating "agile" as intrinsic truth is responded to by many who have seen first hand what a crock it is. The key observation for me is that, if you succeed with "agile", you would have succeeded as well or better without it. It's never why you were successful, but is often why projects fail. The best you can hope for is that it doesn't get in your way...hardly a ringing endorsement.
> if you succeed with "agile", you would have succeeded as well or better without it. It's never why you were successful, but is often why projects fail.
I would love to see some counter examples where companies succeeded writing business/application software using a well-defined process that wasn't 'Agile'. So please, let me know the counter example, what process is the alternative?
Projects of course. They don't have to adhere to some theoretical waterfall process, and anecdotally never have. There's Kanban, and any informal process a group of cross-functional people may come up with. In fact, it's much more important to make something that works, than copying some task-list because it's said to have worked somewhere else.
Incidentally, and this is going to shake up the "Agile" world going forward, is the missing piece, the link to management (budgets and trust really!) and innovation (the creative spark!). That should be less about process, and much more about culture and learning. That has remained elusive and underappreciated.
Google Design Sprint may be the closest "process", though will still depend more on the mentioned above.
With a great team, it anyway depends on the people. Though, the experience and output also depends on who you ask. Which is why great teams may get split up, for good reasons but also bad reasons at the same time.
Though try put the most brilliant people together. They will annoy eachother into oblivion. So it's about the art of building diverse teams and mixing with what's mentioned above, because nothing thrives in a vacuum.
I would call all that you described as Agile, sorry.
> In fact, it's much more important to make something that works, than copying some task-list because it's said to have worked somewhere else.
> That should be less about process, and much more about culture and learning
That pretty much describes the manifesto.
If you are saying regimented Scrum implemented by a rigid command hierarchy that assigns stories and demands story points per sprint. Well, those management hierarchies existed in the pre-scrum project world in spades, just as it exists now. Those management systems do not want to allow creative engineers, they want factory workers who they can measure the relative output and remove the lowest performers. They exist outside of any 'process' framework, or manifesto or the lack of one.
What probably writers of the manifesto did not completely see, is these management structures signing up for their manifesto trainings, then turning around and doing what they always did.
Your question implies that there was no successful software development before the Agile Manifesto was written, and that all successful software development since it was written has been exclusively done as "agile" development.
The alternative is the way that software was successfully developed and delivered for the decades prior to the existence of the Agile manifesto, and the software that was successfully developed and delivered in the time between when the manifesto was written and when it start infecting the software development industry. In other words, just look at literally any other software product created and delivered before 2001, and up until about 2008 or so.
So... some process that wasn't agile... that had no name, no principles... but it worked! Was this a singular process, that everyone just 'knew'? Or did every company have their own, that varied in principles and effectiveness. What if a group of people sat down and tried studying what worked and what didn't... say in the .. 2001 time frame, I wonder what they would come up with.
Well, since they came up with "Agile", it's pretty clear they didn't actually survey what worked and what didn't. They were consultants selling methodologies, and came up with something that was so poorly defined and difficult to implement that it gave them consulting jobs for life. Much better and easier than actually developing software!
>They were consultants selling methodologies, and came up with something that was so poorly defined and difficult to implement that it gave them consulting jobs for life
The dark truth behind Agile's origins. You simply cannot implement anything based on what they had written since nothing was defined concretely; everything was a platitude.
Agile never valued individuals. It does kinda values interactions, because it turns literally everything into formalized negotiation between multiple people. Which leads to social issues, but who cares, because agile dont actually value people.
> Working software over comprehensive documentation
We have no idea what our software should do nor whether it works. There is no place where you can learn how it works. It theoretically makes it easier to pretend it "works", because no one can tell whether it works. Practically, it never really works.
> Customer collaboration over contract negotiation
I was told to use planning rather than grooming because it could offend our British partners.
I suppose it’s because of a reference to sexual grooming ? Is that a English word only?
Or maybe dog grooming is a touchy subject in the US. I have no idea, culture evade me often here.
I sure did bring those items in grooming, it was promptly dismiss because too broad for the scope of the sprint at hand.
and kept for retro, where it could be properly buried for 4-6 weeks while someone take it as a action item of the retro.
The thing is : I kinda like Agile and Scrum. But it can become very circular if Product owner are not keeping everyone feets close to the fire.
This right here is the essence of everything that is wrong with Agile. I sometimes think the term "Bullshit Job" was made to describe Agile/Scrum specific positions :-)
Agree with this,many people think that the right process is agile. In fact, many times they don't consider the practical significance of these processes at all,but just learn from others' processes. This has no significance for agile practice
1. Nobody applies rigor to anything complicated anymore. The only way you can produce anything that works without rigor is to only produce very small things that you can prove work. So you can produce a lot of very small work and say "I'm developing so fast!", and meanwhile all the components connected together can be a tire fire. Short-term wins and long-term pain. This applies to a lot of the Agile process, but also the avoidance of processes and tools. Processes and tools can help or hurt, depending on how you use them. Even Agile is a process.
2. Working "software" is completely meaningless. What is actually valuable is a working solution to a user problem. I can make a program that very reliably spins in a loop forever, doing nothing. Who the fuck wants that? Not a user. And a large system, which is what a lot of software today is written for, cannot be connected together, operated, troubleshooted, etc without documentation. Nobody's going to be able to run your software or fix it when it breaks if they have no documentation, because they didn't write your software, so they have no fucking idea what to do with it. So we end up with a lot of unmaintainable widgets that don't solve user problems.
3. No software developer actually collaborates with a customer, because software developers have never cared about a customer. They care about their precious code, their algorithms, their design, their bugs, their tests. They don't care if the customer hates the software. I have literally never heard a developer utter the words "so how does the customer like the software?" This whole rule of customer collaboration is just an excuse not to have to think about a contract.
4. 'Responding to change' pretty much just means 'push out shit software and wait for bug reports'. For the first year of operation, Agile software is garbage. Because there was no coherent plan, it doesn't work for the customer, it isn't maintainable, and the other parts of a complex system that it operates as a part of will constantly break in weird ways. We don't need to make a plan, so we don't need to figure out how this thing is supposed to work, so in 6 months when we finally get around to a customer use case, we'll realize this architecture never would have worked within the constraints they're running it in.
Software development is not 'too damn complex'. Software developers are just lazy. We know that, because developers brag about being lazy all the time (I do too). They have these huge egos, and they believe that they're intelligent or talented because they can design an efficient algorithm or create 'beautiful code'. But the end result of all that code is what matters, not the code itself.
Agile is simply a 180-degree reaction to decades of work in developing processes that result in reliable solutions. The processes got complicated, so the software developers said, "fuck it, i'm just going to wing it". And as a result, most modern software is less reliable and more complicated than decades-old systems that are both simpler and solve more problems.
> No software developer actually collaborates with a customer, because software developers have never cared about a customer. They care about their precious code, their algorithms, their design, their bugs, their tests. They don't care if the customer hates the software. I have literally never heard a developer utter the words "so how does the customer like the software?" This whole rule of customer collaboration is just an excuse not to have to think about a contract.
You need to start working with better developers. Where I’ve worked, developers consistently and often push back on Product Manager requests when it feels like they aren’t solving the user’s actual problem.
Yeah. I’ve worked with developers with varying degrees of interest in the overall user problem, but that level of interest has never been less than “some”, and usually they fight just as hard as designers and product managers for solutions that truly solve the user’s problems.
I think you should look into why you keep ending up with such a-hole indifferent developers as coworkers. The ones I've worked with have in vast majority of cases cared about the problem being solved and didn't have large egos (too much imposter syndrome if anything).
> Nobody applies rigor to anything complicated anymore.
No one ever did. There was no golden period before Agile, where rigor was applied but now we've gone all soft. Before Agile, tire fires were bigger and found much later. That's why they developed Agile. Process and tools can indeed help, but the point of what we are doing in development, is to build working software, not to just follow the process or use the tools. That may seem obvious, but it wasn't and it still isn't, you see many leaders, even in "Agile" environments, who are more comfortable being able to report the process is working, than to challenge a customer to build a better product.
> What is actually valuable is a working solution to a user problem
That is obviously what they are talking about. You are creating straw men. The root of this sentence comes somewhat from Gall's law,
"A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system"
3. You don't have a very extensive work history if you've never found engineers who care about customers. I would look at successful companies and successful startups.
"Agile is simply a 180-degree reaction to decades of work in developing processes that result in reliable solutions."
Its not, its the gradual evolution of software processes, its over 20 years old at this point, its predated by other Agile processes. It came from study of project failures.
The more I re-read your rant, the more I realize either you are intentionally trying to create a stereotypical reaction to Agile or you are deeply disturbed. If you are a software engineer, you should just quit, why are you doing something that makes you so unhappy?
This is so beyond reality its absurd. Agile is not some "giving everyone a trophy" bs.
I have literally never heard a developer utter the words "so how does the customer like the software?"
I'm not surprised, given how many lousy places to work there are out there. But there are plenty of shops where this isn't the case. And where developers do in fact care what the customer thinks, because this obviously (if only slightly indirectly) correlates with the organization's chances for long-term success and revenue, and all that stuff.
None of which has anything to do with Agile of course (because the same issues existed and were dealt with thousands of years before the AM descended to earth, accompanied by angles).
But just saying - yes, actually giving a shit about what the customer thing is a thing, and always has been a thing (at better shops at least).
The one true takeaway of agile is measurement, metrics, and accountability. Your boss was sold on agile on the premise that it lets him keep continuous, precise tabs on you and the entire team, to ensure that y'all are hitting the OKRs for this quarter and so that when something goes wrong, the guilty party and what they did can be identified; or when the schedule slips, the laggard can be identified. This is the reason why shops are so gung ho about agile development. And this, I am forced to conclude, is agile. A game of Mornington Crescent in which the shop pretends to care about the stuff in the left column when the actual goal is to create a work environment so precisely measured, controlled, and optimized it makes Fred Taylor cum in his grave.
> Your boss was sold on agile on the premise that it lets him keep continuous, precise tabs on you and the entire team, to ensure that y'all are hitting the OKRs for this quarter and so that when something goes wrong, the guilty party and what they did can be identified; or when the schedule slips, the laggard can be identified.
Which is exactly what agile NOT is, according to the core values...
Agile, in a nutshell, says "let's stop being counterproductive (and miserable), it's better for everyone (except maybe sadists and masochists)".
Most of us really want to get things done and sometimes it devolves into self serving tripe but I've been forced to spend 50% of a project planning and then get ridden and harried because that vicious choice that no engineer wanted (but each was forced to do) puts off delivery.
You're right that agile is more positively stated and this declaration violates the more productive tone of the manifesto [0].
> Agile processes promote sustainable development
Did I take license? I did. Yet, I was present for some of the unpleasant and ineffective abuses and waste that "agile" was a counteraction to. Although the same types of leaders continue to act against my interpretation's goal, the industry is broadly improved (though physics continues to allow for localized terribleness). There are those who act as though talent retention isn't an important factor in delivery but they can't talk to us because they're busy trying to mend their velocity.
In my experience using agile, I haven't observed any failures- just failures from teams implementing it incorrectly.
There are many billion dollar consulting companies that have completely committed to agile and are flourishing. If it were as garbage as many people in this thread are saying, I feel like we would see more concrete evidence of it.
Source: Worked in agile teams on-and-off for ~5 years now, not at my current company.
my experience matches yours, but there's something to be scrutinized if a lot of people do something wrong. We can't keep telling people "you're holding it wrong".
For agile, I believe what needs to be stressed is always: adapt the process to your team.
A lot of the frustration is not with the agile manifesto, but with some externally enforced scrum/whatever.
Such negativity. It seems many on this thread did not live through the bad old days of waterfall and .mil standard development. A total nightmare. Scrum and agile are absolutely wonderful in comparison. Yes, it provides for measurement and close management oversight, but it also does not impose impossible schedules with locked requirements. Sometimes a problem and the solution are well-understood, and in these cases scrum works as a decent project management approach. But where agile really shines is when the problem is poorly understood and the solution not well defined. In these cases agile can provide for project successes that would have led to certain failure in a waterfall world. It's likely not an overstatement that agile and iterative development has been the recipe behind web 2.0 and so much else in recent years.
Projects were doing iterative development in the 50s, way before scrum or agile.
“All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities.”
— Weinberg G. M. (Project Mercury)
I think many of the agile principles remain largely unpalatable to your typical command & control org structure.
One of the main premises of Agile, is letting people closest to the work decide how they want to work.
Instead, in many companies, the org co-opted the meaning of Agile for its own agenda. The PMO got what it wanted, putting heavily gated process on top of Scrum. Why Scrum? Because its push, of course (instead of pull).
At the moment, my org is adopting SAFe, and PI planning, which has devolved into quarter long waterfalls for projects. We just use Scrum as 2 week waterfalls in the quarter waterfall.
I lived before agile came around. And bad the old days of waterfall are to large extend carricature. Maybe it existed in the horrible form agile advocates pretend, but I have no seen that. And for me, work was in fact more motivating before agile. Agile turned the work I really liked a lot into something you do for money. Agile at it best is tolerable.
> Sometimes a problem and the solution are well-understood, and in these cases scrum works as a decent project management approach.
It does not. It is micromanagement ritualized and it somehow leaves both customer and workers unhappy.
The book Why We Do What We Do: Understanding Self-Motivation by Edward Deci may give you some answers.
The key thing a "Knowledge Worker" needs/demands is Autonomy over his own Work/Decisions (within the context of the larger problem). In the old days, we had the Freedom along with the Accountability. What Agile/Scrum has done is taken away our Freedom but by micromanaging our day-to-day activities has increased our Accountability. This is completely unjustifiable and unacceptable.
PS: An organisation that treats its programmers as morons will soon have programmers that are willing and able to act like morons only - Bjarne Stroustrup.
I think it comes back to "autonomy, mastery, accountability". You was responsible for bigger chunks of work. You had more autonomy within that chunk - you could make at least some decisions independently. Whether they were bad or good, it was clear they were yours. You could also have some sense of control over your work - you could use own brain to plan and prioritize within that chunk.
Consequently, you felt like you have done and achieved something. You also could create something coherent. Also, over time you became better and better at the thing you was doing. You was thinking about module or project as a whole.
With agile, I feel mostly like cross between factory worker middle management politician. Every little detail has to be negotiated with multiple people and half success is guessing what is on their minds. You work on tiny tasks. You dont get sense of achievement, you just use your hammer to put in nail here, then paint a bit over there.
Weird because that’s exactly the points I feel agile should be help with. In the teams I’m in I always promote the whole team should be part of deciding what to do, prioritizing, and coming up with ideas of valuable things to do, I speak up and give bad feedbacks to PMs that monopolize the decisions
Also the team itself should be very autonomous inside the organization as much as your context allows. In fact, the company I work for had fully independent teams in startup mode for so long, that eventually they started to step on each other toes, there was very few alignment. As company grew, more alignment and therefore negotiations started to be needed but I don’t see much way around that other than staying small and horizontal (which I like too). Still, on parts of the company on “exploration mode” (from Kent Beck’s 3X model), teams still remain very autonomous
As for the small tasks, I see them as a way exactly to promote a sense of achievement, the team is responsible for this big chunk of work and you should be able to see very clearly how things are moving forward. Sure, if you lose connection with the bigger picture because you or your team does not have enough autonomy, then something is wrong and you need to fix it, otherwise you lose this effect.
> In the teams I’m in I always promote the whole team should be part of deciding what to do, prioritizing, and coming up with ideas of valuable things to do, I speak up and give bad feedbacks to PMs that monopolize the decisions
Otherwise said, all those decisions are done by committee. None of them can be done by any employee autonomously and no employee gets to have responsibility. You never do those things nebulous, notion of "team" does. But team is not you, it is group of people. All the situations you described are negotiation between many parties situations.
Also, micromanagement or decisions monopolization can be done by anyone on any level. Many developers love to do so - and exactly these do love agile, because that gives them cover to railroad everyone else. Agile is really big on making PM or management enemies. It does work to some extend, common enemy does bring people together.
> Also the team itself should be very autonomous inside the organization as much as your context allows.
And no single team member has no area of responsibility or autonomy. And that is the thing - if you like to dominate and influence other people, agile gives you plenty of playground. If you like to throw your weight in the meetings, great. But if you dont want to do that constantly all the time, you are expected to be passive follower with no space for any autonomy, mastery nor accountability.
> As for the small tasks, I see them as a way exactly to promote a sense of achievement, the team is responsible for this big chunk of work and you should be able to see very clearly how things are moving forward. Sure, if you lose connection with the bigger picture because you or your team does not have enough autonomy, then something is wrong and you need to fix it, otherwise you lose this effect.
Note how it is still the "team". You don't see nor mention any individual. No individual at that team achieved anything personally. Individuals almost don't exist. They did not made single tiny decisions that would not been discussed to death with five other people. For a methodic that is supposed to favor individuals, there is literally zero concern for individual or human psychology. The only interest is in group.
The small decisions should be made by the individuals of course, as group decisions are too expensive, but the bigger ones should be taken by the team, where it pays off, or at least a pair. I’ve dealt too much with bus factor problems in the past to want fully individual systems
It also happens on scrum teams a lot, the engineer the built certain part of the system is the only one to ever take tasks related to it and the only one that knows how to fix it, so when they are gone you have to deal with their mess
On XP pair programming solves that very nicely, with a lot of comradery, but then you may also argue there is not enough individuality
> On XP pair programming solves that very nicely, with a lot of comradery, but then you may also argue there is not enough individuality
No. That is literally extreme with zero individuality, zero autonomy and so on. Yes, there are extremes that are even worst then average in agile systems. But the extreme existing does not mean the average is somehow good.
> The small decisions should be made by the individuals of course, as group decisions are too expensive, but the bigger ones should be taken by the team, where it pays off, or at least a pair. I’ve dealt too much with bus factor problems in the past to want fully individual systems
Which small decisions are made by individuals in agile system? I am not aware of any, unless one is actively hiding what he is doing. That is my point. There is no autonomy, no mastery no accountability.
You said you was surprised that I found agile less motivating due to lack of above. And now you are telling me that there are systems which are even worst in that regard. Yes, I know they exist. And almost no one wants to work in such teams.
> It also happens on scrum teams a lot, the engineer the built certain part of the system is the only one to ever take tasks related to it and the only one that knows how to fix it, so when they are gone you have to deal with their mess
And now in agile system, I have to deal with everybody mess all the time. Because there is never a point in time when I know the part of system I work on well. I dont know what the system I work on is supposed to be doing. No one knows. So we all muddle through.
It seems many on this thread did not live through the bad old days of waterfall and .mil standard development.
Probably because for most of us they never existed - and yes, of course I mean "pre-agile".
This whole "before Agile it was all waterfall and no one could conceive of anything differently and oh my brothers, let me tell you what a nightmare it was" cant is just that - a cant, something people like to repeat without thinking about whether it has any actual validity in, you know, actual reality. In other words, a myth.
There have always been good shops and bad shops; but like the sibling commenter said, (in the better shops) people have been working iteratively for decades and decades before the Agile Manifesto descended to earth on a ray of light, accompanied by angels.
But where agile really shines is when the problem is poorly understood and the solution not well defined.
Which problems people, again, had already been solving for decades and decades before the AM.
Such negativity
"Negativity" is not only an appropriate, but a healthy and necessary response to the never-ending stream of cargo-cult tripe and talking point nonsense that is Agile.
> It seems many on this thread did not live through the bad old days of waterfall and .mil standard development. A total nightmare. Scrum and agile are absolutely wonderful in comparison.
I did. Scrum and "agile" are far, far worse, based on what I've seen and experienced. Those "bad old days" were heaven compared to the bizarre approach currently in vogue. I will be very happy when this fad passes as well.
> But where agile really shines is when the problem is poorly understood and the solution not well defined.
That's the problem...people think that you can just use "agile" and not worry about understanding the problem and developing a solid solution. You just start slamming out code and checking back with the customer, wasting huge amounts of time and effort.
Waterfall planning + critical path analysis was too brittle for software projects. The choices were:
1. Frequent replanning, every 2-6 weeks
2. Stop replanning and try to meet an unrealistic schedule
Agile introduced tactical planning and dropped precise schedule estimates, which did not hold up anyway.
If more Agile proponents would start with that instead of leading with the collaborative aspects of Agile, and drop the Scrum jargon, they would be better understood.
Making software is not like most other projects. Many more new problems to solve because if you already solved it, just reuse the code. That has practical consequences in project management, specifically: iterating toward the goal.
Collaborative task assignment for each iteration is important. But that's not why iterating toward a goal is how to do software.
I wholeheartedly agree with trying to learn lessons from Taleb's writings and applying it to Software Project/Risk Management.
OTOH, trying to interpret Agile/Scrum methodology in terms of the above is nothing more than Putting Lipstick on a Pig i.e. a valueless/useless endeavour.
If nothing, it was valuable for me already. My goal was not to preach agile (much less scrum), but to actually try to find a common rationale behind the good and bad experiences that people do feel they had coming from agile. And for me (maybe for other devs too), finding an explanation that makes sense, a more general theory, is very rewarding in itself, it allows contradictory ideas to fit together in my head
If you are trying to figure out and model something in your mind, then i am fully there with you.
The problem is your marrying it to Agile and its Manifesto. They are a bunch of platitudes, there is no exact definition far less a process/methodology, it can mean anything, everything and nothing in a given context and thus have nothing concrete to recommend themselves. As you can see in this thread, the words Agile/Scrum invoke rather strong feelings amongst the HN crowd and hence your message has gotten lost.
Instead, if you were to do a article on what/how you feel Taleb's ideas can be applied to Software Project/Risk Management, that would be an interesting and welcome read.
My first experience with agile was not with Scrum but XP, and it was a very good one. There are so many people here conflating Agile with Scrum, and I shouldn’t be surprised I guess, but my goal was not to invoke the “No true Scotsman fallacy”, I don’t care about pointing who is agile and who is not, or what names you use for what, but I do care about the principles behind it, they can give us very good ideas on how to balance certainty vs uncertainty, upfront design vs feedback cycles, in whatever environment you are. Some environments you will need to balance more to one side, some more to the other.
What did you get out of XP? Didn’t XP promote pair programming? Personally hate the idea of pair programming and would never work for a company that requires it.
Do you hate the idea of pair programming or the actual act? Have you tried it? It was indeed weird at first but then I had lots of fun from pairing, I really miss it since the whole corona started
Now that agile has been preached for over a decade, has software actually gotten “better” for end users?
So much focus seems to be on anti-patterns like mining users of their information or minimizing costs by outsourcing development to cheap, unskilled contractor mills and churning out widgets to meet the mad march toward unicorn status along with all the other identical unicorny startups.
The agile crowd talks about pleasing and providing the customers with a product that meets their needs, but is that what we are doing and has agile made this worse, not better?
90 comments
[ 3.4 ms ] story [ 172 ms ] threadI think we should embrace experimentation, but we should improve on natural processes because we are smart, sentient beings, and there are things we can predict and measure much faster, and thus pivot much faster too (as an example, all genetic diseases that are commonly terminal after the early reproductive age in humans are likely to be carried on into descendants because, well, reproduction has already happened — it would take many generations for "nature" to weed out those diseases, but luckily, we've got medicine that helps us not even having to weed them all out, and does stop some altogether).
Agile manifesto: "That is, while there is value in the items on the right, we value the items on the left more."
I am guessing that was not your point though, but that you simply missed that I was only complaining about strictly adhering to "natural" process which the OP seems to hold in very high regard (rightfully so, but they fail to mention that we can and should improve on them, which was the point I was making).
(Side note - when we do this, we try always to use representative language from the article itself)
Pure garbage. Those 4 "values" are incredibly destructive and have led to far more problems than they ever potentially might have solved. You'll get plenty of challenge here, I'll wager.
> after over 20 years they proved their value empirically - but why is it that they make so much sense?
They don't. They certainly haven't proved their value empirically. For every "agile" success, there are countless failures, because "agile" isn't a magic bullet.
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
I'm not sure which of these four you believe has led to far more problems that they potentially may have solved. Maybe the contract negotiation part. In my experience the core values of agile were fine. Where agile often went off the rails is when people implemented agile in a way that contradicts the four values. Definitely not pure garbage.
> You'll get plenty of challenge here, I'll wager.
Please don't speak for me, thank you. I've done well with agile, and continue to do so.
I can't remember a discussion about Agile in which the No true Scotsman fallacy wasn't mentioned at least once.
Please explain how the statement you quoted relates to the One True Scotsman fallacy. “It doesn’t work because you’re doing it wrong” is not an example of the fallacy.
(For example: “A: Light bulbs suck because they don’t light up when you smash them into your head. B: That’s not how lightbulbs work. A: No True Scotsman!” Yeah, no.)
"If your attempt at agile goes off the rails, you probably implemented it in a way that contradicts the four values".
In my experience, "agile" went off the rails the day it was conceived. I've yet to see a legitimate implementation of "agile" that wasn't a cheat in some way. Specifically, any organization I've encountered that claimed to be successful as an "agile" organization wasn't actually being "agile"...they were saying they were "agile", but would abandon it where it conflicted with actually getting work done.
> Please don't speak for me, thank you. I've done well with agile, and continue to do so.
How could anything I wrote be construed as "speaking for you"? Pretty egotistical on your part to assume I was. Based on past "agile" posts, I expect that there will be considerable pushback on what OP presents as empirically proven truth.
Abandoning your process to get stuff done is within the manifesto. However, what I don't ever get about Agile naysayers is any process that would fix their issues with the organizations they encounter. No sociopathic, micromanaging organization will ever enact a process that makes being sociopathic and micromanaging to their engineers harder.
The problem is those organizations, not the processes or frameworks. It makes as much sense as me saying every company I've worked for that made us use Python had an asshole as the CTO. Therefore, python causes asshole CTOs.
So, some managers are only interested in making sure engineers aren't slacking off, they'll measure any way they can. LOC, Commits, Story points, it doesn't matter what, they want the measure, they want to be able to put you on a pip if that metric goes too low.
And yet consistently, everywhere I look, I see that "agile" simply doesn't work as described. Every article like this one treating "agile" as intrinsic truth is responded to by many who have seen first hand what a crock it is. The key observation for me is that, if you succeed with "agile", you would have succeeded as well or better without it. It's never why you were successful, but is often why projects fail. The best you can hope for is that it doesn't get in your way...hardly a ringing endorsement.
I would love to see some counter examples where companies succeeded writing business/application software using a well-defined process that wasn't 'Agile'. So please, let me know the counter example, what process is the alternative?
Incidentally, and this is going to shake up the "Agile" world going forward, is the missing piece, the link to management (budgets and trust really!) and innovation (the creative spark!). That should be less about process, and much more about culture and learning. That has remained elusive and underappreciated.
Google Design Sprint may be the closest "process", though will still depend more on the mentioned above.
With a great team, it anyway depends on the people. Though, the experience and output also depends on who you ask. Which is why great teams may get split up, for good reasons but also bad reasons at the same time.
Though try put the most brilliant people together. They will annoy eachother into oblivion. So it's about the art of building diverse teams and mixing with what's mentioned above, because nothing thrives in a vacuum.
> In fact, it's much more important to make something that works, than copying some task-list because it's said to have worked somewhere else. > That should be less about process, and much more about culture and learning
That pretty much describes the manifesto.
If you are saying regimented Scrum implemented by a rigid command hierarchy that assigns stories and demands story points per sprint. Well, those management hierarchies existed in the pre-scrum project world in spades, just as it exists now. Those management systems do not want to allow creative engineers, they want factory workers who they can measure the relative output and remove the lowest performers. They exist outside of any 'process' framework, or manifesto or the lack of one.
What probably writers of the manifesto did not completely see, is these management structures signing up for their manifesto trainings, then turning around and doing what they always did.
Good post, yes and branding it "Agile".
The dark truth behind Agile's origins. You simply cannot implement anything based on what they had written since nothing was defined concretely; everything was a platitude.
> Working software over comprehensive documentation
We have no idea what our software should do nor whether it works. There is no place where you can learn how it works. It theoretically makes it easier to pretend it "works", because no one can tell whether it works. Practically, it never really works.
> Customer collaboration over contract negotiation
You mean constant negotiation all the time.
But anything threatening to rock the boat will be embraced and corrupted until it's worse than what came before, hence Scrum etc.
This is especially true with Scrum.
I suppose it’s because of a reference to sexual grooming ? Is that a English word only?
Or maybe dog grooming is a touchy subject in the US. I have no idea, culture evade me often here.
I sure did bring those items in grooming, it was promptly dismiss because too broad for the scope of the sprint at hand. and kept for retro, where it could be properly buried for 4-6 weeks while someone take it as a action item of the retro.
The thing is : I kinda like Agile and Scrum. But it can become very circular if Product owner are not keeping everyone feets close to the fire.
This, a national tragedy that isn't yet very well know I guess:
https://duckduckgo.com/?q=uk+grooming+scandal+&t=fpas&ia=web
But be aware that for anyone who knows the full details, this word might bring about a visceral reaction. It is beyond ugly.
Management will tell you that it is not the case, because it is agile so it is you that decided the process even if it was imposed to you...
And that you are negative to complain, because it cost nothing for you "to try"...
1. Nobody applies rigor to anything complicated anymore. The only way you can produce anything that works without rigor is to only produce very small things that you can prove work. So you can produce a lot of very small work and say "I'm developing so fast!", and meanwhile all the components connected together can be a tire fire. Short-term wins and long-term pain. This applies to a lot of the Agile process, but also the avoidance of processes and tools. Processes and tools can help or hurt, depending on how you use them. Even Agile is a process.
2. Working "software" is completely meaningless. What is actually valuable is a working solution to a user problem. I can make a program that very reliably spins in a loop forever, doing nothing. Who the fuck wants that? Not a user. And a large system, which is what a lot of software today is written for, cannot be connected together, operated, troubleshooted, etc without documentation. Nobody's going to be able to run your software or fix it when it breaks if they have no documentation, because they didn't write your software, so they have no fucking idea what to do with it. So we end up with a lot of unmaintainable widgets that don't solve user problems.
3. No software developer actually collaborates with a customer, because software developers have never cared about a customer. They care about their precious code, their algorithms, their design, their bugs, their tests. They don't care if the customer hates the software. I have literally never heard a developer utter the words "so how does the customer like the software?" This whole rule of customer collaboration is just an excuse not to have to think about a contract.
4. 'Responding to change' pretty much just means 'push out shit software and wait for bug reports'. For the first year of operation, Agile software is garbage. Because there was no coherent plan, it doesn't work for the customer, it isn't maintainable, and the other parts of a complex system that it operates as a part of will constantly break in weird ways. We don't need to make a plan, so we don't need to figure out how this thing is supposed to work, so in 6 months when we finally get around to a customer use case, we'll realize this architecture never would have worked within the constraints they're running it in.
Software development is not 'too damn complex'. Software developers are just lazy. We know that, because developers brag about being lazy all the time (I do too). They have these huge egos, and they believe that they're intelligent or talented because they can design an efficient algorithm or create 'beautiful code'. But the end result of all that code is what matters, not the code itself.
Agile is simply a 180-degree reaction to decades of work in developing processes that result in reliable solutions. The processes got complicated, so the software developers said, "fuck it, i'm just going to wing it". And as a result, most modern software is less reliable and more complicated than decades-old systems that are both simpler and solve more problems.
You need to start working with better developers. Where I’ve worked, developers consistently and often push back on Product Manager requests when it feels like they aren’t solving the user’s actual problem.
> Nobody applies rigor to anything complicated anymore.
No one ever did. There was no golden period before Agile, where rigor was applied but now we've gone all soft. Before Agile, tire fires were bigger and found much later. That's why they developed Agile. Process and tools can indeed help, but the point of what we are doing in development, is to build working software, not to just follow the process or use the tools. That may seem obvious, but it wasn't and it still isn't, you see many leaders, even in "Agile" environments, who are more comfortable being able to report the process is working, than to challenge a customer to build a better product.
> What is actually valuable is a working solution to a user problem
That is obviously what they are talking about. You are creating straw men. The root of this sentence comes somewhat from Gall's law, "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system"
3. You don't have a very extensive work history if you've never found engineers who care about customers. I would look at successful companies and successful startups.
"Agile is simply a 180-degree reaction to decades of work in developing processes that result in reliable solutions."
Its not, its the gradual evolution of software processes, its over 20 years old at this point, its predated by other Agile processes. It came from study of project failures.
The more I re-read your rant, the more I realize either you are intentionally trying to create a stereotypical reaction to Agile or you are deeply disturbed. If you are a software engineer, you should just quit, why are you doing something that makes you so unhappy?
This is so beyond reality its absurd. Agile is not some "giving everyone a trophy" bs.
umm I am not sure if this is true. To me ultimate thrill in my job is customers solving their real life problems with the software i built.
I'm not surprised, given how many lousy places to work there are out there. But there are plenty of shops where this isn't the case. And where developers do in fact care what the customer thinks, because this obviously (if only slightly indirectly) correlates with the organization's chances for long-term success and revenue, and all that stuff.
None of which has anything to do with Agile of course (because the same issues existed and were dealt with thousands of years before the AM descended to earth, accompanied by angles).
But just saying - yes, actually giving a shit about what the customer thing is a thing, and always has been a thing (at better shops at least).
Which is exactly what agile NOT is, according to the core values...
Most of us really want to get things done and sometimes it devolves into self serving tripe but I've been forced to spend 50% of a project planning and then get ridden and harried because that vicious choice that no engineer wanted (but each was forced to do) puts off delivery.
Where does "agile" say any of that?
> Agile processes promote sustainable development
Did I take license? I did. Yet, I was present for some of the unpleasant and ineffective abuses and waste that "agile" was a counteraction to. Although the same types of leaders continue to act against my interpretation's goal, the industry is broadly improved (though physics continues to allow for localized terribleness). There are those who act as though talent retention isn't an important factor in delivery but they can't talk to us because they're busy trying to mend their velocity.
[0] https://agilemanifesto.org/principles.html
In my experience using agile, I haven't observed any failures- just failures from teams implementing it incorrectly.
There are many billion dollar consulting companies that have completely committed to agile and are flourishing. If it were as garbage as many people in this thread are saying, I feel like we would see more concrete evidence of it.
Source: Worked in agile teams on-and-off for ~5 years now, not at my current company.
For agile, I believe what needs to be stressed is always: adapt the process to your team.
A lot of the frustration is not with the agile manifesto, but with some externally enforced scrum/whatever.
“All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities.” — Weinberg G. M. (Project Mercury)
I think many of the agile principles remain largely unpalatable to your typical command & control org structure.
One of the main premises of Agile, is letting people closest to the work decide how they want to work.
Instead, in many companies, the org co-opted the meaning of Agile for its own agenda. The PMO got what it wanted, putting heavily gated process on top of Scrum. Why Scrum? Because its push, of course (instead of pull).
At the moment, my org is adopting SAFe, and PI planning, which has devolved into quarter long waterfalls for projects. We just use Scrum as 2 week waterfalls in the quarter waterfall.
See Barry Boehm's https://en.wikipedia.org/wiki/Spiral_model for what has already been done correctly.
> Sometimes a problem and the solution are well-understood, and in these cases scrum works as a decent project management approach.
It does not. It is micromanagement ritualized and it somehow leaves both customer and workers unhappy.
The key thing a "Knowledge Worker" needs/demands is Autonomy over his own Work/Decisions (within the context of the larger problem). In the old days, we had the Freedom along with the Accountability. What Agile/Scrum has done is taken away our Freedom but by micromanaging our day-to-day activities has increased our Accountability. This is completely unjustifiable and unacceptable.
PS: An organisation that treats its programmers as morons will soon have programmers that are willing and able to act like morons only - Bjarne Stroustrup.
Consequently, you felt like you have done and achieved something. You also could create something coherent. Also, over time you became better and better at the thing you was doing. You was thinking about module or project as a whole.
With agile, I feel mostly like cross between factory worker middle management politician. Every little detail has to be negotiated with multiple people and half success is guessing what is on their minds. You work on tiny tasks. You dont get sense of achievement, you just use your hammer to put in nail here, then paint a bit over there.
Also the team itself should be very autonomous inside the organization as much as your context allows. In fact, the company I work for had fully independent teams in startup mode for so long, that eventually they started to step on each other toes, there was very few alignment. As company grew, more alignment and therefore negotiations started to be needed but I don’t see much way around that other than staying small and horizontal (which I like too). Still, on parts of the company on “exploration mode” (from Kent Beck’s 3X model), teams still remain very autonomous
As for the small tasks, I see them as a way exactly to promote a sense of achievement, the team is responsible for this big chunk of work and you should be able to see very clearly how things are moving forward. Sure, if you lose connection with the bigger picture because you or your team does not have enough autonomy, then something is wrong and you need to fix it, otherwise you lose this effect.
Otherwise said, all those decisions are done by committee. None of them can be done by any employee autonomously and no employee gets to have responsibility. You never do those things nebulous, notion of "team" does. But team is not you, it is group of people. All the situations you described are negotiation between many parties situations.
Also, micromanagement or decisions monopolization can be done by anyone on any level. Many developers love to do so - and exactly these do love agile, because that gives them cover to railroad everyone else. Agile is really big on making PM or management enemies. It does work to some extend, common enemy does bring people together.
> Also the team itself should be very autonomous inside the organization as much as your context allows.
And no single team member has no area of responsibility or autonomy. And that is the thing - if you like to dominate and influence other people, agile gives you plenty of playground. If you like to throw your weight in the meetings, great. But if you dont want to do that constantly all the time, you are expected to be passive follower with no space for any autonomy, mastery nor accountability.
> As for the small tasks, I see them as a way exactly to promote a sense of achievement, the team is responsible for this big chunk of work and you should be able to see very clearly how things are moving forward. Sure, if you lose connection with the bigger picture because you or your team does not have enough autonomy, then something is wrong and you need to fix it, otherwise you lose this effect.
Note how it is still the "team". You don't see nor mention any individual. No individual at that team achieved anything personally. Individuals almost don't exist. They did not made single tiny decisions that would not been discussed to death with five other people. For a methodic that is supposed to favor individuals, there is literally zero concern for individual or human psychology. The only interest is in group.
It also happens on scrum teams a lot, the engineer the built certain part of the system is the only one to ever take tasks related to it and the only one that knows how to fix it, so when they are gone you have to deal with their mess
On XP pair programming solves that very nicely, with a lot of comradery, but then you may also argue there is not enough individuality
No. That is literally extreme with zero individuality, zero autonomy and so on. Yes, there are extremes that are even worst then average in agile systems. But the extreme existing does not mean the average is somehow good.
> The small decisions should be made by the individuals of course, as group decisions are too expensive, but the bigger ones should be taken by the team, where it pays off, or at least a pair. I’ve dealt too much with bus factor problems in the past to want fully individual systems
Which small decisions are made by individuals in agile system? I am not aware of any, unless one is actively hiding what he is doing. That is my point. There is no autonomy, no mastery no accountability.
You said you was surprised that I found agile less motivating due to lack of above. And now you are telling me that there are systems which are even worst in that regard. Yes, I know they exist. And almost no one wants to work in such teams.
> It also happens on scrum teams a lot, the engineer the built certain part of the system is the only one to ever take tasks related to it and the only one that knows how to fix it, so when they are gone you have to deal with their mess
And now in agile system, I have to deal with everybody mess all the time. Because there is never a point in time when I know the part of system I work on well. I dont know what the system I work on is supposed to be doing. No one knows. So we all muddle through.
Very right! They were never true; mere blame-game when they screwed up something else.
>It is micromanagement ritualized and it somehow leaves both customer and workers unhappy
Well Said! Even stronger : A Ego/Power trip for people who can cut it as neither Developers nor actual Managers :-)
Probably because for most of us they never existed - and yes, of course I mean "pre-agile".
This whole "before Agile it was all waterfall and no one could conceive of anything differently and oh my brothers, let me tell you what a nightmare it was" cant is just that - a cant, something people like to repeat without thinking about whether it has any actual validity in, you know, actual reality. In other words, a myth.
There have always been good shops and bad shops; but like the sibling commenter said, (in the better shops) people have been working iteratively for decades and decades before the Agile Manifesto descended to earth on a ray of light, accompanied by angels.
But where agile really shines is when the problem is poorly understood and the solution not well defined.
Which problems people, again, had already been solving for decades and decades before the AM.
Such negativity
"Negativity" is not only an appropriate, but a healthy and necessary response to the never-ending stream of cargo-cult tripe and talking point nonsense that is Agile.
I did. Scrum and "agile" are far, far worse, based on what I've seen and experienced. Those "bad old days" were heaven compared to the bizarre approach currently in vogue. I will be very happy when this fad passes as well.
> But where agile really shines is when the problem is poorly understood and the solution not well defined.
That's the problem...people think that you can just use "agile" and not worry about understanding the problem and developing a solid solution. You just start slamming out code and checking back with the customer, wasting huge amounts of time and effort.
Agile introduced tactical planning and dropped precise schedule estimates, which did not hold up anyway.
If more Agile proponents would start with that instead of leading with the collaborative aspects of Agile, and drop the Scrum jargon, they would be better understood.
Making software is not like most other projects. Many more new problems to solve because if you already solved it, just reuse the code. That has practical consequences in project management, specifically: iterating toward the goal.
Collaborative task assignment for each iteration is important. But that's not why iterating toward a goal is how to do software.
OTOH, trying to interpret Agile/Scrum methodology in terms of the above is nothing more than Putting Lipstick on a Pig i.e. a valueless/useless endeavour.
The problem is your marrying it to Agile and its Manifesto. They are a bunch of platitudes, there is no exact definition far less a process/methodology, it can mean anything, everything and nothing in a given context and thus have nothing concrete to recommend themselves. As you can see in this thread, the words Agile/Scrum invoke rather strong feelings amongst the HN crowd and hence your message has gotten lost.
Instead, if you were to do a article on what/how you feel Taleb's ideas can be applied to Software Project/Risk Management, that would be an interesting and welcome read.
So much focus seems to be on anti-patterns like mining users of their information or minimizing costs by outsourcing development to cheap, unskilled contractor mills and churning out widgets to meet the mad march toward unicorn status along with all the other identical unicorny startups.
The agile crowd talks about pleasing and providing the customers with a product that meets their needs, but is that what we are doing and has agile made this worse, not better?