194 comments

[ 2.1 ms ] story [ 248 ms ] thread
this smells very much like selection bias, in that it seems pretty plausible that

* if you have a project that is Very Important To Succeed, a team might be more likely to adopt waterfall

* if you have a project you just want to trial and don't care if it fails, maybe agile practices are more likely

I don't think that's what a selection bias is. If I have two cars, and electric for getting around locally, and a guzzler for road trips, you wouldn't call it selection bias if I always chose the guzzler for the road trips.

You'd just recognize that they are better for that task.

There is choosing the right tool for the job, and then there is evaluating the tools against each other on the properties specific to the job.

If after making your tool selection you went on to author a study about how gas cars result in more frequent bathroom breaks while driving, without controlling for the length of the trip, you’d be doing what these folks did, and that’s selection bias.

    > if you have a project that is Very Important To Succeed, a team might be more likely to adopt waterfall
Yes, but that is directly in contrast with what Agile promises to do, namely helping teams to succeed.
Not if the project succeeds because it was important enough for more resources and time to be allotted.
I like to think about it like this. If you are building a skyscraper in Manhattan you need a plan. It has to be engineered. The materials must be specified and ordered long before construction starts. There are engineering practices and building codes that create a measure of safety. There is an order to the construction process. The project plan would likely be called “waterfall”. If you attempted an agile approach of dropping off a team and saying for our first sprint let’s just dig for two weeks, then we’ll plan what’s next is not going to cut it. Agile on the other hand is great for smaller, less complex efforts where the desired outcome is “fluid”. Different methodologies work best for different things.
All I can say is the big waterfall projects I was on pre-agile had excellent planning for the wrong things, and took up most of the time allocated for the project. So we were out of time, the quality and people suffered. Another project was cancelled because we took so long planning, the company lost interest.
There are more direct selection biases in the study: "projects which had clear requirements before starting development were 97% more likely to succeed" because hitting a moving target is more difficult, with all methodologies.
Also, how is success vs failure defined? Whether you implemented what you intended to on time and on budget?

That's a reasonable definition, but it doesn't capture whether the project actually achieved the business goals it set out to achieve. For many projects, it's better to fail quickly than to succeed and then discover afterwards that it wasn't worth doing the project.

>Even though the research commissioned by consultancy Engprax could be seen as a thinly veiled plug for Impact Engineering methodology,

Aaaaaand close tab

Clear requirements are needed regardless of methodology. The simpler (briefer) the requirements, the more likely to be understood and followed. Iterative development processes can leverage more concise (and thus clearer) requirements

> One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.

Also, if a project doesnt have requirements, how can it either fail or succeed?

EDIT: agile is one type of iterative, and iterative exists in contrast to waterfall. go ahead flame away :)

I can count on one hand the number of "agile" projects I've seen that were actually iterative, regardless of what the manifesto says. It would be a mistake to confuse what this article is talking about (projects that profess to be agile) with anything based on small, self-organizing teams.
Clear requirements without an artifact to test against is impossible. That is my main focus when thinking of agile.

I think Shape Up is one of the most reasonable modern interpretations of the agile manifesto.

> However, while the Agile Manifesto might have its problems, those stem more from its implementation rather than the principles themselves.

Is the author an Agile coach? This is the classic response when people observe that Agile has failed to transform their team: you're doing it wrong, the problem is you. When systems seem good in theory but tend to fail in practice, you should just try to learn what you can from them and move on to the next thing.

Honestly, 99% of the time people are doing it wrong. People are either too strict with it or too "loosey goosey" with it.

*EDIT* It doesn't have to be correct, it just has to work for the people on the team. If it works, then use it, if it doesn't then change it. That's agile.

If people can only ever get into the goldilocks 'not too strict, not too loose zone' 1% of the time, then it the process is by default not fit for purpose in almost any circumstance.

Likely the 1% of the time it works is the rare case that you have a small and exceptional team working together, in which case the project management methodology is unlikely the reason they are working well.

You don't have to, just find what works for your team and use it. It may not be perfect or exactly how it is supposed to be but if it works then use it.
(comment deleted)
What would be the Agile approach to a process with a 1% success rate?
Have the team reflect and come up with a better process, or they will be disbanded. Give them any and all the resources they ask for.
So, not tell them that they're doing the process wrong? Weird.
No, they need to come up with that on their own. What exactly "wrong" means? And why do you think you know it better than them? What if they find some better way that you didn't know about? Agile says that it needs to be sustainable also for the sponsors. Show the team that if the sponsor isn't able to sustain the next sprint, there won't be a next sprint. The process is up to them.
That certainly applies to Scrum but agile is more of a religion than a system. It's a bunch of extremely vague "principles" whose meaning is often anyone's guess.

What constitutes agile is basically a matter of opinion.

No, it really is true. Agile is widely misunderstood. The way scrum is implemented in 90+% of the cases is a complete mockery of what it was supposed to be. A lot of coaches completely miss the mark as well.

And the fundamental problem is that agile is completely incompatible with the way most enterprises are built. A truly agile team talks to the users of the product, and they have complete control and responsibility over the development process. That just doesn't happen because managers want to manage, and as much as developers would like freedom, they'd often rather not have the responsibility.

> The way scrum is implemented in 90+% of the cases is a complete mockery of what it was supposed to be.

Can you really say it is misunderstood if 90+% of people are "doing it wrong"?

Is it a useful thinking framework if there is a 90+% chance of "getting it wrong"?

I would NEVER buy a tool which advertised only a 10% success chance, so why are we falling for scrum?

Phrasing it as "doing it wrong" misses the mark.

They want the name (scrum, agile ...) without paying for it through the reorganization the would be necessary.

Electric cars have electric motors. So if you attach an electric motor to an F150 without removing the gas engine, you have an electric car. And it has terrible gas mileage.
Yes, it's misunderstood.

A lot of us were doing some form of agile before someone came up with the word agile.

I'll give you an example:

Imagine the process of creating a retro video game. First, you might write a function that clears the screen. Next, you'll get something simple drawn. Maybe just a simple block. Then you write a function that allows you to move the block with a controller. Then you swap the block with some sort of character animation. Next, you'll write a routine to draw girders. Then you'll implement gravity. Etc., etc. You'll just keep iterating until you have a well-balanced and engaging game.

That's agile. You may have had a lot of the game designed beforehand, or none. In fact, you could even create the game design documents with the same method. Make a quick first draft, get feedback, make adjustments, and repeat. And then keep going back to it whenever you notice something doesn't work or some better idea comes along during the course of development.

Originally, agile had nothing to do with rigid meeting schedules. Or "adapting to changes," or whichever buzzwords people like to throw around.

> a complete mockery of what it was supposed to be

I started programming long before "scrum" or "agile" or even "XP". One consistent observation I've made around every methodology that comes along is that management manages exactly the same way they used to manage after they roll out the methodology as they did before the methodology.

If a process fails most of the time, then the process is wrong. You can't keep blaming the implementors.
at some point ppl have to acknowledge the differences between a screw and a nail and that a hammer is just not suited for both
Most times I've been in a doomed to fail agile project, it was almost always a manager or higher who overpromised with deadlines, not accounting for real world blockers. One of said projects has insanely awful app store ratings for their mobile apps.
Agile is great when you want to see if you can make something work.

Agile is, in my opinion, terrible when you want to ensure the thing never fails.

"Ensuring the thing never fails" is a question of having plenty of tests, which can be practiced as part of an Agile methodology. In fact, it sounds more agile to write tests while (or before) you develop a feature.
Sure, if the Product Owners get that.

"Let's slow down development and make sure we've written enough tests" is a phrase I haven't heard too often in Agile.

That's very true. Same with refactoring, there's never any time set aside to clean up and reorganize the code after it's written.
I'm not really sure what to make of this article, it says itself it's a study commissioned by promoters of another methodology. If they're consultants trying to pitch their consulting services, they need these types of things to sell.

But my fundamental reaction is, what does failure mean. Because in my world view failure should be expected and accepted and learned from. It's entirely possible to spend a bunch of time avoiding every possible failure mode, and really not delivery much value at all... but we've successfully avoided the work being considered a failure.

The article also doesn't go into what kind of software was being produced, etc.

If you're building control systems for the ISS, or an interplanetary probe, of course you need requirements hammered out pretty well before you start development; you can't just continually push updates out incrementally.

But if you're building Yet Another CRUD App at your startup's third pivot, then getting every requirement written down beforehand is virtually impossible.

> But my fundamental reaction is, what does failure mean. Because in my world view failure should be expected and accepted and learned from.

That's fine in some domains. It's not acceptable in others (i.e. you cannot just try things and see if they stick).

But regardless of that, you still want to minimize failure. So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront, and this leads to higher failure rates, this is still an issue.

Agile is about reacting to changing requirements. If the requirements never change, then agile doesn't promise to be better than other methodologies. It came from a rapidly changing crashing markets in the .com bubble. How would the other methodologies fare if requirements changed two or three times in the middle of the project? That's when you need to be agile.

How did they even measure "success" if the requirements weren't given up front? What did they expect at the end?

I mean you are entirely discounting that it also allows engineers to pick their tasks and pace them as they see fit, as well as have input into the creation and prioritization of those tasks.

Everytime I hear an engineer complain about this, I ask them if you wish the PM would just write a PRD full of guesswork tasks and you execute on them unquestioningly.

Some engineers complain about having the ability to pick their tasks and have input in prioritization and planning? This has always been the most attractive part of agile for me. Self-organized team. Built around motivated individuals. Striving for technical excellence.
In forums almost every engineer that complains about agile to me then goes on to describe a deeply regimented waterfall system they are actually in.

The second common complaint is “I don’t want to do the hard work of giving input, just tell me what to code”.

This is why I hire students. I have to teach them my way before some other company ruins them forever.
I have never worked in an agile setup in which engineers pick tasks, we're given a queue and we take them in priority order with no room for choice.

How common is that really?

That's cargo scrum, very common disease. Teams without autonomy or self-organization. The queue should consist of higher level tasks, milestones, and the team should rework them into the small tasks that will actually go into sprints. Of course there is a motivation to pick the higher priority tasks first, but team members should be able to pick a task with which they can most efficiently contribute to the sprint goal, it's not always the top one.
Agile means delivering very small increments to users at a frequent pace, allowing them to provide feedback early and often. When done correctly, this eliminates a significant amount of guesswork. However, if the client (or, indeed, your own team or management) does not fully understand the process, things can go spectacularly wrong in many different ways.
We still need to explain what "agile" is. Ask yourself why do we "need" to deliver small increments at frequent pace? Isn't it to be able to react to changes?
I thought I said it. Its main purpose is to eliminate guesswork.
It's main purpose is to react to changes, and make the whole process sustainable. It's literally one of the 4 main lines of the manifesto and the second of the 12 principles, and included in the meaning of the term "agile" itself, while guesswork isn't mentioned. But I can see how trying to predict what the markets will look like in 6 months involves a lot of guesswork, which is eliminated. I hope you can see that the only reason why guesswork is tricky is that there are a lot of changes in the meantime, so the ability to react to them is the deeper, actual main purpose.
> It's not acceptable in others (i.e. you cannot just try things and see if they stick).

Why not?

> So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront, and this leads to higher failure rates, this is still an issue

That is an incorrect characterization of Agile. Maybe some "agile" companies use it as an excuse to avoid requirements, but that is definitely not part of the process.

> Why not?

If you fulfill the role of a supplier of a component in a bigger product (especially in an industrial setting), you cannot "try to build something".

People are not happy if you tell them that you want to iterate on their control system and deliver incremental value, or tell them that it's not possible in the contractually agreed way.

> That is an incorrect characterization of Agile. These are also the cases where you just have to work out overarching requirements upfront.

Ah, fair. This is a startup site so my mind was on organization that have the freedom to pivot.

It is supposed to be that new requirements come from the customer though, working closely in conjunction with engineering team reps. So it’s more like “deliver MVP control system software, test it out with actual users, then incorporate their requested changes into the next sprint.”

I’m not surprised if most so-called agile teams didn’t do that though.

Yes, that's possible for product development in a startup (even necessary, because you don't know what you need to build), and in web dev, end user facing apps,...

For many other domains, the full incremental / Agile loop is not possible, and becomes disaster if it results in skipping upfront planning:

The counterparts you need to integrate with are often not done yet. Maybe the hardware is not finalized. The end user is years away. Maybe each test run is hugely expensive.

There is no MVP here, as in the system works as expected or it doesn't.

You cannot meaningfully iterate the core software running an MRI machine, a surgical system, financial clearing, pick-and-place machines, automatic rail protection, etc. towards correctness with the end user.

My startup is a hardware company where we are our own customer. You absolutely can iterate these kinds of projects instead of top-down waterfall planning based development. It just requires dog-fooding the product and using mockups or simulations for parts that haven’t been implemented yet.
Projects with fixed, clear and unambiguous requirements (and the necessary counterpart - fixed budget) I worked on typically turned into a nightmare, because they were not actually fixed, clear and unambiguous. You could pretend everything was fine for like 80% of the allocated time, the next 80% were spent in bickering over possible interpretation of some not-so-unambiguous line in the spec, death marches, blaming each other (client and supplier, but unfortunately also within the company).

The main innovation of agile is showing failures early and thus being able to quickly react and correct course, meanwhile in the waterfall the project spent most the time in a quantum superposition without possibility for correction.

> So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront

Fundamental to agile is to have clear requirements, the time is not fixed.

With reference to the quality (scope), time, and cost triangle, each of the major methods picks one point to emphasise, another to fix (not allow to change), and another it manages the change of.

Agile wants speed so time is its emphasis, it does that by fixing cost (over a release period) and allowing changing scope, and it does that by limiting the features that go into a release. That doesn't mean that the features required should be unclear. I will deliver these 5 features out of 30 in the next 2 weeks at X dollars. If one of the features isn't ready, we'll drop it. No, I'm not doing features 6,7,8 till later. Yes, you can change you mind about what you want for later releases.

Lean emphasises quality and fixes cost while allowing time to change. You can get your burger for X dollars but you're 5th in the queue, you'll have to wait.

Traditional is about predictable cost, it (tries to) fixes time and allows change in quality to remain on budget. If everything goes to plan you should come in on time and on budget but maybe with less features or worse features than you wanted.

Of course, we know that traditional done badly produces over-budget, over-time projects that don't do what they were supposed to. It seems that agile methods are being used as cover for bad management because, as this entire thread is evidence of, there is widespread misunderstanding of the basics of project management. If you're going into an "agile" project and the requirements that you have are unclear then you need to kick up a fuss and hold the PM and the product owner to account. They should certainly be clear by the beginning of each sprint.

> what does failure mean

I've been reading that 50+% of software projects "fail" ever since I started programming in 1992 (long before the term "agile" crept into our lexicon). They consistently fail because they consistently have deadlines but no actual definitions. That was true before Agile, was true during Agile, and will be true after Agile.

I sometimes wonder if our definition of failure is flexible and moves so that half-ish of projects are below average thus failures. I once had to do a post-mortem on why a project went 5% over schedule -- that's a pretty soft definition of failure.
This is the issue of agile. It is a blanket prescription for every scenario which of course … fails.

If you are talking about customer expectations, yes, you can manage customer expectations. As long as you are honest and upfront, customers tend to be forgiving.

On the other hand, if you are talking about a core API or building block that downstream APIs will depend on, then no. You can't deliver a API that writes to the database/storage layer that only works 90% of the time.

- Based on a random survey of software engineers (who somehow had visibility to project inception & outcome)

- Doesn't define project

- Success was defined by not violating the iron triangle (so nobody knew what they were saying OR they're working on trivial "projects")

- If you read reviews for the book it's totally AI generated nonsense

This very obvious PR stunt appears to be based entirely on redefining "working software over comprehensive documentation" to mean "don't gather requirements and don't write a spec".

Here's the report itself which makes it very clear that the survey was commissioned deliberately to help promote a book about an alternative development process: https://www.engprax.com/post/268-higher-failure-rates-for-ag...

Article source seems to be mainly an ad for some agile alternative which makes the whole thing suspect.

Most of the "results" can boil down to: scope creep kills projects. That's entirely independent of whatever methodology your team follows and they don't make it clear how their "Impact Engineering" concept would address it

This is such a strange phrasing. The problem with "agile" is when it becomes manager-heavy and meeting-heavy.

The last firm I was at was ridiculous. One project had about 30 hours of meetings a week. There's no time to get any work done. It's just lopsided manager bullshit.

That's the actual criticism, not some strange dichotomy of things being successful if only it was meeting, manager and also authoritarian focused with some specs decreed by some deputized opinion makers with holier than thou thoughts they wrote down two years ago.

I can't find any link to the actual study or even a description of its methodology. The research appears to have been done for the book Impact Engineering, which introduces the new methodology, and yet the research also claims to have measured projects following that methodology. From the marketing summary here [1], it seems like what they actually did was to isolate which individual virtues of various methodologies had an impact on project success (which also goes without a definition of course). Then they choose which of those individual virtues would be promoted by their new methodology, and use projects adhering to those virtues as a proxy for scoring their new methodology. And then claim the result as statistically significant. It would be interesting to see if this was the actual methodology, because if so, that's clearly nonsensical.

Also, it would be highly relevant to know the definition of success here. Evaluating an Agile project's success as "completition of the project's initial requirements on time" would be completely asinine, given the entire point of Agile is to adapt to changing requirements.

[1]: https://www.engprax.com/post/268-higher-failure-rates-for-ag...

maybe it's for the best? fail fast and move on with what you've learned.

268% higher failure rates over waterfall is a feature not a bug.

Yea, failing fast and cheaply can be enormously valuable. In fact, of the projects that "didn't fail" one can wonder how many of them are still in a death march.
I’m sure that agile done right is great.

I’m also fairly sure most organisations don’t do Agile right.

At least within my organisation, we do not design anything up front. We’re agile.

We don’t think about proper api modelling. We’re agile.

We also do barely any testing. We’re agile.

We do rewrite the UI a dozen times based on user feedback. After all, we’re agile.

Can you point to any of the 12 principles which says that we should do "barely any testing"? I can point to some that say "deliver working software" and "it's the only valid measure of progress". If testing helps you deliver working software, then do testing, that's agile.
I think you missed the invisible /s tags to those sentences.
Sarcasm does always, without exception, sabotage a factual discussion into confusion about what people are actually saying (because that remains unclear even when you fully recognize the sarcasm).
True. Unfortunately I've seen similar arguments discussed unironically before, so I just didn't catch it here.
> At least within my organisation, we do not design anything up front. We’re agile

Agile doesn't mean forgo design completely.

> We don’t think about proper api modelling. We’re agile.

See previous point.

> We also do barely any testing. We’re agile.

It's difficult, if not impossible, to be agile without testing. If you want to move fast, you want to be confident that your latest change didn't break anything.

> We do rewrite the UI a dozen times based on user feedback. After all, we’re agile.

Sounds like you built a complete UI before any users could give you feedback. That's the opposite of agile.

I like how you quoted all the text except the part where I said “I’m sure agile done right is great”. (And that most organisations don’t do agile right).

But in the name of agile, business tends to get away with a lot of bad decisions, at least in the places I’ve worked.

Is it all as bad as it sounds? No, but the times I’ve heard we’re not thinking about design upfront under the name of agile is staggering.

Also what I dislike about the agile way of working is that whenever somebody presents any argument against it, people always say “that’s not agile”. Great. But if it is how people DO agile, that argument is moot.

It’s like REST. Rest is great. But nobody (almost nobody) actually does Rest like described in the original whitepaper. We can’t just ignore the issues resulting from this discrepancy.

Which is funny because agile allows for almost anything. Even the Scrum Guide says that the process must be adjusted to the needs of the project. It's not even an option. Yet scrum meetings are treated as if they were religious gatherings.
It's funny, the root of the idea is that Agile allows teams to change how they are doing things midstream so we can constantly improve. But I have never met a scrum master or PM who has both the authority and the desire to advocate for changing how work is done. So all it ends up being in practice is a series of rituals that people sleepwalk through.
If you're not hacking the process, you're not agile. "We are agile according to this rigidly defined process" is an oxymoron.
Dilbert captured it best: "Our boss can't judge the quality of our work, but he knows when it's late". For everybody doing programming work, there are ten people tasked with making sure it's done by the "due date", whether there's any value in that due date or not.
> "Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout."

I almost laughed out loud. Isn't that agile? :P

Every agile coach always tells me that agile is whatever works for the team.

Conveniently, if something isn’t working they dismiss it as not true agile.

Of course, all of the processes and meetings they push on to the team don’t actually work out, but they’re long gone by then.

I mean.. "science" is what works too. If something turns out to be false, we throw it out and say it's not science. This is good. I don't see a problem with this at all.

But pushing useless stuff that doesn't work and then leaving, yea that's not agile :P

Science has proven methodologies to it and those are explicitly taught, followed, and reviewed. Agile coaches offer no real guidance if the bulk of the advice is "do what works". No shit...
The methodologies were discovered after the fact though. The only real scientific method is "if you are a winner you are with us". In fact, plenty of scientific methodologies are NOT tested and probably super bad, like the modern peer review system, or grant writing/proposal/reviews.
Neither of those are methodologies for actual science, but rather publication and funding issues. The methodology most closely related to peer reviews would be idea of reproducibility. There are plenty of peer reviewed things that fail reproducibility.
Many people think it's not science if it's not peer reviewed.
> Every agile coach always tells me that agile is whatever works for the team.

Most so called agile coaches are scrum prescriptivists in my experience.

Probably the originally intended Agile [1]

These days its been devolved into micromanagement musical chairs of pain. The kind that promotes what it professes to prevent and provide.

[1] https://agilemanifesto.org/

Yea, like when people say they are Christians but are against helping the poor. Pretty clearly not Christians in any reasonable definition...
Isn't "Christian" one of those predicates where identifying as such (public profession of faith) is what makes you one? I mean unless you're going for a denomination that requires baptism.
Yea, that's what it has devolved into. At least Catholicism can protect its brand against crazy stuff like that. The agile guys should have done something like that from the start... if they had known what was coming.
Before someone says "No true scottsman", consider that "Agile" is supposed to mean:

> People over processes

and

> Responding to change over plans

And capital-A-agile is literally a textbook process.

"prevent developer burnout."

...Frankly I think all the standups & the pressure from constantly asking me if I am contributing (every single day?!) instead of trusting that I am... burns me out.

I don't mind 2-3 standups per week, 15 mins max.

But 5x 30 minute standups... Is exhausting. It feels like a lack of trust from product managers.

Oh yeah... and unqualified, zero engineering experience, diversity-hire product managers. That's super annoying-- being managed by people who did not obtain their role via the route expected of an engineer (having demonstrable engineering skills)

It’s a synchronization primitive, not the only kind available but most popular. It should not be 30min. Maybe 10.

And it’s not a test of your contribution. It’s part of being a team.

"Why is it called a stand-up meeting?"

"So we are incentivized to get it done quickly and go back to our desks."

"Ah, so if we sit down we can make the meeting longer...."

The equation of "agile = standups" has done more harm to agile (and projects using it) than anything else. I run a company and we use a lot of agile project planning methods, but threw out the standup. Especially in the era of remote work and varying timezones, it makes no sense at all.
That's all kinds of wrong. A stand-up is supposed to take 15 minutes max, and if no one is stuck with anything, it shouldn't take more than just a couple of minutes. It's not supposed to be a "status meeting" at all.
Unfortunately, in practice,that doesn't seem to happen very often.
If you read about agile, standups are supposed to be conducted among the dev team members who are working together on something, and primarily for their benefit, not as a status update to or for someone else. The PM or whoever is considered the "product owner" in scrum could listen in and help unblock issues, or report delays to other people, but not demand more information or change things up that have already been planned. Of course, it actually working this way is rare, although I have seen it.

I haven't really seen this, but a poorly performing product manager or product owner completely breaks the scrum or agile model (to the extent it works at all). They are assumed to basically know the domain and know what needs to be built at a high level, and the software requirements originate from them in the scrum model. If they don't know what the requirements should be or how to communicate them or how to collaborate with the engineers on the requirements, it is completely garbage-in-garbage-out. On the other hand, working with a product owner who is a domain expert and happy to help define software to solve their needs can be a joy.

"robust requirements engineering process"

I worked on one team that had this and it was amazing. It implicitly helped with burnout too. It was an "Agile" team, but we followed more of a scrumifall model that worked well for that project.

I've been on a bunch of Agile teams (some more than others) since then, but the requirements processes are all garbage. Then the leaders wonder why things take so long, there's so many bugs, burnout is high, etc. It's really not surprising.

Could you expand a little bit on what the "robust requirements engineering process" was like on that one team?
It was an app that was heavily focused on business processes, like routing items between different groups. We had spreadsheets and process maps for each business process. This means the business actually had to think about the possible routes it could take and the data required on each step. So the spreadsheet would tell us all the possible data exposed on the step including how it was treated (R, RW, required, formats/types, etc). It was magical.

Once a process was created the business would have to sell any process changes to the product owner by either showing it was a regulatory/legal issue, or using a cost study to show it would save money. So not back and forth or superfluous changes.

This is the type of engineering method that I try to instill in my teams. It's very dependant on good practices at every level of requirements management.
Obviously, it's a biased study that shouldn't be taken seriously, but while we're on the topic of Agile:

Consulting Agile, grew out of web development consulting firms as a defensive methodology for managing shitty stakeholders. They incorporated some genuine industry best practices, but they also have a lot there that's simply meant to blunt the worst impacts of having to deliver software in an environment where the person paying you doesn't understand the basics of software development (or project management really).

It actually works pretty well in those environments - you don't get hyper-productive engineering teams, but you do get somewhat consistent delivery and a stable enough working environment that the worst outcomes (not delivering anything at all) are avoided.

No large tech company that I'm aware of implements this type of Agile but plenty of big non-tech companies do. If you have a CIO in your company and the business refers to software engineers as 'IT' there's a good chance you do this. This is inefficient but it's probably good enough for those companies, especially if it's established, the switching costs alone would be a nightmare.

If you're using it as a startup then something has gone terribly wrong and you should seriously reevaluate what you think the future of the company will be.

Yep, and the big companies basically use Agile as an excuse for not thoroughly planning and vetting ideas. Just build it and figure it out as you go. Then rebuild everything in 2 years since you're left with outdated and unmanageable spaghetti code and you burnt out all your SMEs so they took the knowledge with them.
Do they ever rebuild things? Or just add features ontop of features till the whole thing creaks.
generally they wait until something explodes or they cannot add any features to the unmaintainable mess and they have no choice :)
Yep, this is what my experience is. It could be a cost explosion too, like using expensive mainframes and migrating to something else before the MIPS cost explodes.
Did somebody successfully migrated from a mainframe? Asking for a friend.
Yes, I've heard of some systems successfully migrating from the mainframe at my company. It tends to be a painful process. There are still some apps that need to be migrated.
Several years after a project completes when the system is finally sort of working and the end users know what to expect new management will come in at a high enough level and demand everything be rebuilt using their tech stack of choice because they need the resume entries.

Then of course you'll get a buggy second system that is back at square 0 which the business doesn't understand/hates and the developers hate because they're constantly fighting fires.

You make it sound like Waterfall had full plans and completely vetted ideas.

It didn't.

> Just build it and figure it out as you go.

That sounds like hundreds of waterfall projects that I was pulled into after they hit the 1/2 way mark and we're running behind because of unexpected scenarios.

It had better plans than the way Agile has been done on the teams I've been on. You can certainly misuse either. In my experience, planning is neglected more Agile.
Waterfall and spiral model failed in the general case for the reason of plan everything in advance, so unless it's a safety critical system I'd say Agile is still better.

I say this because no matter what methodology is used there will be spaghetti code regardless, mid tier devs like to over engineer solutions, junior devs don't know better and do whatever is first on stackoverflow.

Business execs also don't ever know what they want so change a system half way through when they see prototypes so then you still end up with Franken-systems.

To that end, you might as well start with a half spec'd system as it saves time.

I have observed that there are two kinds of Agile.

"Agile in doctrine" is the kind all of the anti-Agile crowd rail against because they've been burnt by working for some company or manager who drank the Koolaid without actually understanding what Agile is. Blindly following some consultant's Agile playbook is not Agile.

"Agile in spirit" is a bit like The Zen of Python: X is usually better than Y but there are times where it makes complete sense to do Y. In order for Agile to work, you need to have leaders and CIs who understand that. Agile emphasizes communication and flexibility. Agile is advisory, not prescriptive.

I largely agree with you, but when it comes to evaluating competing methodologies, what you refer to as “agile in spirit” turns quickly into No True Scotsman. “If you project failed you mustn’t have been doing agile correctly.”
To me the single most common evidence of failure to perform "Agile in spirit" is management holding specific teams responsible for missing a deadline after a feature or due date was changed. It shows that the management doesn't understand software development. If you think you can add features or move delivery dates to the left without consequences, you've misjudged your entire industry.
Yeah, my former wannabe-boss was very surprised to realize that splitting the company in two and selling one part during my onboarding period might have a negative effect on my productivity.

We're not doing a very good job of picking the right people to manage things; quite the opposite, in many cases whatever success is despite the leadercrap we have to deal with.

There is some virtue to your criticism, but it is also not entirely true. There are things you can check to see if a team is doing Agile for real, like, what's the last process change you experimented with to see if it improved your outcomes? What experiment failed and was dispassionately removed from the process because it didn't work out? When your team's workload nature changed (e.g., from new development to maintenance), how did you adapt your processes to match? Do your processes come up from the team and their experience or do they source from above with effectively no feedback?

At that point, if you are doing those things, there is a certain amount of validity to the criticism that if it didn't work, you really weren't doing it right. Either something external jammed you up so you weren't able to adapt and truly follow agile, your team was personally unable to execute on the adaptation for some reason (structure, personality issues, experience levels, being simply too large to be able to be this flexible because large teams simply need more structure), or the task was simply too hard in the first place for some reason (such as "it doesn't matter how agile and smart the team is, you're not getting 4 people to produce a standards-compliant browser that is also an office suite in six months").

The problem with "Agile in spirit" is that if adopt it and don't get the desired results, Agile aficionados will quickly remind you that you weren't actually doing Agile.
This is a common refrain, but I don’t think it’s true. “Agile in spirit” people will recognize the failure and look for ways to adapt so similar failures are less likely to occur. Rejecting that opportunity to reflect and adjust is where “not actually doing agile” comes into the discussion, and it doesn’t really have anything to do with “agile” at that point.

If you applied the same reasoning to any other iterative self-adjusting process, you’ll find they all tend to fail before the iterative aspect has been applied. Because that’s their nature, they’re iterative processes as a mechanism to respond to failure.

As another commenter said, it’s difficult to compare “Agile in spirit”, but I’ve found that remarkably successful

Evolving the methodology to suit the team, product, and company in an additive way seems the best outcome I’ve seen.

Rarely will I have daily meetings, we can slack. Maybe twice a week hop on a call.

For story points I’ve seen models of using Fibonacci sequence, having 4 categories rated which then convert to points, or even (less desirable) 1:1 mapping to days of work. Personally I like the 4 categories approach, it feels less subjective as a whole despite each category being subjective to a degree

And in the end, if process gets in the way of productivity, I say go for it, we’ll either go back and document what happened or refine the process later (though rarely if ever have we skipped QA for good reason)

> Evolving the methodology to suit the team, product, and company in an additive way seems the best outcome I’ve seen.

This to me is the core of agile: the people doing the work have the flexibility to adjust to match their goals and needs. So much of the consultant-industrial complex comes down to finding ways to helping business people who aren’t willing to give flexibility or clear requirements ways to not do that which don’t sound like saying no.

Big A, small a.

Agile — scrums, scrum masters, epics, stories

agile — http://agilemanifesto.org/principles.html

Agile is a bunch of rules and processes to follow.

agile is an ideal to strive towards, and never perfectly reach.

—-

Edit — this bit from the article is probably the important line

> However, while the Agile Manifesto might have its problems, those stem more from its implementation rather than the principles themselves.

Agile is not the same as agile.

At least in this humble dev’s mostly irrelevant opinion.

Good observation.

Having been on the “customer” side in such companies, I’d say the improvement is the result of more frequent interaction which improves mutual understanding.

The internal customers are forced to think through requirements in detail (albeit in pieces). The developers are forced to explain what they are doing.

Too many projects I’ve seen suffered from a lack of communication. Customers would hand things off (“just build x”, lots of customization requests that aren’t thought through). Developers would start guessing and working in the wrong direction, often going on complicated tangents as a result of the customizations).

Consulting Agile as you call it really helps both sides to understand what’s actually needed and how to get there.

It's not just a biased study, it's an article about a study of which the data is not published, so it's impossible to tell if there's any merit to it.

The excerpts of the study published in the article do not inspire confidence either:

1. Agile ("Agile Requirements Engineering") is defined as: Development starts before clear requirements, no complete specification, significant changes late in development.

I think even those who are opposed to Agile would agree that's a severely lacking if not completely incorrect characterisation of Agile development practices. It sounds more like a list of things that would cause an Agile project to fail.

2. Impact Engineering is defined as: Use of all engineering practices studied which increase success rates.

Why doesn't the author dare to give any properties of Impact Engineering up front? This sounds like he's just baiting for people to buy the book so they could learn what magical engineering practices are studied therein. It smells scammy.

The by far strongest correlate of a complete clusterfuck of a project is how often, per meeting, agile is invoked.

In fact, I've discovered you can use agile as a sort of meeting hand grenade, if you don't like the direction a meeting is headed, like they're about to decide on something stupid, you can just throw in "wait, is that agile though?" and the rest of the meeting will discuss methodology, never arriving at any sort of conclusion.

This is my methodology. Set two PMs on each other and go and do some work that generates ROI.
This is a good way to get laid off. A developer has never generated ROI in the history of finance, it was ALWAYS an MBAs idea.
somebody's got to give the MBAs their ideas.
I've prevented a lot of losses :)
what might have been usually doesn't show up in the books though
Yea. I do Fractional CTO consulting and usually have good results with developers because I focus heavily on removing bottlenecks, unhelpful processes and meetings, etc.

The challenge is always when you run into sacred cows from other levels of management.

- Hard deadlines while acknowledging we don't have enough information to commit to those deadlines

- Story points as a measure of time

- Absolutely overloaded teams being asked to deliver new features without being given time to improve underlying issues in the system...and then having to constantly stop what they are working on to deal with production issues caused by those underlying problems.

It's like clockwork sometimes and the conversations are always hard.

So kind of like the stereotypical communist meetings where they spend all their time in meetings debating each other and obsessively discuss the finer points of socialism instead of ever accomplishing anything.
I still remember one PM's response to my complaints about the number and length of meetings on one Scrum project I was on:

"Let's have a huddle on the question of whether we have too many meetings?"

(She tended to use a rising terminal in a very passive-aggressive, velvet-glove-on-iron-fist sort of way.)

I’m so so happy no one at my company has a job title called PM

That’s always a bad sign people justify with the work load generated by these same busy bodies

Oh, it's worse than you think.

My team were employees. But the PM and several developers all came from a Canadian consulting firm. We were being project-managed from the outside. And any time some piece of functionality needed to be added, the new boss kept asking, "Why don't you just use a stored procedure for that?"

In the end, the boss decided to just fire everybody and replace us with Oracle stored procedures. Not even kidding, this is not a "things that didn't happen for $400, Alex".

It seems like this is in service of promoting some new competing methodology. I also can’t find any details on the research methods, but it seems like it may be relying on software engineers’ judgment of the success/failure of projects they’ve been involved with, which…I think is pretty questionable. The reality is that there are a lot of software engineers who might describe a project that was hugely profitable as a failure because it has a shitty database schema and lots of dead code because of a last minute change.
It looks like, over a span of four days, they had 600 engineers (heavily biased towards UK engineers) answer survey questions. The vast majority of the higher failure rates were for the whether "Development starts before clear requirements, no complete specification, significant changes late in development."... which is placed under "Agile Requirements". Of course, these are things that agile recognizes as factors that create risk, and the methodology is about trying to mitigate said risk as best as possible. So, it's not an indicator of methodology but of the context. Yes, if such risk never presents itself, you are much less likely to fail, regardless of methodology.

https://www.engprax.com/post/268-higher-failure-rates-for-ag...

What about "achieving failure" that is the outcome of many non-agile projects?
Agility is implemented as a way to increase productivity. It does it with more detailed scrutiny of what workers do.

Management understood very well that it's taylorism applied to our profession. Taylorism started with measuring every movement of the workers and have their expertise removed from themselves. Daily stand-ups and points kind-of do that, even if there is no magical 4x productivity gain to ever be found, unlike was what won in manufacturing (cf. Braverman). It's a game played against workers, if they self-organized there would be no Agile.

All I know is that Agile means a lot more meetings and a lot more meetings means I'm a lot less productive.
Agile vs. what? Who out there isn't claiming to be doing Agile these days?
pretty much all the big tech cos. since i jumped to top tier companies i don't think i've even heard the word agile
I'm sorry I didn't RTFA, but I wonder if there's a correlation / causation issue here. Did the "agile" projects fail because they were agile, or because their project management methodology was prescribed by management than is higher up than appropriate. Maybe they happened to choose agile because it's the flavor de jure at the moment, but it wasn't the "agile" that was the problem exactly, as much as the undue influence of the wrong layers of leadership.
Because agile has itself been made so heavy weight that it is no more agile. You have product managers and then you have scrum masters and then there are meetings about whether how pure and religious we were about our agile practices, how pure we were, would gods of agile be pleased with every single second that we spent in past few weeks or our hearts had some impurities deep down somewhere? And no kidding, there are then meetings about those meetings and their outcomes and action items and their evaluation.

The obsession with the process itself is a sight to behold. It is now a whole cottage industry, major technology journal and websites have whole sections devoted to just Agile. Consultants, Coachs, Gurus and what not.

"One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed"

The issue with a majority of projects are the requirements aren't always clear from the start or even if they are it tends to deviate while the project is in progress. Agile tries to mitigate that by allowing teams to be able to react more easily to changing requirements.

That being said not many companies understand or implement agile correctly, unfortunately.

> Agile tries to mitigate that by allowing teams to be able to react more easily to changing requirements.

This often breaks down into a practice where maintaining requirements is not given enough priority and in very short time the actual product has deviated from the documented requirements to the point where they're basically useless and there's no clear goal for any of the features.