They are if you allow it. I haven't had to put up with waterfall since 2007. Interview carefully, deliver good work to gain trust then read them the riot act if you have to. And if you can't change your Company, change your company.
Just once in my career I'd like to work on a project that resembles the original intent of agile. So far I've had 3 jobs that "did agile" and in each case there was a long list of requirements and a fixed deadline many months in the future for each "agile" project. No one wants to hear "We don't know when we will deliver X because we're working on Y this sprint".
This has been my experience as someone who has worked at a variety of large and small companies. Do you have a (generally B2B) product to market, with fixed release increments, and features set by some external marketing / sales team? Your agile is probably going to be "that agile." Are you a small company developing a new product that you don't fully understand yet as you're still developing it? Your agile will probably be closer to the experience people are seeking.
> No one wants to hear "We don't know when we will deliver X because we're working on Y this sprint".
You need a good PM who knows how to handle this for you. I did agile correctly in a large bureaucratic organization because our PM shielded us from those requests, and basically told the business that they could prioritize the requests appropriately in the next sprint planning session.
If they showed up and participated, they got to duke it out with the other competing business units about who got priority. If they didn't show up, we told them that their work was not prioritized because they failed to attend the planning session.
- You can only ever work on projects which are finished in one sprint.
- You have no idea how, when, or even why the big initiative you're working on will finish.
It's not practical to build a business this way, what happens when one team pulls priority and a different one next week? what happens when a multi-sprint project becomes a multi-year forever project?
You can get around this if someone (such as a manager/PM/lead) converts the sprint process back to a practical roadmap/design. However not involving the sprint team in this process merely robs them of autonomy. At best, the absence of autonomy for an individual in a scrum team might be the point of the whole process.
Your first point is exactly true. We design concepts with certain goals in mind, then that gets cut down into something that takes one or two sprints of work "to see the results", which never come because it does not even cover mvp use cases. Then the rest never gets built.
Absolutely lack in long-term thinking, goals and vision.
If you have a large project, you break it down into smaller parts. If you know what needs to be done, throw it in the backlog, give it a broad estimate, sort it, and you have an estimate for about how long it will take. If you have unknowns, make resolving them a task. If you need things from the business, give them tasks.
> It's not practical to build a business this way, what happens when one team pulls priority and a different one next week?
Someone has to own the project, and have the final say on priority. If the project is at risk due to shifting priorities, you need to escalate this to them for a final decision.
> You can get around this if someone (such as a manager/PM/lead) converts the sprint process back to a practical roadmap/design. However not involving the sprint team in this process merely robs them of autonomy.
If you have everything in your backlog, you can judge velocity to come up with an estimated timeline given all of the assumption at the current state. This isn’t a promise, it’s an estimate. When the business shows up to sprint planning and sprinkles around new ideas for high priority tasks, you always remind them that this means deprioritizing other tasks which will affect timeline.
> what happens when a multi-sprint project becomes a multi-year forever project?
Yes, at the end of the project you will likely have done something very different than what you originally put in your backlog, because the business will have changed their minds 100 times. This is the point! The point is to track close to building what they want, not to build what they wanted 8 months ago.
I like to say that the whole point in the estimates is to set expectations. At that point a discussion is had with the stake holders.
If you know the velocity of the team, because it’s actually been measured (a rare thing in my experience), you will have a good idea on what is actually possible within a time frame.
> you have a large project, you break it down into smaller parts. If you know what needs to be done, throw it in the backlog, give it a broad estimate, sort it, and you have an estimate for about how long it will take. If you have unknowns, make resolving them a task. If you need things from the business, give them tasks.
What’s the difference between this and “waterfall” then?
It’s a vague todo list instead of a project plan. There are no dates or resources assigned to tasks. And you’re not committed to doing any of it yet. You might change items, add items, or remove items.
> What’s the difference between this and “waterfall” then?
The difference between this and “traditional waterfall” is that you do this continuously, simultaneously with implementation, and not just once, up-front before building anything.
(At least that's the common picture of “waterfall” -- which of course proponents of various “agile” methodologies prefer to contrast their product against -- though IIRC Brooks or someone had proposed / recorded iteration on this in parallel with implementation long before the Agile Manifesto.)
Yeah, in theory you’re technically correct, but in my experience, because “scrum masters are on the team” they’re usually a developer with little political or managerial power. I prefer having the scrum master and the PM on the team. As a pair they can navigate technical challenges and business politics effectively.
In one company that proclaimed to do “agile”, our release was suddenly halted by the business, because they came up with some new requirements. These were regulatory requirements they of course long knew about, but only now decided to throw at us, because “agile” and it’s an ongoing product. All good and well, except that they now demanded the product to be released within two weeks, because that’s when the marketing campaign went live.
Of course, our PM and scrum master, just went along and put us all into a room, to estimate and figure out how we could deliver. All our estimates were proclaimed wrong, and we had to estimate again, eventually devs just gave a number to make it fit within the two weeks.
Needless to stay, the product did not launch, and the developers got blamed for not being flexible enough. That’s how agile works in big companies, use agile as a term for everything wen it fits, use it as a way to blame others.
Scrum masters are just there to pass along communication from the business, and absorb complains from developers. It’s just another one-way street of communication.
I've seen something like this happen, and I must wonder. The only ones that can give an estimation are the people who will actually do the work. This estimation will likely be also wrong, but it's their job and their estimation. How can anyone else dispute it?
Very tempted to reply "well, how about you do it in two weeks then?". I never do, because I want to keep my job. And nobody is ever punished for missing deadlines, anyway.
They dispute it because they assume that developers are padding their estimates. You know, it will probably take 2 weeks, so double it and then double it again.
> I never do, because I want to keep my job. And nobody is ever punished for missing deadlines, anyway.
Pretty much, the general consensus amongst developers was to just go along with whatever the scrum master is saying, just so that we could at least get out of the meeting and start our work. Missing deadlines was pretty common, getting blamed was pretty common, but you won't get fired. Perhaps you'll get moved to a different team, where the same story repeats itself.
And for every story like this, where developers can't do things they know they need to because all decisions are top-down . . . I say unionize. Management needs to know that IC devs are a force to be reckoned with and not code-bots. And then use the union to get contracts that allow devs more control over decision-making.
We don't need unions for better wages. What we're often missing is respect, dignity. (Yes, sorta like sex workers.)
I’ve worked on three teams which have to some degree been agile in spirit. The first didn’t really have a planning philosophy at all, and was mostly agile in spirit by virtue of that other than when crunch time got really chaotic and stressful. The next took a minor opportunitistic rebellion to achieve, in the midst of a significant culling of management and a sympathetic view from one of the remaining leadership. My current job is thematically between the two and surprisingly adaptable even while being conscious of pressures.
All of these have been unusual work environments so I’m not sure I have reproducible insight to offer (though the second case probably is more reproducible than not at all, and I highly encourage anyone with the opportunity to throw down and choose your own planning adventure to coordinate with your team and see what shakes out; you have nothing to lose but your estimation poker hand).
There's nothing wrong with a full backlog. And also not with a deadline defining when something should be available. Sometimes such deadlines come from third parties (e.g. retirement of an already deprecated API version), a larger corporate program that requires multiple projects to deliver in sync, or opportunities based on external factors where marketing is already concentrating on a specific date, ...
My approach is the MVP way, where the team delivers everything that is necessary to meet the customers goals. All the bells an whistles can be qdded once those goals are met. If the goal can't be met within the given timeframe, then it's a problem of unrealistic expectations. But most of the times we have met the deadline while otheres with their bloated waterfall projects were late.
I recently wrote about my experiences here, and especially, what makes a great project manager who can move a project in a flexible way. I quote myself:
--------------------
What about Scrum, Agile, Kanban, PMP and CAPM?
At this point many of you will be wondering, where is the conversation about all of the formal methodologies? Some of you will think it’s strange that I’ve written a whole chapter about project management without mentioning Scrum, Agile, Kanban, PMP and CAPM.
...I have not personally noticed that credentials (such as Scrum Master or PMP) are necessarily associated with real skill as a project manager. Rather, the key things that make a great project manager tend to be more qualitative:
• Does this person have the moral convictions necessary to take a tough stand against upper management, for the benefit of the overall company?
• Does this person have the confidence necessary to fire a poor performer for the benefit of the overall team?
• Does this person have the perceptiveness to correctly read team members and know when they are lying or boasting?
Certain people gain these attributes as they gain experience, but these attributes tend to be things that are not emphasized in formal credentialed courses. Strong moral convictions tend to be picked up in childhood and are only learned slowly in adulthood. In formal credentialed courses the focus on various rote processes tends to shift the focus away from the things that are really important.
Described my current gig to a T except our useless project manager outright said he doesn't believe in agile and runs everything from excel. He came from a defense contractor and it shows. Everything has fallen apart, we've been late on every release, lost huge contracts and now we're outsourcing work to India, but, inexplicably, he still has two brand new cars and just bought a hobby farm. The bastard.
The only thing he did in response to failure was to pivot us to feature driven development, and then he screwed that up by not clearly defining teams and muddying responsibilities.
Pick your jobs carefully or be prepared to walk away :(
A funny enough YouTube video (comedy) and really, it kind of reminds me of the agile methodology in the large company I worked at ... https://www.youtube.com/watch?v=bB340S0tGf8
I mean, yes, thats literally what it is. Hilariously we attribute waterfall from Royce[0] in which he visually shows a waterfall like process, only to describe it as an iterative process (and essentially agile scrum).
> still makes projects late.
But under the concepts of the agile manifesto there is no concept as "late" so I'm not sure why the author refers to a project as being late. The whole point of an agile process is to be able to iterate quickly. 2 weeks is a generally accepted timeframe in which you can revert attention if what you develop is incorrect. So whats the gripe?
Running waterfall on a two week cadence burns out engineers by forbidding them from doing any work with a timeline greater than one week.
After all, half a week is burned, minimum, at the beginning and end of the sprint on process paperwork and screaming at disconnected Zoom windows.
Next sprint, their week of work is frequently reverted or irreparably hacked to meet an arbitrary deadline.
Also, the product sucks, but management metrics ensure people that focus on shuffling the titanic deck chairs instead of fixing it get promoted.
If a group of engineers puts in a heroic two month effort to save the project, their metrics tank, and they end up reporting to the first group until their options run out or they are laid off.
Metrics. The way the MBAs will throw the world (further) into a meat grinder.
My wife is a teacher and she was telling me that a panel who was responsible for hiring a new permanent teacher was "not allowed" to consider their prior knowledge of the applicants' teaching abilities, and had to go with the teacher who best answered the Department of Education's paperwork questions.
They chose (against their personal better judgement) the worse of the two teachers because "on paper" they appeared better.
I'm currently struggling to find a job (my current contract ends in a couple of weeks), and I just don't have the ability to be the level of ass-kiss dishonest, straight-laced, official textbook answers that seem to be what lands the fish. I'll get something eventually, and that's all I want, but it really feels like the world is tuned to the sizzle even when you can smell that the steak is rotting.
With Agile, I find more ceremonies than actually work - Planning, estimations, Re-Planning, Cross Prioritization, Retro, Ticket Updates, Blockers, Blah, blah...
Our daily "15-minute" stand ups regularly tip over to 30+ minutes. This should be a sign that the team needs to be split up, as more than half of the stand-up is discussion of topics that are totally unrelated to my context.
It stops my momentum, annoys me, which ruins my motivation.
> more than half of the stand-up is discussion of topics that are totally unrelated to my context.
or the team is doing too much at a time, and delegating different "projects" to different people in the name of efficiency.
I think that's actually wrong - the whole team should work on the "same" task. For example, coding a module could be done together. It might seem like only 1 person is writing code, and everyone else is over the shoulder looking, but you save knowledge transfer times in the future, and you save code review time (you do review at the same time when everyone is there).
I had a manager that gave me shit for not being verbose enough. And our stand-ups were already in that 20+ minute territory. Why do we have a Jira board if no one wants to read it? It's all right there! No blockers. Next.
I stopped going to stand-up and simply posted a Slack thread for status. The people who didn't like it are the people who would ramble on at stand-up, uncomfortable being curt or ending a meeting early. They'd ask "anyone have anything else to talk about?" just to fill up extra time. No preparation ahead of time, so a lot of "oh yeah and also..."
Don't forget that it wastes team's energy before, during and after. We would have 3~4 hour estimations & planning and it would just completely wipe everybody out. On top of that we had to track time to the minute and were thrown into debt because we owed "extra hours". On top of this our estimation would always be off due to people being burnt out and it would mean more hours were owed. This was a salaried position in Canada btw a long time ago and I don't know why there is so many of these small nanny startups that think they own people because they pay them a salary.
These used to wipe me out, but ever since I stopped paying attention to them this issue got solved. I use them for exercising, cooking, reading HN, etc while half listening to it like it would be radio.
And literally nothing was lost in the process. I am as productive as before, know as little about product we are developing as before.
really? being on conference call for 3 hour straight, talking, thinking doesn't tire you out? the whole team takes breaks between and after too. even then I feel tired from such a lengthy session.
> being on conference call for 3 hour straight, talking, thinking doesn't tire you out
Being there straight, talking, thinking and listening would tire me out. But, I only half listen, dont focus on parts that dont seem relevant and get rest as I please.
These meetings were super tiring for me when I was trying to learn requirements from them. But the result was that I did not learned much anyway and not paying attention made no change.
I think one of the main reasons for wagile is that software development is often seen as capital expenditure rather than operational expenditure (or R&D, although companies love saying it's all R&D to the tax dept). CapEx assumes a fixed(ish) upfront cost and then an ongoing maintenance cost afterwards - which means that you need to be able to estimate the upfront expenditure.
In reality, software development doesn't fit neatly into this model. It's part R&D, part operational expense and part capital expense all at once (even in maintenance phase) - but it's impossible to distinguish which parts are which. The resulting product's value is constantly in flux, which means it is difficult to plan depreciation like a traditional asset.
Agile will always fail unless the way software is accounted for changes too, because the people at the top are looking at budget spreadsheets, not JIRA boards/burndown charts.
Basically a new thing has been introduced into the world and the world has tried to fit it into an existing paradigm and when it doesn't fit, well let's invent a new paradigm that looks like the old one but better...
The problem isn't how the business solves software. It's how software dissolves the business.
> I think one of the main reasons for wagile is that software development is often seen as capital expenditure rather than operational expenditure
Depending on where you are, you might find that allocating expenses as capitalisable is much more tax-efficient, hence the desire from the business (finance, specifically) to treat it that way.
Even 10-15 years ago I was at a small Thoughtworks event where Martin Fowler was speaking and back then he was using the term “Water-Scrum-Fall” saying most Agile projects he’s seen are waterfall projects wrapped in to two week sprints. At the time Agile/Scrum didn’t have the popularity and only just gaining traction at larger companies.
Today everywhere I work is now “Agile”, doing scrum or similar. It’s all lip service, it’s just another process and ceremonies over the actual spirit and ethos of the original manifesto.
The teams I’ve worked on who didn’t practice agile where actually the most “agile” in the spirit of the manifesto. The teams that did agile, by the book, with the ceremonies and all, dogmatically, doubling down on it, where the least agile in the spirit of the manifesto and not that productive or creative, often micromanaging every tiny detail in a Jira ticket.
Having worked in actual "waterfall" (which is just a perjorative term) environment in the defense industry long ago, scrum is quite a bit worse in overhead, amount of nonsense, waste, inflexibility, etc.
Agile/scrum is a codeword for "we don't want to / know how to / haven't ever experienced long term planning, but we're going to do it anyway".
Yup. If you're at a shop that is openly waterfall, they do the requirements gathering and design/prototype work up front so the planning is based on something approaching reality.
"Scrum" has become waterfall without the pre-work so it often descends into iteratively pulling estimates out of your asses.
Waterfall was a problem too for those who focused on it to the exclusion of "individuals and interactions". Planning is good, being attached to a plan is not.
I feel you, and I'm not in software, nominally I'm a Product Owner from the business side. As a business we didn't understand the problem we are facing yet, we have no strategy how to tackle the problem in limited time and with the limited resources we have. And since two weeks Dev Ops is pushing me to come up with a headcount estimate to develop the software solutions we need. Because, of course, upper management needs resource estimates to come up with a reduced budget. It's a mess. And our BPO is fully on-board with this, because career opportunities. And the other POs have close to no experience. The DevOps head never had any IT experience before he became DevOps. We have a whole bunch scrum masters, agile coaches and the like. We have zero functional consultants, and we are looking at an ERP implementation project here. Oh, and the last developer we had on the project didn't see the difference between a technical and a business go-live or the need to conduct proper use-case based user acceptance tests. Or proper requirements definitions, because, you know, agile. It's a mess.
I have done CMMC3 waterfall development, though the maturity model certainly did not dictate that. I have also seen agile done in the way you describe. I have also been on teams and run teams that used agile ceremonies to foster communication and teamwork and set a cadence, while long term planning was going on elsewhere. The two are not mutually exclusive, and even mediocre agile is miles better for delivering working software than bad waterfall.
Scrum was designed for low trust environments where you have to constantly demonstrate that you are worth your salary, ie consultants. Scrum gives a constant stream of status updates to show you make progress and blocker reports to cover you ass and shift blame to the client, that is very important for consultants but shouldn't be needed in high trust environments.
Agile is also for projects where the customer doesn't know what they want exactly, but they do have an idea what problem they want solved. With an agile project they can pull the plug if stuff doesn't work and they can pivot (within reason) to new features that are discovered during the process.
I have friend in a defence sector waterfall project. It was meticulously planned years ago, the design was locked. Production was started. Everyone knows it'll get scrapped for being out of date when it's finished on schedule in 2024. But they need to finish it anyway, because the deal is hard-locked and there is a big penalty for not delivering or delivering late.
Since the system your friend is implementing is already outdated... does that mean that someone else on the defense sector created a better system with a faster development process? Or did the competing developers simply make a better guess at what they should develop in their own own waterfall process?
Military secrets gotten from a drunken friend and all.
But let's say that the problem they're solving doesn't exist anymore, tech advanced and you can get 90% of the same functionality with off the shelf stuff =)
To be clear you and the comment above are describing two different problems. "Water-Scrum-Fall" implies that too much planning was already done up front and the work is just divided into arbitrary 2 week milestones. You are describing a problem where product thinks they don't need to plan long term because they are using "agile". A good product team is somewhere in the middle. A long term user/product vision without getting too specific in implementation or feature priority so that the team can pivot as needed without eating too much cost (there is still a cost to a pivot).
Yeah, at my current place agile is also used for the hardware development side of things. And in all cases, hardware and software, it is, IMHO, just a pure excuse for non-existent requirements management (hardware) or no clue what the business actually needs (software).
Scrum is ritual heavy process by definition. There is no way to do scrum without being heavy on process or dogmatic. The moment you start doing that, your process cease to he Scrum (and any advocate will point at your difference against process as reason why it issues are not cause by Scrum and why it is not scrum)
Honestly, from all I've heard and read about "SAFe", it feels like "Take the already-not-very-agile Scrum, and squeeze the last vestiges of agility out of it to make it Enterprise Certified".
Scrum is waterfall in disguise and it always has been. I have never seen scrum be anything but a catastrophe, or a token gesture to appease those who would replace accountability with process.
I have seen scrum turn into hell more then once and it had zero to do with rest of organization doing anything. Literally each time, there was no evil customer pressuring us too much or making bad demands, nothing like that.
Thing is, Scrum doesn't work for most of the organization.
Sales use a sales funnel as their guiding organizational tool. IT operations and support do reactionary work on incoming tickets, and some rather routine and planned projects. HR is almost all routine paperpushing. Legal has to take the time it takes to answer the questions they get, because they have to answer correctly. Market analysis is timeboxed analysis - a spike in agile lingo. Hardware design is innately waterfall. Hardware production is just mindless repetitiveness as fast as you can go... and so on...
I found Kanban a lot better for this. You have a sorted list of tasks, and you can only work on a small amount of them at once. A lot less time is spent estimating tasks to fit them in a pointless two week bucket.
Backlog grooming can be done asynchronously, constantly. The PM can push a task to the top without making the developers work harder this sprint. They can go faster or get delayed, because they don't have to meet the arbitrary requirements of a sprint.
>The teams I’ve worked on who didn’t practice agile where actually the most “agile” in the spirit of the manifesto. The teams that did agile, by the book, with the ceremonies and all, dogmatically, doubling down on it, where the least agile in the spirit of the manifesto and not that productive or creative, often micromanaging every tiny detail in a Jira ticket.
Personally I have noticed opposite scrum works if you run it by the book first and then start tinkering process based on retros and this "agile" does not work where you implement something that kinda looks scrum or kanban and kinda mess major part like estimates are hours etc. It is not ideal but low trust or multiple stakeholders it works.
I'll add that empowering the development team via retrospectives and actual authority while having a single interface to the business via product owner are huge contributors to the agility of a team in my opinion.
I don't know if I agree with every sentence in TFA, but this is obviously true:
> "Soon there will be a new project methodology that will promise to deliver software projects on time and on budget. It won’t work, but we will all have fun learning new terms and joining a new methodology cult."
I think everyone here will agree this will happen. Again, and again, and again.
For the next on we need something that uses the terms "back-propagation" (blah ... self-correcting planning ... blah) and "blockchain" (blah ... ensures to meet estimates ... blah).
I think it'll only work if the people paying for the product want an agile project.
And that they understand agile to mean the same thing as the engineers. That is to say to small deliverables within a short period. Review the small deliverable with client, make adjustments, then repeat until final product is done. Basically get things working a small piece at a time. Change directions if needed. Don't boil the ocean.
What usually happens is that the clients take "agile" to mean changes on-the-fly AND quick delivery of final product. They also usually prefer the waterfall deadlines, because then can line things up for Q2, Q3, etc.
I think a big reason for the disconnect is the word "agile".
Witnessed agile/DevOps degenerate into resource constraint paralysis far too often. In general, such approaches are not appropriate for every project... even if you are not the one paying for the billable hours.
The Module pattern design built into many languages like Python is perhaps why some folks get away with it for awhile. ;)
I think the reason why Agile worked, back in the days of the Manifesto authors, is not because of any specific thing in that Manifesto ... but because you had a small team that could decide how they wanted to work. This is antithetical to most corporate management.
Also presumably they were experienced engineers who could be trusted to make good decisions. But not all teams are like that. I have worked with many engineers who are completely lost, or will routinely make the worst decision possible, any time the requirements are exactly spelled out for them.
Little a agile was supposed to be an iterative system that the team could adopt and adapt to their needs. These forms were intended to make developers more effective.
The manifesto was so small and stark because those were the few things that all of the manifesto signatories could agree on. They were trying to find consensus on principles that made products successful.
Big A Agile is much more rigid and top-down, marketed to management. Success there means selling training and certifications. Nothing more.
Also, they were thought-leaders with an interest in exploring better ways of working, and presenting what they found. As such, they surely adapted themselves and bent some rules. They were not following recipes in some book about Agile. But if you look at agile coaches preaching on LinkedIn, there is an endless stream of posts about the true way to do agile.
Small teams can only get a small amount of work done. How long would a team of 7 take to deliver something like Word or GNU/linux (both not just the kernel, but everything involves in a OS/distribution )
I call it 'Wagile'. Everyone uses all the agile buzzwords yet all the project management is focussed on estimates and hard deadlines.
To me the most disturbing part is people read the 'Agile Manifesto' and say, "yep, that's what we do", yet for every item it is the exact opposite. For example, "people over process" seems to be addressed by considering every team member identical and focussing purely on the tickets.
I am to the point now where I think corporate 'Wagile' is the Agile process and I simply don't understand what it was years ago.
What you describe has nothing to do with waterfall and is simply agile. Literally nothing in what is supposed to be waterfall prevents specialization or treating people nicely. However, multiple agile methodologies explicitly prevent specialization and explicitly are tickers focused.
They do it for a reason, to be fair. But, it is ridiculous that for how long agile existed, every single uncomfortable or dysfunctional thing about any agile process gets blamed on waterfall.
No. These are all issues with agile methodologies. Agile manifesto is just feel good document meant to make you buy into the process in abstract. It is inspirational, but that is all there is to it.
So, it seems to me, you are saying that we should not be taking any notice of anything clearly stated in the Agile Manifesto. Next time someone says to me "Don't bother telling me about the real work, just keep the ticket time spent updated", I'll keep that in mind. (A real quote from an Professional Agile Practitioner to me).
Waterfall - the mythical process that if ever existed it was 30 years ago, but we bring it up when we want to make our 30 years old process sound hip. And when we don't want to say agile both often massive sux and also is the normal way of doing things for decades.
I wonder if there’s one of those observational “laws” that states that any idea, no matter how flexible and open-ended, must eventually harden into a rigid unthinking cargo cult.
As a corollary, all ideologies end up as self-parody, contradicting their original intentions in sometimes egregious ways.
> The retrospectives are gone
Oh, they’re still there, they take like seventy-five minutes, they do produce some good observations, aired-out grievances, and action items, the latter of which are never ever followed upon in full or even half.
John List, an economist, has a new book out called The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale [1]. I think it address your point in a more general way and has identified some failure modes that lead to a negative end state. [1] is a summary of the ideas from the author.
Phenomenal article, but I'm frustrated by his focus on Google, because it implies that you can only do 'Good Agile' when you have more money than God and no deadline. Although, perhaps that was his point.
In that timeframe a lot of his blog posts related to the experience of moving to Google - eg comparisons with Amazon etc.
It's now apparent that Google is both very a different place in recent years, AND that all the "Good Agile" stuff from 15 yrs ago kinda screwed most of their products and reputation long term.
From my experience, there is usually a fundamental lack of trust and communication problems behind all of these project management approaches and initiatives. Ultimately it all stems from anxiety about the potential for a date to be slipped or something to not get delivered. It's not about peers, it's about tracking everything in a ticketing system so nothing is missed.
In practice, the only thing that ever really happens and I've never see anything deviate from this is as follows:
1. Product team wants something feature or new product
2. A plan is formulated to get there, often by a certain date with engineers present.
3. An individual or team is entrusted with the delivery of said imitative
4. People go off to do the work to make it happen. Often for months, often without a ticket associated with the work, they just want to get their work done.
5. If you're lucky, the senior resources on the project will flag any issues as they arrive or foresee them.
6. A the end, if everyone isn't too burned out, it's nice to have some time of retrospective about how things went.
Ideally through this process, there is a lot of open and healthy conversation about the progress and status of the work. Anything outside of this is really almost always just cruft, a distraction and leads to bad feelings because the people who need to get the work done are distracted with silly things like poorly ran daily stand ups.
Sometimes things make it so the project is late, no level of agility or number of waterfalls can avoid the sometimes inevitable so it's better if product acts under the assumption things might not always be on time and have a more flexible marketing schedule etc.
None of this would be a problem, except now there's a lot of anxiety about not "doing agile", or waterfall or something...
Sounds like someone is working at a growing company.
Agile is great when you can be agile. Not every busyness fits into that mold. Some software projects are big, regulated, public, etc. They need more structure.
For instance, you don’t update the cloud provider at a 500 company with agile. It takes planning, coordination, contingency plans, and often 2-3+ year timeline. Agile is for smaller companies or smaller projects that can be broken up into sprint size pieces. Many are not.
"Agile is for smaller companies or smaller projects that can be broken up into sprint size pieces. Many are not."
Just not true. I know it's hard to know if you haven't seen it but I assure you it's possible and much for effective for a large team to be truly agile
i call cap too. agileis the way to go, but requires focus, lots of communication and a concrete foundation of trust. that does in fact not scale well, in that angle.
You're just applying the concept at the wrong scale.
Honestly I hate the term agile, it should really be "open communication", and it can apply on even a daily timeline
Waterfall "style" is a directive comes from on high to get every service running on the new cloud provider, and 2 years later those on high find out we're still 2 years away from completion and along the way we spent months working on tiny aspects of it because the directive couldn't capture every nuance.
Agile "style" is where you get to return feedback to "on high" before that point. If there's a service that represents 5% of our revenue but required 30% of the effort, you get a chance to push back and talk to stakeholders before committing that.
-
It's really the idea of "further unraveling uncertainty as you go along instead of assuming you accounted for all of it at the start" that makes agile useful, and that can work on any project of any scale.
No matter how rigid the parameters of a project, at some level there's uncertainty (otherwise you'd have perfect estimates and nothing would ever be late). So better that "on high" learns more details about that uncertainty as you go along, and you get input on how to handle that uncertainty... than in a giant "we're flailing" moment later down the line.
I think people focus too much on the whole "no committing to deadlines" idea, when that is not at all central to the value proposition, and honestly feels like a feel good platitude that doesn't actually work when it meets reality regardless of how small you are
If you’ve worked in software for longer than a single project, you already know that software is impossible to precisely estimate and no two projects are alike. Anyone who claims that waterfall requires perfect estimates is using the term incorrectly.
_All_ software projects include an element of discovery as new details are discovered, technical challenges are encountered, etc. To think otherwise is naive.
Waterfall in software means there are fixed timelines and a fixed starting scope to work against. This helps align the efforts of multiple teams that have their own deliverables and timelines. As a project progresses, the timelines and scope are reassessed. This is true of any methodology.
Full agility is when you can pivot quickly to new opportunities. This flexibility requires a small scale. Most larger companies have too much going on to allow for something like this yet they also allow for agility. For instance, a security incident is not going to be neglected because of an ongoing project. Waterfall in software is not what the author makes it out to be.
No one said anything about waterfall requiring perfect estimates?
No one proposed or implied that any software doesn't have an element of discovery?
Obviously there's wiggle room for all of these terms but you're defining waterfall so loosely that you didn't describe waterfall at all (nothing about fixed starting scope and fixed timelines requires waterfall...)
Then on the flip side you're defining agile so precisely that almost no one can meet it. I mean what is "full agility"?
It feels like you invented a term for a theoretical "perfect" implementation of agility, but my entire point is that you don't need to go off the deep end with the concept of agile to reap the benefits.
-
Waterfall and agile mentalities don't need to be mutually exclusive, which is exactly what the article is about.
I lead projects on self-driving cars. When the NHTSA has hard requirements we don't get to go back and ask them to reconsider, but that doesn't mean that within implementing long-tail features we can't be have an agile process for making sure the teams implementing parts of those features aren't blindly banging away with no iteration on their approach.
Waterfall and agile should just be means to an end. The weird want some people have for one to be the "valid" approach has never made sense to me.
I figure it's implied "no one in this conversation", since outside of this conversation agile and waterfall have both been described in every possible way.
Waterfall "as it should be" was presented as a flawed model: In reality there is no competent team, using waterfall or otherwise, that relies perfect estimates and I wouldn't insult anyone by claiming they're doing so.
> In reality there is no competent team, using waterfall or otherwise, that relies perfect estimates and I wouldn't insult anyone by claiming they're doing so.
The waterfall is process where you start by collecting requirements, make estimation and sign contract with set deadline. The rest is development in one go, then testing and then you give result to customer. So yes, it relies on precise estimations. Whatever else you have in mind is not waterfall process. Waterfall does not prescribe how exactly you make estimation nor how do you communicate inside the team.
It has disadvantages which is why no one was using it for decades. Except as straw process to blame when other processes fail or dont prevent dysfunctions. So, management tries to implement scrum, it fails or turns into hell. Then next guy will try to implement Scrum or whatever, claiming he is bringing agile to save us from waterfall. But that is just a way to avoid saying that it was scrum that failed and repeating exact same cycle.
There is a difference between a perfect estimate and what you're describing.
Again, literally no competent stakeholder anywhere is expecting a perfect estimate.
Both agile and waterfall both aim for precise estimates.
With agile you're still aiming to have roughly correct story points, or having roughly enough work for your sprint. But you're moving the yardstick on how often you estimate and how you handle missing those estimates.
All of which is why again, I don't see the point in being so rigid in where one ends and the other begins. Waterfall has gone from a punching bag to a real process people follow by adjustment. People seem to miss that while the "OG" waterfall diagram was supposed to be flawed, even in the same document practical improvements were introduced.
> Again, literally no competent stakeholder anywhere is expecting a perfect estimate.
Incompetent stakeholders are real world phenomenon. In this sense, even stakeholders that are otherwise smart people and competent in other areas often fails in exact this way. Real world companies pay fines for being late. Real world customers without contractual guarantees end up paying more then project is worth too.
> Waterfall has gone from a punching bag to a real process people follow by adjustment.
This is the thing I disagree with strongly. Waterfall is punching bag and is not used in practice at all. It was past being used when I was young. Waterfall was also far from the only process that existed before agile came.
Iterative development process existed long before agile came to be. Agile means nothing and everything these days. Waterfall however is canvas where people project their grievances with other processes. That is why it is undefinable in these threads - it is either about communication, or people being treated bad, or about iteration or host of other practices that companies combine together.
The worst part about agile is it has become whatever each company wants it to be. Product owners dominate the pace and direction of development often to the detriment of engineering quality. At least in waterfall model, there is some one like a program manager or delivery manager taking overall responsibility for delivery and coordinating dates with stakeholders. With agile and scrum, there is no one taking responsibility for achieving things. Often product owners end up doing this for which they might or might not qualify for.
Worse even, 'agile' gets used as an excuse for project managers and architects not to document requirements or plan properly, even in situations where all the goals can be known ahead of time, because that's a lot of work.
It's also rife with "It's not working? Clearly what we need is more interference, more monitoring by management, hire more 'agile coaches' to inflict more 'ceremonies' on developers and use even more of their time."
Never mind that planning is always thrown out of the window because priorities shift. Never mind that none of the changes that were identified as necessary in the retro get any sort of management buy-in, rendering it pointless. Never mind that 'stand-up' has become a management-run progress-report session. Never mind that this kills the feedback loop and developer-lead benefits that are supposed to come from this. No! Do more agile! Whatever the hell that means! Do it harder!
> Worse even, 'agile' gets used as an excuse for project managers and architects not to document requirements or plan properly, even in situations where all the goals can be known ahead of time, because that's a lot of work.
What's the complaint? The higher ups have a scapegoat, so they're happy. And you can rejoice in it not being you or any of your friends!
The problem is that programmers answer to idiots, but nevertheless those idiots have certain reptilian sensibilities in which they exceed us, and a consequence of this is that they have a knack for zero-sum power grabs and pissing contests. So, even though those people are 30-50 IQ points below us, they nevertheless end up remaining on top, and there isn't really much we can do about it unless we're prepared to burn down a whole socioeconomic system (which I am, but most people aren't there yet).
"Sprints" are supposed to be humiliating. The very message is that you're not trusted with even two fucking weeks of your own (!!) working time. It could not be clearer. If you work in sprints, it's because the higher-ups think you're a child and a loser, and you'll never be able to overcome that negative inference, because if you perform poorly you will confirm it and if you perform well, you will confirm that their micromanagement actually works--there is no winning.
Also, "Waterfall" never really existed. It was a straw man invented to sell this Agile bukkake.
In any case, a number of the dysfunctions are, in effect, intentional. Standups that go on for 45+ minutes? Long meetings are a classic way for managers to punish perceived underperformers (or, in Agile terms, "impediments") when they're not entirely sure who problem player is. The theory is that the rest of
"the team" (I put this term in quotes because coworkers in corporate aren't an actual team--that's just management speak--but are often pitted against each other) will get sick of the incessant meetings and rat out the underperformer who is causing this wastage of time. The humiliations of Jira and "user stories" aren't bureaucratic accidents that occurred in good faith; they hurt because they're supposed to hurt.
> It was a straw man invented to sell this Agile bukkake.
That part is colorful, but retconning. A fellow named Royce coined the term in 1970 and used it as a strawman for common project iteration organization (V-model?) a the time.
I agree with most of the spirit of the rest of the parent's screed, though not with all the details.
Having managed agile projects at startups and big tech, there's a few problems that were common:
1) lack of customer feedback loop - agile is meant to ship features quickly to customers, get feedback, and iterate/enhance/pivot based on their feedback. Often times the projects were simply things that "had to be done" and the whole agile process was not necessary.
2) inaccurate estimation - projects would start with asking engineers to estimate "t-shirt size" efforts on user stories, and what happened in reality is the projects always ended up taking longer than expected due to unknowns, dependencies etc. I've only worked with a few senior engineers who can accurately estimate, but junior engineers starting their careers in a big company, it's a tall order to ask them to be accurate.
#1 is I think often the biggest issue, or rather a direct easy to spot symptom of the central issue which is that "agile" is just blanket applied as a development methodology to everything, including many projects that have no business being agile, while the project requirements aren't agile at all.
Agile is better thought of as a product management than a development methodology. It is meant to be agile in developing your product to iterate quickly and respond to feedback from real users. It's a great fit after a MVP is released to users to iterate on their interaction in a responsive way. It's not an ideology to be followed religiously for all projects with all constraints (particularly those where those paying for it want specific features/dates are a poor fit).
- If a given project is only agile on the development side and not also the project and product management/sponsor side, there's going to be a ton of friction.
- If all projects use the same methodologies (be it agile or waterfall or some other new hip flavor) to develop regardless of their scope and needs and sponsor, there's going to be a ton of friction.
- If your teams adjust their approach to meet the needs of the project at hand, and all aspects of the team for a given project are in sync with that approach not just developers, you just may be able to glean a relatively smooth experience.
I've found that there's no single method that works well over time. Agile gives you crappy code and a stressed team and water fall gives you late projects. I generalize.
People want to blame the method of project management but it comes down to the team and the project manager. There's a reason why project managers get paid so well in every field. They know how to manage a team while bringing in the project on time. It's a hard job. What method is used is not as important.
209 comments
[ 3.8 ms ] story [ 245 ms ] threadYou need a good PM who knows how to handle this for you. I did agile correctly in a large bureaucratic organization because our PM shielded us from those requests, and basically told the business that they could prioritize the requests appropriately in the next sprint planning session.
If they showed up and participated, they got to duke it out with the other competing business units about who got priority. If they didn't show up, we told them that their work was not prioritized because they failed to attend the planning session.
- You can only ever work on projects which are finished in one sprint.
- You have no idea how, when, or even why the big initiative you're working on will finish.
It's not practical to build a business this way, what happens when one team pulls priority and a different one next week? what happens when a multi-sprint project becomes a multi-year forever project?
You can get around this if someone (such as a manager/PM/lead) converts the sprint process back to a practical roadmap/design. However not involving the sprint team in this process merely robs them of autonomy. At best, the absence of autonomy for an individual in a scrum team might be the point of the whole process.
Absolutely lack in long-term thinking, goals and vision.
> It's not practical to build a business this way, what happens when one team pulls priority and a different one next week?
Someone has to own the project, and have the final say on priority. If the project is at risk due to shifting priorities, you need to escalate this to them for a final decision.
> You can get around this if someone (such as a manager/PM/lead) converts the sprint process back to a practical roadmap/design. However not involving the sprint team in this process merely robs them of autonomy.
If you have everything in your backlog, you can judge velocity to come up with an estimated timeline given all of the assumption at the current state. This isn’t a promise, it’s an estimate. When the business shows up to sprint planning and sprinkles around new ideas for high priority tasks, you always remind them that this means deprioritizing other tasks which will affect timeline.
> what happens when a multi-sprint project becomes a multi-year forever project?
Yes, at the end of the project you will likely have done something very different than what you originally put in your backlog, because the business will have changed their minds 100 times. This is the point! The point is to track close to building what they want, not to build what they wanted 8 months ago.
I like to say that the whole point in the estimates is to set expectations. At that point a discussion is had with the stake holders.
If you know the velocity of the team, because it’s actually been measured (a rare thing in my experience), you will have a good idea on what is actually possible within a time frame.
What’s the difference between this and “waterfall” then?
The difference between this and “traditional waterfall” is that you do this continuously, simultaneously with implementation, and not just once, up-front before building anything.
(At least that's the common picture of “waterfall” -- which of course proponents of various “agile” methodologies prefer to contrast their product against -- though IIRC Brooks or someone had proposed / recorded iteration on this in parallel with implementation long before the Agile Manifesto.)
Filter out the crazy (which can be the entirety) and then pass down to the team the best way you can figure to move things forward.
I managed to do this for a while. It's hard on the ol' mental health though.
<insert meme of soldier protecting sleeping child>[0]
The SM is supposed to be part of the team, not part of management.
[0] https://pics.me.me/the-silent-protector-sales-department-man...
Of course, our PM and scrum master, just went along and put us all into a room, to estimate and figure out how we could deliver. All our estimates were proclaimed wrong, and we had to estimate again, eventually devs just gave a number to make it fit within the two weeks.
Needless to stay, the product did not launch, and the developers got blamed for not being flexible enough. That’s how agile works in big companies, use agile as a term for everything wen it fits, use it as a way to blame others. Scrum masters are just there to pass along communication from the business, and absorb complains from developers. It’s just another one-way street of communication.
I've seen something like this happen, and I must wonder. The only ones that can give an estimation are the people who will actually do the work. This estimation will likely be also wrong, but it's their job and their estimation. How can anyone else dispute it?
Very tempted to reply "well, how about you do it in two weeks then?". I never do, because I want to keep my job. And nobody is ever punished for missing deadlines, anyway.
This should be fairly easy to communicate to the business people.
Pretty much, the general consensus amongst developers was to just go along with whatever the scrum master is saying, just so that we could at least get out of the meeting and start our work. Missing deadlines was pretty common, getting blamed was pretty common, but you won't get fired. Perhaps you'll get moved to a different team, where the same story repeats itself.
We don't need unions for better wages. What we're often missing is respect, dignity. (Yes, sorta like sex workers.)
All of these have been unusual work environments so I’m not sure I have reproducible insight to offer (though the second case probably is more reproducible than not at all, and I highly encourage anyone with the opportunity to throw down and choose your own planning adventure to coordinate with your team and see what shakes out; you have nothing to lose but your estimation poker hand).
My approach is the MVP way, where the team delivers everything that is necessary to meet the customers goals. All the bells an whistles can be qdded once those goals are met. If the goal can't be met within the given timeframe, then it's a problem of unrealistic expectations. But most of the times we have met the deadline while otheres with their bloated waterfall projects were late.
--------------------
What about Scrum, Agile, Kanban, PMP and CAPM?
At this point many of you will be wondering, where is the conversation about all of the formal methodologies? Some of you will think it’s strange that I’ve written a whole chapter about project management without mentioning Scrum, Agile, Kanban, PMP and CAPM.
...I have not personally noticed that credentials (such as Scrum Master or PMP) are necessarily associated with real skill as a project manager. Rather, the key things that make a great project manager tend to be more qualitative:
Certain people gain these attributes as they gain experience, but these attributes tend to be things that are not emphasized in formal credentialed courses. Strong moral convictions tend to be picked up in childhood and are only learned slowly in adulthood. In formal credentialed courses the focus on various rote processes tends to shift the focus away from the things that are really important.The only thing he did in response to failure was to pivot us to feature driven development, and then he screwed that up by not clearly defining teams and muddying responsibilities.
Pick your jobs carefully or be prepared to walk away :(
Enjoy! All your problems are solved.
I'll tell you what I'd do, man, two teams doing Feature Driven Development at the same time, man.
> still makes projects late.
But under the concepts of the agile manifesto there is no concept as "late" so I'm not sure why the author refers to a project as being late. The whole point of an agile process is to be able to iterate quickly. 2 weeks is a generally accepted timeframe in which you can revert attention if what you develop is incorrect. So whats the gripe?
[0] - http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970...
After all, half a week is burned, minimum, at the beginning and end of the sprint on process paperwork and screaming at disconnected Zoom windows.
Next sprint, their week of work is frequently reverted or irreparably hacked to meet an arbitrary deadline.
Also, the product sucks, but management metrics ensure people that focus on shuffling the titanic deck chairs instead of fixing it get promoted.
If a group of engineers puts in a heroic two month effort to save the project, their metrics tank, and they end up reporting to the first group until their options run out or they are laid off.
My wife is a teacher and she was telling me that a panel who was responsible for hiring a new permanent teacher was "not allowed" to consider their prior knowledge of the applicants' teaching abilities, and had to go with the teacher who best answered the Department of Education's paperwork questions.
They chose (against their personal better judgement) the worse of the two teachers because "on paper" they appeared better.
I'm currently struggling to find a job (my current contract ends in a couple of weeks), and I just don't have the ability to be the level of ass-kiss dishonest, straight-laced, official textbook answers that seem to be what lands the fish. I'll get something eventually, and that's all I want, but it really feels like the world is tuned to the sizzle even when you can smell that the steak is rotting.
When you start a 2-week sprint how do you magically understand what you're planning to build if you don't do any planning...?
It stops my momentum, annoys me, which ruins my motivation.
;)
or the team is doing too much at a time, and delegating different "projects" to different people in the name of efficiency.
I think that's actually wrong - the whole team should work on the "same" task. For example, coding a module could be done together. It might seem like only 1 person is writing code, and everyone else is over the shoulder looking, but you save knowledge transfer times in the future, and you save code review time (you do review at the same time when everyone is there).
And literally nothing was lost in the process. I am as productive as before, know as little about product we are developing as before.
Being there straight, talking, thinking and listening would tire me out. But, I only half listen, dont focus on parts that dont seem relevant and get rest as I please.
These meetings were super tiring for me when I was trying to learn requirements from them. But the result was that I did not learned much anyway and not paying attention made no change.
In reality, software development doesn't fit neatly into this model. It's part R&D, part operational expense and part capital expense all at once (even in maintenance phase) - but it's impossible to distinguish which parts are which. The resulting product's value is constantly in flux, which means it is difficult to plan depreciation like a traditional asset.
Agile will always fail unless the way software is accounted for changes too, because the people at the top are looking at budget spreadsheets, not JIRA boards/burndown charts.
The problem isn't how the business solves software. It's how software dissolves the business.
Depending on where you are, you might find that allocating expenses as capitalisable is much more tax-efficient, hence the desire from the business (finance, specifically) to treat it that way.
Today everywhere I work is now “Agile”, doing scrum or similar. It’s all lip service, it’s just another process and ceremonies over the actual spirit and ethos of the original manifesto.
The teams I’ve worked on who didn’t practice agile where actually the most “agile” in the spirit of the manifesto. The teams that did agile, by the book, with the ceremonies and all, dogmatically, doubling down on it, where the least agile in the spirit of the manifesto and not that productive or creative, often micromanaging every tiny detail in a Jira ticket.
Agile/scrum is a codeword for "we don't want to / know how to / haven't ever experienced long term planning, but we're going to do it anyway".
"Scrum" has become waterfall without the pre-work so it often descends into iteratively pulling estimates out of your asses.
Well, mediocre anything is usually better than bad something.
I have friend in a defence sector waterfall project. It was meticulously planned years ago, the design was locked. Production was started. Everyone knows it'll get scrapped for being out of date when it's finished on schedule in 2024. But they need to finish it anyway, because the deal is hard-locked and there is a big penalty for not delivering or delivering late.
But let's say that the problem they're solving doesn't exist anymore, tech advanced and you can get 90% of the same functionality with off the shelf stuff =)
But they also know they need it by Tuesday.
Sales use a sales funnel as their guiding organizational tool. IT operations and support do reactionary work on incoming tickets, and some rather routine and planned projects. HR is almost all routine paperpushing. Legal has to take the time it takes to answer the questions they get, because they have to answer correctly. Market analysis is timeboxed analysis - a spike in agile lingo. Hardware design is innately waterfall. Hardware production is just mindless repetitiveness as fast as you can go... and so on...
They're all different!
Backlog grooming can be done asynchronously, constantly. The PM can push a task to the top without making the developers work harder this sprint. They can go faster or get delayed, because they don't have to meet the arbitrary requirements of a sprint.
Personally I have noticed opposite scrum works if you run it by the book first and then start tinkering process based on retros and this "agile" does not work where you implement something that kinda looks scrum or kanban and kinda mess major part like estimates are hours etc. It is not ideal but low trust or multiple stakeholders it works.
> "Soon there will be a new project methodology that will promise to deliver software projects on time and on budget. It won’t work, but we will all have fun learning new terms and joining a new methodology cult."
I think everyone here will agree this will happen. Again, and again, and again.
And that they understand agile to mean the same thing as the engineers. That is to say to small deliverables within a short period. Review the small deliverable with client, make adjustments, then repeat until final product is done. Basically get things working a small piece at a time. Change directions if needed. Don't boil the ocean.
What usually happens is that the clients take "agile" to mean changes on-the-fly AND quick delivery of final product. They also usually prefer the waterfall deadlines, because then can line things up for Q2, Q3, etc.
I think a big reason for the disconnect is the word "agile".
The Module pattern design built into many languages like Python is perhaps why some folks get away with it for awhile. ;)
But agile is often sold as a tool to fix such incompetencies i reckon!
The manifesto was so small and stark because those were the few things that all of the manifesto signatories could agree on. They were trying to find consensus on principles that made products successful.
Big A Agile is much more rigid and top-down, marketed to management. Success there means selling training and certifications. Nothing more.
To me the most disturbing part is people read the 'Agile Manifesto' and say, "yep, that's what we do", yet for every item it is the exact opposite. For example, "people over process" seems to be addressed by considering every team member identical and focussing purely on the tickets.
I am to the point now where I think corporate 'Wagile' is the Agile process and I simply don't understand what it was years ago.
They do it for a reason, to be fair. But, it is ridiculous that for how long agile existed, every single uncomfortable or dysfunctional thing about any agile process gets blamed on waterfall.
No. These are all issues with agile methodologies. Agile manifesto is just feel good document meant to make you buy into the process in abstract. It is inspirational, but that is all there is to it.
As a corollary, all ideologies end up as self-parody, contradicting their original intentions in sometimes egregious ways.
> The retrospectives are gone
Oh, they’re still there, they take like seventy-five minutes, they do produce some good observations, aired-out grievances, and action items, the latter of which are never ever followed upon in full or even half.
[1] https://nextbigideaclub.com/magazine/voltage-effect-make-goo...
https://steve-yegge.blogspot.com/2006/09/good-agile-bad-agil...
It's now apparent that Google is both very a different place in recent years, AND that all the "Good Agile" stuff from 15 yrs ago kinda screwed most of their products and reputation long term.
15yrs of hindsight later and it has probably become the biggest product pathology cliche identified with Google.
In practice, the only thing that ever really happens and I've never see anything deviate from this is as follows:
Ideally through this process, there is a lot of open and healthy conversation about the progress and status of the work. Anything outside of this is really almost always just cruft, a distraction and leads to bad feelings because the people who need to get the work done are distracted with silly things like poorly ran daily stand ups.Sometimes things make it so the project is late, no level of agility or number of waterfalls can avoid the sometimes inevitable so it's better if product acts under the assumption things might not always be on time and have a more flexible marketing schedule etc.
None of this would be a problem, except now there's a lot of anxiety about not "doing agile", or waterfall or something...
Agile is great when you can be agile. Not every busyness fits into that mold. Some software projects are big, regulated, public, etc. They need more structure.
For instance, you don’t update the cloud provider at a 500 company with agile. It takes planning, coordination, contingency plans, and often 2-3+ year timeline. Agile is for smaller companies or smaller projects that can be broken up into sprint size pieces. Many are not.
Just not true. I know it's hard to know if you haven't seen it but I assure you it's possible and much for effective for a large team to be truly agile
Honestly I hate the term agile, it should really be "open communication", and it can apply on even a daily timeline
Waterfall "style" is a directive comes from on high to get every service running on the new cloud provider, and 2 years later those on high find out we're still 2 years away from completion and along the way we spent months working on tiny aspects of it because the directive couldn't capture every nuance.
Agile "style" is where you get to return feedback to "on high" before that point. If there's a service that represents 5% of our revenue but required 30% of the effort, you get a chance to push back and talk to stakeholders before committing that.
-
It's really the idea of "further unraveling uncertainty as you go along instead of assuming you accounted for all of it at the start" that makes agile useful, and that can work on any project of any scale.
No matter how rigid the parameters of a project, at some level there's uncertainty (otherwise you'd have perfect estimates and nothing would ever be late). So better that "on high" learns more details about that uncertainty as you go along, and you get input on how to handle that uncertainty... than in a giant "we're flailing" moment later down the line.
I think people focus too much on the whole "no committing to deadlines" idea, when that is not at all central to the value proposition, and honestly feels like a feel good platitude that doesn't actually work when it meets reality regardless of how small you are
_All_ software projects include an element of discovery as new details are discovered, technical challenges are encountered, etc. To think otherwise is naive.
Waterfall in software means there are fixed timelines and a fixed starting scope to work against. This helps align the efforts of multiple teams that have their own deliverables and timelines. As a project progresses, the timelines and scope are reassessed. This is true of any methodology.
Full agility is when you can pivot quickly to new opportunities. This flexibility requires a small scale. Most larger companies have too much going on to allow for something like this yet they also allow for agility. For instance, a security incident is not going to be neglected because of an ongoing project. Waterfall in software is not what the author makes it out to be.
No one proposed or implied that any software doesn't have an element of discovery?
Obviously there's wiggle room for all of these terms but you're defining waterfall so loosely that you didn't describe waterfall at all (nothing about fixed starting scope and fixed timelines requires waterfall...)
Then on the flip side you're defining agile so precisely that almost no one can meet it. I mean what is "full agility"?
It feels like you invented a term for a theoretical "perfect" implementation of agility, but my entire point is that you don't need to go off the deep end with the concept of agile to reap the benefits.
-
Waterfall and agile mentalities don't need to be mutually exclusive, which is exactly what the article is about.
I lead projects on self-driving cars. When the NHTSA has hard requirements we don't get to go back and ask them to reconsider, but that doesn't mean that within implementing long-tail features we can't be have an agile process for making sure the teams implementing parts of those features aren't blindly banging away with no iteration on their approach.
Waterfall and agile should just be means to an end. The weird want some people have for one to be the "valid" approach has never made sense to me.
That is literally how definition of waterfall starts. Unless you use waterfall as "the mythical thing I project all my grievances at".
Waterfall "as it should be" was presented as a flawed model: In reality there is no competent team, using waterfall or otherwise, that relies perfect estimates and I wouldn't insult anyone by claiming they're doing so.
The waterfall is process where you start by collecting requirements, make estimation and sign contract with set deadline. The rest is development in one go, then testing and then you give result to customer. So yes, it relies on precise estimations. Whatever else you have in mind is not waterfall process. Waterfall does not prescribe how exactly you make estimation nor how do you communicate inside the team.
It has disadvantages which is why no one was using it for decades. Except as straw process to blame when other processes fail or dont prevent dysfunctions. So, management tries to implement scrum, it fails or turns into hell. Then next guy will try to implement Scrum or whatever, claiming he is bringing agile to save us from waterfall. But that is just a way to avoid saying that it was scrum that failed and repeating exact same cycle.
Again, literally no competent stakeholder anywhere is expecting a perfect estimate.
Both agile and waterfall both aim for precise estimates.
With agile you're still aiming to have roughly correct story points, or having roughly enough work for your sprint. But you're moving the yardstick on how often you estimate and how you handle missing those estimates.
All of which is why again, I don't see the point in being so rigid in where one ends and the other begins. Waterfall has gone from a punching bag to a real process people follow by adjustment. People seem to miss that while the "OG" waterfall diagram was supposed to be flawed, even in the same document practical improvements were introduced.
Incompetent stakeholders are real world phenomenon. In this sense, even stakeholders that are otherwise smart people and competent in other areas often fails in exact this way. Real world companies pay fines for being late. Real world customers without contractual guarantees end up paying more then project is worth too.
> Waterfall has gone from a punching bag to a real process people follow by adjustment.
This is the thing I disagree with strongly. Waterfall is punching bag and is not used in practice at all. It was past being used when I was young. Waterfall was also far from the only process that existed before agile came.
Iterative development process existed long before agile came to be. Agile means nothing and everything these days. Waterfall however is canvas where people project their grievances with other processes. That is why it is undefinable in these threads - it is either about communication, or people being treated bad, or about iteration or host of other practices that companies combine together.
Actually, the point of agile is that it should become whatever the team needs it to be.
Also, scrum is an agile methodology, not a separate thing.
Worse even, 'agile' gets used as an excuse for project managers and architects not to document requirements or plan properly, even in situations where all the goals can be known ahead of time, because that's a lot of work.
It's also rife with "It's not working? Clearly what we need is more interference, more monitoring by management, hire more 'agile coaches' to inflict more 'ceremonies' on developers and use even more of their time."
Never mind that planning is always thrown out of the window because priorities shift. Never mind that none of the changes that were identified as necessary in the retro get any sort of management buy-in, rendering it pointless. Never mind that 'stand-up' has become a management-run progress-report session. Never mind that this kills the feedback loop and developer-lead benefits that are supposed to come from this. No! Do more agile! Whatever the hell that means! Do it harder!
What's the complaint? The higher ups have a scapegoat, so they're happy. And you can rejoice in it not being you or any of your friends!
"Sprints" are supposed to be humiliating. The very message is that you're not trusted with even two fucking weeks of your own (!!) working time. It could not be clearer. If you work in sprints, it's because the higher-ups think you're a child and a loser, and you'll never be able to overcome that negative inference, because if you perform poorly you will confirm it and if you perform well, you will confirm that their micromanagement actually works--there is no winning.
Also, "Waterfall" never really existed. It was a straw man invented to sell this Agile bukkake.
In any case, a number of the dysfunctions are, in effect, intentional. Standups that go on for 45+ minutes? Long meetings are a classic way for managers to punish perceived underperformers (or, in Agile terms, "impediments") when they're not entirely sure who problem player is. The theory is that the rest of "the team" (I put this term in quotes because coworkers in corporate aren't an actual team--that's just management speak--but are often pitted against each other) will get sick of the incessant meetings and rat out the underperformer who is causing this wastage of time. The humiliations of Jira and "user stories" aren't bureaucratic accidents that occurred in good faith; they hurt because they're supposed to hurt.
That part is true.
> It was a straw man invented to sell this Agile bukkake.
That part is colorful, but retconning. A fellow named Royce coined the term in 1970 and used it as a strawman for common project iteration organization (V-model?) a the time.
I agree with most of the spirit of the rest of the parent's screed, though not with all the details.
1) lack of customer feedback loop - agile is meant to ship features quickly to customers, get feedback, and iterate/enhance/pivot based on their feedback. Often times the projects were simply things that "had to be done" and the whole agile process was not necessary.
2) inaccurate estimation - projects would start with asking engineers to estimate "t-shirt size" efforts on user stories, and what happened in reality is the projects always ended up taking longer than expected due to unknowns, dependencies etc. I've only worked with a few senior engineers who can accurately estimate, but junior engineers starting their careers in a big company, it's a tall order to ask them to be accurate.
Eg. 2 hours becomes 4 days. It allows for delivery, QA, bug fixes and documentation.
Agile is better thought of as a product management than a development methodology. It is meant to be agile in developing your product to iterate quickly and respond to feedback from real users. It's a great fit after a MVP is released to users to iterate on their interaction in a responsive way. It's not an ideology to be followed religiously for all projects with all constraints (particularly those where those paying for it want specific features/dates are a poor fit).
- If a given project is only agile on the development side and not also the project and product management/sponsor side, there's going to be a ton of friction.
- If all projects use the same methodologies (be it agile or waterfall or some other new hip flavor) to develop regardless of their scope and needs and sponsor, there's going to be a ton of friction.
- If your teams adjust their approach to meet the needs of the project at hand, and all aspects of the team for a given project are in sync with that approach not just developers, you just may be able to glean a relatively smooth experience.
People want to blame the method of project management but it comes down to the team and the project manager. There's a reason why project managers get paid so well in every field. They know how to manage a team while bringing in the project on time. It's a hard job. What method is used is not as important.