Because we’re still dealing with it everyday. I can’t even count the number of basic trainings I’ve gone through and then seen everything they preach go directly out the window
As many pointed out, unfortunately most businesses employing 'agile' don't seem to be 'agile' at all from management side, so we still have all sorts of broken practices used industry wide which are absolutely idiotic.
I'm working with a team that the PM follows Agile like its a silver bullet solution but is obviously terribly flawed.
I fully agree. In general, any flavor of agile for software development that doesn't include the engineering focus that Extreme Programming's technical practices bring in (fast feedback, refactoring, good testing & design, CI/CD deployment, team ownership of the solution - weather you pair program or not) is doomed to become intractable legacy after 12-18 months at the most optimistic.
I implemented Agile scrum on my team at the behest of my organization. I did it enthusiastically at first, because I thought it might make a difference. My opinion changed within the first 6 months. I think it's a waste of time.
Our stand-ups are pointless because they're just status updates. Not much of consequence can happen in 10 minutes anyway. I ignore our metrics at the end of each sprint. They're meaningless. Our retrospectives are largely useless other than giving shoutouts to team members. I feel the sprint structure leads to Parkinson's law.
This situation, however, is perfectly normal because Agile advocates "people over processes." My team doesn't give a shit about the Agile process, so it's mostly like forced physical training but without the fitness benefits.
Scrum should at best be viewed as training wheels for agile teams. Not the end state. Good teams abandon scrum for Kanban / Lean concepts once they start building good collaboration structures, better communication with business, earn trust by delivering and recognize (as you point out nicely) that the sprint timebox is artificial B.S. in practice. I think the Parkinson's law can be a common trap for teams to fall into with the Sprint timebox malarkey.
Managers trying to control engineers. I don't know how we are OK when people in charge don't understand what they are managing. Agile was an attempt to take control back to engineers but it was co-opted.
Can we just replace agile, and every modern management practice with some mix of common sense, talking to the customer, talking to the team, and being able to voice concerns to upper management where people actually listen instead of running their own agenda? =]
But this is the root of all these issues... Because everyone has a different perspective that underlies their "common sense," so this system of unspoken agreement just falls apart.
When you throw in the incentives of compensation and career advancement, politics and human nature are bound to corrupt the process further.
This is why codified process will always be better in the long run for most companies, because it's the only tool possible for combatting this
People that corrupt a process are often the ones it’s designed to stop. It’s not even done with ill intent more often than not.
How do you create a paradigm or process that can be amended adapt to a persistent threat without that amendment process itself being used as a tool to corrupt it?
I completely agree that you don’t need to assume ill intent. But by that same token, the fact that a paradigm or process isn’t invulnerable does not mean that it shouldn’t be used.
Bureaucracy, heavyweight process, and all rest are ways that systems protect themselves. Wasteful though, which is why new paradigms such as Agile come along.
Now the Agile ecosystem has become corrupted by a process not comparable to all the above. Not solvable by those means either. It’s a toughie.
“being able to voice concerns to upper management where people actually listen“
I think that’s what’s missing in most companies. We have 360 reviews, retrospectives and all kinds of stuff but there is no feedback channel up the chain. At my company whenever there are obvious problems management secludes itself and then a solution will be revealed to the underlings who never got heard. Especially in tech it’s pretty safe to assume that engineers at the bottom of the pyramid are as intelligent and educated as people further up the chain so I think their input is valuable.
It is human nature that when talking to your boss you put things in the most optimistic way that fits your understanding, and your boss hears it even more optimistically. After a few steps up the chain this results in a complete disconnect from reality.
Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
> It is human nature that when talking to your boss you put things in the most optimistic way that fits your understanding, and your boss hears it even more optimistically. After a few steps up the chain this results in a complete disconnect from reality.
I learned this the hard way, maybe 3 months into my first real job. I was testing safety critical systems, but it was still being developed so the tests procedures were also being developed and executed. I was asked for my status, I said: I've tested about 60% of the system (this one was small enough that that sort of statement was actually valid), everything has passed so far. My boss heard: Everything passed. He released the build to the customer, who found a failure almost immediately (about the same time I did, but I had no idea it had been released). I was taken into a conference room and chewed out for making us look like amateurs (we were, when it came to software). I learned several things: 1) I needed a new job; 2) Never use the words "done", "everything", "all" until everything is actually all done, managers only hear those words and nothing else.
> Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
A manager, fortunately not mine I was his peer though he got promoted for his "successes", would always say that his products never shipped late. "How can you say that? I know your last release was 3 months late." "We updated the schedule and we hit the new schedule." "But you didn't update it until the last month. It was 2 months late by then already." "But we hit the final scheduled release date." He just got lucky that the customers weren't loud enough for his boss to realize the spectacular, repeated failure until after he got promoted. The new manager got hit with it instead.
I too have seen that hears-half-of-what-you-said-and-only-the-good-parts, then gets mad at you because they are shitty listeners, in action. Hell, I've seen it with a guy who appeared to constantly be taking diligent notes, they were just very often incorrect! It's infuriating and there's no arguing with them because they just think you're lying or misremembering (even if you also took notes, or even prepared them in advance then read from them, and, unlike theirs, yours don't suck). The only way to deal with it is to communicate in writing.
None of that is ever going to happen until management accepts that developers can't give an accurate timeline on the spot, immediately, after a 10-minute overview of the project. Which means never.
I experienced a similar situation. The team leader looked back at the past few sprints and worked out how many story points we had completed, and calculated how much our salaries had cost the company in that time, to work out a "cost per story point".
He then went off on his own to talk to upper management and committed, without our input, to large new pieces of work, and was told what the budgets for those deliverables were.
At the next sprint planning, he was basically telling developers their estimates were wrong because the number of story points we were deciding equated to more money than the budget for those features.
Rather than play along, I simply said "Well why don't you come up with the estimates for us, then?".
To his credit, he did then stop trying to influence the votes after that, but it wasn't long before the product was put into maintenance mode and he was assigned to a different team.
The main problem for me is the misalignment of goals between upper management, middle management and the rest, the higher you climb the less interested they are in they user/client, they only care about some bullshit metric that will end up paying their bonuses.
This is specially visible in public companies where they only chase the metrics that the market looks in that decade, ignoring the rest or the actual user satisfaction (they do try to put a metric on that, with bullshit forms and all :))
XP, and several other processes, identify the problem as a mismatch between the goals of the developer and the actual customer (not necessarily the end user). Management doesn’t figure prominently in the solution which is probably one reason why a different form of Agile is usually chosen. I also don’t know why businesses are so reluctant to have their customers involved in software development. But you’re definitely right, when all the incentives align on the axis of management and their needs, you get garbage.
This is not to say that management isn’t useful or necessary, but they are a facilitator of work, not the patron.
Because the goals and needs of the upper management and of the customers do not align.
There's a lot of companies where delivering what the consumers want/need is not the end goal of the company, at best it's needed to help fulfill the end goal.
Most process pathologies are the result of common sense. Waterfall is a description of the usual reaction to undesirable results on the first project—add more planning steps and make people sign off on phases before moving to the next, more expensive phase.
A key success of agile was to provide a buzzword that let engineers say no when they were asked for a fixed estimate for a fixed-until-it-changes scope to fit in a Gantt chart.
Take agile away and they'll be straight back to demanding to know how long it's going to take to build something when no one's really sure what the something is.
I like the development methodology laid out my David Cancel in his book Hypergrowth. I like it because he has founded several large successful companies and actually used this methodology successfully. Unlike most of these "agile gurus" who have never sold a product to a customer in their lives.
I was saying in a team meeting the other day that using jira stories as any kind of performance metric is wrong-headed and counterproductive and metrics should focus on the business value provided by the team and people looked at me like I grew a second head.
I had a situation where the CEO was bringing up complains from the PM to me about how there were so many Jira issues in the back log that I should have been hiring more people.
I said that it would take me 5 minutes to go into Jira and close out all the existing issues if that is what they are worried about?
I've written something similar on here recently, but to repeat: I am, at this point, pretty sure that trying to combine manager-facing and team-facing planning/tracking/task/reporting tools into one thing is fundamentally a bad idea.
The problem is that jira gives you a unified interface into all the stories that all your teams are doing, while not doing anything to track the business value they’re providing.
Ultimately, people need agreed upon terms for communicating planning and expectations.
That's it. The more people are involved, the more complicated the process gets and all of these approaches evolve out of trying to find an agreeable and effective way of doing it so that everyone doesn't have to figure out a new solution every time.
As much of a buzzword hell as it is, I really believe that Scaled Agile Framework (SAFe) is the closest thing to the right balance of trade offs.
> At the time, software development was suffering from a mistaken belief: that building things fast and building things well were fundamentally opposed.
I thing “good, fast, cheap; pick 2” is still generally valid.
Developers are your biggest cost, so I never understood why fast and cheap would be opposed to each other. And if you buy off the shelve, most of the time it's both faster and cheaper than building it yourself.
So I always felt that fast and cheap go hand in hand.
I think the triangle is:
- features
- quality
- cost (=both price & time)
Edit: I forgot to mention that you cannot just throw more engineers at a problem to make it go faster. There is such a thing as an optimal team size.
Another edit (sorry it's Friday evening here;)): The faster you can get it done, the cheaper it will be.
We have to peg quality with rigorous testing or else cost (time spent) will be variable upwards to infinity (due to accumulation of untenable technical debt).
Therefore, if cost must be pegged due to limited time/funds and quality must be pegged due to existential risk then quantity/richness of features must float and be flexible.
All good product owners will learn this truth eventually.
This is unlikely to be true. Bad decisions are your biggest cost.
An employee has a "return multiplier" - for every dollar you spend on them, what is the return to the business? Sometimes it is high. If so, the developer is very cheap. Sometimes is it low, or negative. If so, the developer is expensive.
Why would it be high or low? Part of that is developer skill, but mostly it is how well your organization runs itself and makes decisions.
I 100% agree with you. I have this in my head like this: a developer with a low rate might be very expensive, and a developer with a high rate might be very cheap.
Organization is definitely a big factor.
But when I said developers are your biggest cost, this is from an accounting point of view.
Note to self: Remember whenever you think "Good,fast,cheap; pick 2", it frames your choices as either "produce something not-good" or "produce something and take your time or hire lots of people". You are again regressing to your ivory-tower-inventor tendency, you spend too long perfecting and the perfecting gets worse and worse. Instead it's "fast+cheap, fast+cheap, fast+cheap" to minimize the time between customer-hypothesis and customer-trial. You do that until you achieve "good enough". This is the surest and most economical way to achieve "good". This is agile.
A colleague once had a paper taped to the outside wall of their cubicle which read “Why do we never have time to build it right, but we always have time to build it over again?”
Management was not a huge fan of this piece of paper.
For the past several years now, each time I join a new team or project, my first objective is to detach and destroy the "Agile" process workflow.
When you finally liberate a team from Agile, it's just breathtaking how much you can focus on delivering working software that gets deployed with quick iterations that's closely aligned with the business and customer's needs. When free from the tyranny of Agile, teams can be effectively self-organize, remove micro-managers, and quickly adapt to changing needs and requirements. My experience is that staff are usually much happier, more productive, and less stressed once agile is gone.
I mean contemporary Agile as pushed by corporations and those awful "coaches" (who never seem to be actual developers) --- if you were you design a system whose end goal was making great developers unhappy, unproductive and locked into a dysfunctional system, Agile would be it.
> it's just breathtaking how much you can focus on delivering working software that gets deployed with quick iterations that's closely aligned with the business and customer's needs
That's literally the agile manifesto.
"Scrum" is one implementation of what agile was trying to achieve. Maybe that's what you're thinking of?
I know, that's the irony. Agile as pushed today ends up with the compete anithesis of that. It's the difference between "Agile" (Capital A), and "agile"
There's Big-A Agile and agile. The manifesto is really the latter, the former is a set (different depending on where you are) of very specific practices that neglect the principles in the manifesto (though, typically, they were derived from them originally).
Read SAFe Distilled or SAFe 4.5 Reference Guide for what happens in large enterprises (god SAFe is a fuck up). It consists of some good ideas, but also some very explicit practices that don't help in every effort (and sometimes hurt). It is literally the opposite of Agile, which is supposed to be about flexibility.
If you want to hear about its "success", just remember it was used for F-35's software...
Big-A Agile is based on the belief that adoption of practices is sufficient, and that deep understanding is unnecessary. This is the realm of cargo cults.
> If you want to hear about it's "success", just remember it was used for F-35's software...
This is the example I use all the time!!
Maybe agile is great when you're making an online shopping cart for a e-commerce website, but the idea of using Agile for complex, engineered systems is laugh-out-loud hysterically absurd. And this does not just mean spaceflight software, it basically means anything more sophisticated than a CRUD app.
Totally agree. I believe Agile works well in environments with shallow tech stacks, but it has worked terribly in my experience in product companies that build hardware, FPGA dev, embedded software, and other disciplines that can't deliver on two week increments or increments that align well with other disciplines.
Agile can work well in embedded, I've seen and done it.
The thing people have to forget is the notion of "deliver on two week increments". It's not about delivering a product that can be fielded each increment, in the case of an embedded system, but about delivering something that has some improvement or new testable component. I can't make an embedded radio handle, in two weeks, a completely new message type (well, depends). But I can do things in each two week interval that is verifiable. I can show that I've actually received the new message, that I can send it back out, that I can pass it through the various internal processors (if multiple processors are used) or processes (if a single one). Then I can start transforming it, storing it, changing other things about the radio state based on the message contents. Each of those is independently verifiable and completable within a short period of time. But taken as a whole, it's a 6-month project. The agile way has you make those small things, verify them, and then move on to the next thing. I can deliver (to the test team, to others using the system) the partially-completed system, it just can't be fielded (and that's fine).
And that's not unique to embedded. If you only focus on things that can be fielded in each increment, you'll never develop the more complex tasks, or address the tech debt.
I never really understood the amount of criticism aimed at agile by developers until I switched to a large enterprise. I feel like I’ve gone from a company focused on delivering working software which was supported by the common agile practices to a company which is Agile, really wants you to know just how Agile it is in every other company email, and one where our team will only be judged based on how Agile we truly are.
For example, there are days towards the end of the sprint when I’m not allowed to pick up new work and the infinite wisdom of the agile coach is that a ‘clean’ board by the end of the sprint is the what we’re really being paid to deliver (and starting something else would compromise that). This runs alongside serious customer deadlines which are hidden by the dates the Agile program runs by.
Agile/software engineering practices broadly suffer a lot from the fact that not only is selling software a business model, selling Agile is also a business model, and when you buy Agile consulting it's impossible to disentangle whether it's being sold to you because it works or because it makes that person money. They don't even know because it's their job and they convinced themselves of its usefulness.
It can still be evaluated on the merits but IMO this greatly pollutes the speed at which software devs as a broadly conceived community can come to consensus understanding of this.
Also I think the comparison to lean manufacturing has always been very shallow. I get the metaphor, I just don't think that human resources in engineering can be optimized like manufacturing processes. This quote is the best part of the article:
> "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".
As for the rest of it, I'm not dying to hear what the person who invented Agile thinks we should do next lol.
There really is no methodology called "Agile". There are things like Scrum, SAFE, AUP, XP, etc. Some of those may or may not mandate things like "sprints", but the essence of agile (the Agile Manifesto) doesn't even mention the term "sprint".
The closest it gets to that is where it say:
We follow these principles:
<snip>
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
> I'm not dying to hear what the person who invented Agile thinks we should do next
Agile has plenty of good ideas.
The problem is the almost religious following and, as you mention, the whole industry that has sprung up around it.
Even the initiators said that it was supposed to be a rough framework, to be adapted to your individual circumstances and teams. It was also a (much needed) counterpoint to the then prevalent waterfall model.
Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")
When you step back, a lot of the ideas make sense, and many teams will even implement similar workflows without having ever heard about "Agile".
I remember someone giving a lecture on software engineering methodologies who gave a pretty good summary by saying that "The methods are there to help your brain, not to replace your brain."
Haha, this was hilarious to read, mainly because, on some occasions, I've been a cowboy, at least in the context of this post. I like to think it worked pretty well, since, for what we were trying to do, the processes were fighting against us, so I just... did what I considered right technically, regardless of tickets and sprints and who's team should do what.
Whether it was right, I'm not sure, but people have been happy with my work, so that allowed me a bit of leeway.
When you have a large program team with mixed experience levels who don't fully understand fundamental principles, sometimes the only way to keep your sanity is to force everyone to strictly follow a defined agile methodology (LeSS or SAFe or whatever). That isn't optimal, but if you have to deliver working software right now you can't wait around to figure out an optimal process for your unique circumstances. Pick something good enough and move forward, then improve as you go
Agile doesn't ask you to figure out the "optimal process", it asks you to learn. One of the key things I've seen missed in almost every attempt at Agile/DevOps/whatever is the retrospective or the learning component.
The team/organization has to become a learning organization. That means a number of things, but the critical one here is learning from past failures and successes and incorporating that feedback into their model (ideally continuously, but in Scrums it'd be the end-of-sprint retrospective). You start down a path, and you find it's difficult. You don't press on just because it's the one you selected, that's the way of idiots. You examine the hardships you're facing, and you address them.
Absolutely. To my mind, the one vital agile practice is the weekly retrospective where the team looks at how things went, thinks a bit, and decides to try something in following weeks. Everything else is just tools people can try applying to solve the problem of the week.
> ... it was supposed to be a rough framework, to be adapted to your individual circumstances and teams.
> Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")
Yeah. Part of agile is that your process is an adjustible parameter. If you're doing it with a rigid process - any rigid process - then you are not actually doing agile.
> Also I think the comparison to lean manufacturing has always been very shallow. I get the metaphor, I just don't think that human resources in engineering can be optimized like manufacturing processes.
Agile and Lean are empirical process controls, they are based on the same concepts. Ken Schwaber explains all this on the first chapter of his book "Agile Software Development with SCRUM":
Defined process control: same inputs always result in same output (manufacturing widgets on a production line).
Empirical process control: same inputs not always result in same output.
Schwaber conceived SCRUM (and was among the founders of Agile) after realizing software development required an empirical process control: give 2 dev teams same specs, 2 different apps will come out (they might do the same thing, but in different ways)
Which bothers me much, much less than selling things like CMMI/PSP certifications or EUP/RUP which are done purely for paper pushing and selling the paper value.
Agile is an improvement on waterfall and you don't need to be certified to do it.
Absolutely. And, having done some agile consulting long ago, I'd say it's more pernicious than that.
The people who truly want to make deep change in pursuit of deep improvement are a small segment of the market. Worse, they don't need a lot of help. In the aughts, I had a few clients who really got it after 3 months of focused work, and then they were off and running.
But a large company that only wants to talk about change and maybe make some 5% improvements if they aren't too hard? That can be milked forever. Well, I can't, because I care about results. But consultants who either don't care or don't notice? They're golden.
I've seen several director types get promotions promising to get faster and more reliable work out of existing engineers by implementing this new religion... Agile.
The results were always predictable. Layering more meetings and stress on people who already have too much work to do, doesn't help things. But, you're still vice president.
> > "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".
This quote is silly.
To me, agile is just good engineering practice, applied to software. Of course mechanical engineers apply its principles, and have for decades before the term Agile was coined.
And as such, this practice is far older than software.
The Apollo space programme is my favourite example: the ultimate goal remained fixed (man/moon/before end of decade), but all steps of the way were discovered and redefined over the programme's course.
Mission objectives were changed depending on what was learned, often even in flight.
This was a very nice and agile (and sensible) approach, regardless of what it was called.
You're just offering your own special definition of "agile", that of course doesn't fit anyone else's definition, and especially any agile consultant's.
Of course, even more ironically, it doesn't fit the original agile manifesto:
Welcome changing requirements, even late in
development. Agile processes harness change for
the customer's competitive advantage.
> You're just offering your own special definition of "agile",
I'm not so sure.
What's the official definition? Don't give me the manifesto, that's just an implementation of common sense in development, as applied to software.
As a trained mechanical engineer (maybe that's why the quote about mechanical engineers irked me especially), the agile manifesto just reads like good engineering practice to me.
And indeed the signatories of the manifesto expected to start a huge discussion on what agility meant for any particular team or product, and how to best live up to its principles.
> of course [it doesn't fit]
That was uncalled for.
> and especially any agile consultant's.
I'm (something akin to) an agile consultant though :-)
Also, just to point out: maybe my definition of Agile diverges from canon. That's OK. I don't care to follow the canon, I care to do right by those who I've been asked to support.
> Of course, even more ironically, it doesn't fit the original agile manifesto:
> > Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
I'm not sure how you came to that conclusion.
How is "welcome changing requirements" a bad fit for "changed the mission objectives (as new information emerged)". Clearly the [competitive advantage/likelihood of success] was increased? In fact the entire programme was shaped such that change was explicit part of the plan.
I've stared at this quote for a while now, and I still can't see the contradiction.
> but I have to iterate on features holistically and sometimes speculatively.
YES. THANK YOU! I've been trying to tell people this for ages. Maybe other people are somehow capable of perfectly predicting every conceivable edge-case and consequence without even starting to work on something, but I can't. I have to start building something in order to see it clearly. And this ENRAGES certain kinds of managers, I have found.
Managers just want you to deliver on a predictable schedule.
Here's how you can solve your problem: Make as many tickets as you can think of, it doesn't really matter if they contain the essence of the problem or not, it just matters that there are many tickets associated with the task. Once you have your tickets, estimate them with as much padding as possible. If the manager complains that you're padding a ticket too much, make two tickets and split the estimate.
Once you have a big pile of tickets and estimates, you've bought yourself enough time to do the work. Go do the work, then mark all the tickets as complete. You'll figure out the edge cases as you go, as long as you've made enough tickets and time to do the job at a reasonable pace.
If you have extra time left over, get started on the next thing, improve your tooling, or try to help someone else with their tasks. If you don't have extra time left over, make more tickets next time.
Managers want a fiction to present to their superiors - the most happy my managers are is when what you describe happens. I like your approach :) I might have to be more explicit in creating a fiction. The only issue is if/when you actually run into things that significantly delay things right towards the end.
Where exactly is the problem with my strategy? I'm also a manager of a small team. My main goals are to make sure everyone is not idle, not burned out, and not missing deadlines.
My boss's goal is to try and get everyone to work every waking hour so we can deliver faster. My job is primarily to keep the peace between the executives and the workers. When I fail the workers, it looks like unpaid overtime. When I fail the executives, it looks like undelivered features.
I think the actual purpose of agile, from a developer's perspective, is to create more time to deliver quality engineering work product. From an executive's perspective, agile exists to squeeze developers into completing tickets faster. As a manager, it's my job to keep both parties as satisfied as possible. If engineers have enough time to build and executives get enough deliverables, then the company succeeds.
I read your comment as "create as many tickets as necessary to throw off management's understanding of the effort and complexity of the problem." Creating more tasks than necessary sounds like a waste of everybody's time.
As I noted in a parallel comment, on my team, developers are never creating tasks on their own or in a vacuum. It's done as a team effort during planning meetings.
Perhaps I should add that my boss (senior director) has never asked me about my board, velocity, or anything like that. Discussion is higher level - "is project X on schedule?" or "are you going to miss anything on your roadmap?" not "why is task X or story Y not done yet?" If your director is asking that, he has too much time on his hands.
Perhaps project managers who aren't good at their jobs.
I've reported to two different directors since moving into management. Both were very clear that they'd rather hear about problems early. And both have been nothing but reasonable when I've reported problems. Of course, the entire team needs to be reasonable for this to work, from the product managers to project managers to individual contributors.
And as a manager, nothing annoys me more than an employee who tells me everything is fine then misses a deliverable. Tell me you're having problems so we can find a solution. Please. That's why I'm paid what I'm paid.
Eh, those of us who work with morons need to figure something out, and a form of this strategy has worked pretty well for me. They're not even really morons, they're just more focused on having something to cover their ass than giving a shit about building the product. I can get away with making stuff up because they don't actually care about how I get the work done, they just need a long list of things they can check off to show progress to whoever is hassling them.
> "I simply cannot "decompose" a single feature into 500 different sub-task Jiras."
Nobody can; that's the primary failure mode of waterfall. Anybody who is telling you to decompose a feature into that many subtasks might claim to be doing agile but they really are trying to do waterfall.
That's the primary failure mode of a straw man software development methodology called waterfall that Scrum practitioners like to use as a bogeyman to appeal to for rhetorical purposes during the inevitable arguments about niggly little details of how to implement Scrum during sprint retrospectives.
I've never actually done real waterfall, but I did study it way back when I was in college, and I have worked in government contracts, which tend to handle the work in a way that is similar to the waterfall I read about in school. One thing I distinctly remember is that it was designed to be a flexible and iterative methodology that would have been difficult to cram into a ticketing system in the first place, let alone cram into Jira while simultaneously carving the work into tiny pieces, up front, all at once.
FWIW, the only place I've ever been asked to do something like that is in ostensible Scrum shops. In fact, the only time I've ever seen anything that looks like the Waterfall of scrum lore is in shops that are trying to do Scrum. I am beginning to suspect that the Waterfall of Scrum lore isn't actually a thing that happens outside of Scrum at all. It's actually what naturally tends to emerge when you try to apply Scrum methodology in a situation where something along the lines of the real, textbook waterfall model would have been more appropriate.
As a concrete example, I currently work at a team that ostensibly uses Scrum, but where QA is a separate department with its own practices that the dev team does not control. They do still want some ability to anticipate work that's coming their way, though, so they're monitoring the scrum board for that purpose. This is the moment where it gets messy, because we're then asked to document things ahead of time, and it's a minor crisis if we keep it flexible during the sprint because any changes to the set of tickets becomes an inevitable hassle as they complain that we've screwed up work product that they generate based on those tickets. It's absolutely something straight out of waterfall horror stories from Scrum lore. But it's only happening because we're doggedly insisting on something that's at least cosmetically similar to scrum. If we weren't doing that, we'd be freer to choose modes of interaction and business artifacts that are better suited to the reality we occupy.
A reason you often see Waterfall-in-Scrum is because people who were using Waterfall wanted to get away from it, and they made it as far as Scrum (the name). They usually keep the same practices of massive design/spec documents that are developed in advance. They try to plan out the next 20 sprints (hah, I saw a team that tried to plan out 60 sprints 4-week sprints, it was kind of hilarious if I wasn't dying inside from the thought of it).
The biggest difference for Waterfall-in-Scrum versus Waterfall is that they've sufficiently (hopefully) decomposed tasks to have short target dates for delivering something to a test team or facility. They don't use it to get feedback from customers (the actual users, not the ones writing the checks). They don't use it to replan when things go wrong. They just use the whip and OT and get back on track.
Waterfall was the dominant model of software engineering up until the late 1990s. It is how managers want software (and projects generally) to work. I promise, it was real! Back then, a quarterly release cycle was fast; 12-18 months was more common. With the rise of the Internet, everybody knew that couldn't work, but didn't know what to do, so there was a lot of experimentation in the late 1990s: Scrum, FDD, Crystal, Extreme Programming, and more I've forgotten or that were never named.
Out of this chaos, eventually came order. It turns out what mattered most in practice, just like before, was pleasing managers and executives. Effectiveness was generally secondary. So what we ended up dominating is Scrum, the most waterfall of Agile processes. Most shops "doing Agile" these days are still following a top-down, control-oriented, ineffective and unrealistic process, just like before. But now they use Agile labels for their mostly-unchanged process, albeit with a faster release cadence.
I don't believe that to be the case. The surveys I can find suggest that releasing quarterly or less often is very much in the minority. If you have data otherwise, I'd be interested to see it.
Don't get me started when they try to push all clients into one jira, each one of them with their own complexities, release cycles etc.
Or when they try to measure team effectiveness using Jira un some way or another, every team is capable of doing more Jiras in the same time by just splitting them more!
It can usually be done. The problem is that people assume that now that you have that list of subtasks, you can iterate through them one at a time until you've built the feature. That's the major disconnect.
What I do with my guys is I make them decompose the feature, but with the understanding that they'll probably be jumping from one subtask to the other, back and forth, as they understand what it is they'll be building. AND they'll probably come across new subtasks as they work through the feature.
It makes Jira look crappy, but it Reflects Reality (tm) which to me is far more important than keeping Jira clean.
That's how it works. Touch something here, touch something there, learn from it and slowly build up the whole thing. I think Scrum and Kanban may work if you have a somewhat mature system where you add incremental features but if you do something relatively new and big the backlog can not reflect reality. Or it forces a workflow that doesn't make sense. I have seen it multiple times where people refused to make changes to something because their story was done already. Never mind that the design turned out to be unsuitable.
I saw this happen before Agile was even a thing. The possibility of a bug in the spec was never a consideration. You had to hope your assumptions going in were correct, or your users were screwed.
A story I love to tell is the time where I was in the medical device industry. A requirement was formally reviewed with the owner and he signed off on it. The design was completed and reviewed and found to match the requirement, I implemented it, tested it and found it worked as expected and handed off to QA. QA tested it and verified that it correctly implemented the Requirement. The feature was turned over to its owner for so he could use it and his response was "that's not what I asked for!"
I work in medical too and I have seen the same thing. But to be honest, the same happened when I was consultant. You painstakingly go through every detail, everybody is happy, but when the product gets delivered, it just doesn’t feel right. That’s where rapid development with quick customer feedback is useful. Unfortunately often the customer doesn’t have the time to look at things.
What I find helpful as a product manager, I discuss each feature with engineers - and we collaboratively break it down into a bare-bones "MVP" version, but consider nice to haves as future stuff or TBD based on what we learn. That helps push scope down and manage holistic iteration. Every feature should have an MVP version of it. At least asking the question can help break down that first user story.
I don't know if something is wrong with me, but I simply cannot "decompose" a single feature into 500 different sub-task Jiras.
No, nothing is wrong with you. But let me note that the Agile Manifesto doesn't say anything about Jira, or anything prescriptive about how small to make your stories.
If your methodology asks you to do that, I'd argue that your methodology is broken. But it's broken because it's shit, not because it's "agile."
Well "individuals and interactions over process and tools" is in the Manifesto, so I would say JIRA is implicitly covered. I am being asked to do "proper structured agile" by writing up many JIRA tasks in advance of learning what best to write, and I find it funny. It's not the disease though, just a symptom of having too many nonprogrammers around. Will resolve itself.
I am being asked to do "proper structured agile" by writing up many JIRA tasks in advance of learning what best to write, and I find it funny
Yeah, it's hilarious how people are being told to do something in the name of "Agile" that is almost completely opposite of the spirit of the Agile Manifesto. sigh
> just a symptom of having too many nonprogrammers around. Will resolve itself.
Parkinson's Law doesn't get enough of the blame for bad dev environments. It's so easy to feel like you're doing 40 hours of difficult, important work a week, while actually accomplishing fuck all at best (or generating more work overall at worst).
I bet a big chunk of the problems raised in this thread don't exist if everyone experiencing them cuts their team size in half.
This is not a question of agile or not. This is software estimation and design.
To be able to decompose a feature, you need to learn how to build the software in your head before you go to paper. Consider a large billing page, for example.
Step 1 is to look at the front end. What components do I need to build? (React might use components, or a set of partials and templates in rails, for example.) So, then, I know I need X subtasks, where I will build the pieces I need to expose that behavior to the client to start.
I then look down into the backend. That might be a simple task on the backend, or given 20 minutes of looking, I might see problems that I'll need to tackle. The problems are their own subtasks. I may be able to group some of the front end to a single back end ticket, or I might need a separate subtask for each.
Now, I'm likely to miss something. However, that's why this is an estimate rather than a concrete task list. It's to get started so that we know, at minimum, what it's going to take.
It's not necessarily simple, but it is a skill that can be developed.
> To be able to decompose a feature, you need to learn how to build the software in your head before you go to paper.
Even easier than building the software in your head is actually building the software. I often don't know what the shape of the problem is until I've built something. The users often don't know what they want until they've seen something.
The failure of Agile as implemented is that it's still all about extensive meetings and planning. Software is not like building a bridge where you write out the blueprints and the expensive part is then building the bridge. Software is the blueprint.
To be truly agile is to be able to get software features and changes running and out to users as quickly as possible. You can never get all the information -- if users could provide the details absolutely necessary to build the software on day one then they'd be the programmers. Being experienced helps you naturally scope problems but you'll still be wrong from time time. Being able to iterate on users feedback and dealing with your own failed assumptions is the key.
> I often don't know what the shape of the problem is until I've built something.
This is the majority of estimation failures as far as I've seen. You can only "estimate" so far and then leave significant padding for all the other minutia that will likely creep in.
This is why I like to estimate on a more macro level. Define features loosely, with a few lines of text about what they are and how they relate directly to a business goal (how will it make money, make the product better etc), then as a dev, you should be aligning the work to that (which is a piece of cake because they're still flexible).
You should still have more granular planning, but it should serve the purpose of aiding the dev in completing the feature. You decide to do more granular planning when, as a dev, you look at a feature and go "The implementation is still ambiguous or I still have questions about how it relates to business goals".
Basically, invert the communication channels and shift responsibility to the devs. They have some loose business goals to hit, and the other staff provide support when they need it.
This solves so many problems I run into in enterprise dev. It kills Parkinson's Law (since the team members that are in the bottleneck are now responsible for the bulk of work generation within the team), it also gets rid of a whole class of poor decisions that are made without the proper context (since now you have a closer relationship between business goals and development).
Yes and no. Testing is a very important part of agile, yet if you don't have a somewhat final internal design, you will find yourself changing your unit tests every time you refactor your code. As a rule of thumb, changing your unit tests is a no-no.
First off, it means your code is brittle. Second is that you can't trust tests that must change every time your code changes. Might as well not even have them.
Unit tests are supposed to tell you if your refactoring is still working. For the most part, you should only refactor implementation details, not the "API".
There is also a design to uncertainty trade off. The less you know about something, the less time you should spend designing because you can't design something you don't understand. But the more your understand something, the more time you should probably spend designing.
Knowing how much design is too much is subjective and personal. I forget where I read it, XP(Extreme Programming) or Code Complete, but someone worth listening said, to paraphrase, "I've never regretted spending more time on design before I started coding". And I'm pretty sure it was part of XP that said the main purpose of XP(Agile) is to deal with the unknown. The more you know, the less Agile and the more waterfall you should be, saying that for some projects, a 6 month sprint is perfectly valid.
I am in an interesting situation where I am both the programmer and the domain expert. Generally customers come to our team describing a problem that other companies can't figure out. We're pretty much allowed to design everything how we feel like because we're the best at it. We can do virtually any project in a single go of it without feedback from the customer, but we still do agile because it helps with scheduling, planning, prioritization, etc.
We're a "value add" team, which makes things really interesting. None of our work is sold, but the work is important to land large contracts for the companies core services.
Decomposing a new feature into digestible pieces is hard work and a skill that is learned over time. It's also a skill to know when something is small-enough, or has enough unknowns that further decomposition is wasteful.
Good product owners and managers know this and are good at it themselves. And the decomposition is typically at a story level - "can we take story X, break it into 2 smaller stories, and still deliver something of value?" The tasks are usually pretty self-evident once you break down the stories a few times.
For a lot of my work, if the story is "As a user, I need to do X" the tasks are simply broken into distinctly testable pieces.* Is there a database change? That's a task. Is there an API change? That's another task. Front-end change? Third task. And if you get half-way through and need to start over, that's life and it happens reasonably often. Yeah, there might be some re-testing. Or, a new story for next sprint.
It's up to me, as the manager, to make sure the stakeholders are informed and on-board. At the end of the day, as long as the team is making progress, I'm happy. And if the team isn't making progress, it's probably my fault (and almost never the fault of an individual contributor).
* In my world, tasks don't exist in isolation. They're all subordinate to a user story. Every task (piece of work) we do is done in the furtherance of a well-defined need of the user. And developers are never defining tasks on their own - it's done as a team.
It sounds like you're trying to decompose software into developer tasks, which is not what (my interpretation of) agile is about.
Forget the developer for a moment and take the user perspective. What's the first thing the user wants to do? Log in? Ok, that's a story. What's the next thing the user wants to do? See a list of widget prices? Ok, that's a story. What's the next thing the user wants to do? Change a price? Story.
If you can't sit down and think of a dozen narratives like this, then the fundamental problem that you don't really understand how people are supposed to use your software or what your software is supposed to do.
That depends on business requirements. In simple cases, auth can be as simple as dropping a little javascript on the page.
But sure, you can have follow-on stories like "user is denied access protected resource" or "admin sees resources accessed by user". Enough individually-useless stories and you end up with a useful app. That's kind of the point of agile.
To some extend, you could if you worked on the task and, as you worked, said, "what am I doing now and what do I need to do next?" and then plugged that into something.
It wouldn't be a complete decomposition, of course, a person watching would see your thinking unfold. I think speculative decomposition would be very interesting, especially in retrospective if we could see the rabbit holes we went down.
Decomposing is easy since we naturally have to do that as we work. Heck, those 500 tasks are probably in your shell, browser and commit histories, mixed in with a bunch of other junk.
The problem is tools like Jira make this heavy-weight, doing it up front makes it nigh impossible, and having others review all this stuff and it becoming promises blows up the LOE significantly. And, I don't think I'd want to be under that much of a microscope as I work.
But if it were very lightweight, where I'm just posting my thoughts and can see them as a quick dependency diagram, and maybe attach notes, commits and URLs to them, and other people can see them, that'd be pretty helpful.
> But if it were very lightweight, where I'm just posting my thoughts and can see them as a quick dependency diagram, and maybe attach notes, commits and URLs to them, and other people can see them, that'd be pretty helpful.
That's precisely the objective of "user stories" and other things. You write a high level version, put it in a backlog with priorities. When you get to it, you realize it's bigger than anticipated ("Oh, I can't just do X, I have to do A, B, and C."). So you turn X into three things, one of which you work on now, and the others in the backlog. Repeat until done.
I don't think Jira is necessarily too heavy-weight, it's the way everyone seems to use it. They want to assign the tasks now, not treat them as backlog items or things that can be modified in the future. Which pushes them back towards big-design-up-front and entirely defeats the objective, you're back to low throughput, high latency development.
The reality is that your ability to break things down depends on experience. I’ve built so many landing pages, for example, that I can tell you an exact breakdown of tasks. Been a part of so many SaaSes, I can tell you exactly what non-core features you’ll need, when, and how much work they are. I can also predict where you’ll hiccup.
But ask me to build something new that’s never been done before (at least by me) and the most I can do is shrug and guess. Maybe give you a rough sketch of an outline of subtasks.
Yeah, I think trying to perfectly describe the work in advance is Waterfall thinking. Most of what gets sold as "Agile" these days is Waterfall dressed up in Agile terms.
You'll note that we never did detailed planning more than a week in advance. We could have, but it would have been a waste, because it would have been speculation on speculation. Instead we'd agree on something small to build, get it working, and then see what we thought.
That's not agile, that's waterfall. With my team, I usually have just 2 or 3 "stories" that are taken up during a sprint. The rest is up to the development team. Works quite well. No one's asking to create dozens of tasks. A rough outline of what we're trying achieve combined with a high-level overview of what we think would need to be done is usually enough to get started in a sprint. If we fail, we fail, no one gets mad. That's agile if you ask me.
The likes of SAFe argue otherwise, having you plan out and commit to _three month's_ worth of stories in advance. It's supposedly still "agile" because the plans you've "committed to" are expected to change as you go. Oh, and it's set up such that other teams are expecting to be able to depend on your team delivering stories by the sprints you said they'd be done, of course. No problem, as those dependencies probably existed anyway and now they're made explicit and public. As long as you've delivered the increment of work you'd promised when the three months are up, nobody gets fired. Totally agile.
You're right that there is a limit to the usefulness of breaking down sub-tasks. It's up to a team how granular they get.
Here is why it is important:
a) Risk management - if you break a feature down into sub-tasks at any granularity, you are creating an agreement (or at least a conversation) among your peers that this is how something will be implemented, and digging in beforehand to uncover areas which might impede shipping the feature sooner.
b) You're going to have to break down the feature at some point. Being able to think through this ahead of time can be challenging, but often times is the meat of the work you do. You have to get into the hang of it -- think top-down or bottom-up ways of approaching it.
But what if you don't have enough information or understanding yet to do that? Agile is not great for these tasks where you need to take some time learning or experimenting.
I would offer you a couple extra tools here:
- A "spike" ticket -- Agile is all about deliverables and commitments, but sometimes that doesn't work. So create a spike ticket, and define what it is you want to investigate. If you deliver something, great! If not, no worries. The important part is that in the future you've done work that enabled you to learn how to estimate or break down that task in the future.
- A time-boxed ticket -- similar to the spike, but you just make sure you don't spend more than an allotted time on a task.
Spiking is great as just a general development process even if you don't have tickets/time-boxed items just for them. For example in Basecamp's "Shape Up" process spiking may be an initial uphill technique used to suss out the rest of the cycle's work and uncover all the hidden issues.
Basically, you blaze a trail to the most disgusting, kluge-y, and otherwise slapdash implementation that validates your assumptions and satisfies your constraints while making note of everything that needs further work. That lets you bail early if some unknown issue will block success given the current criteria and constraints, and sets you up with the a list of the 80/20 work required to deliver the completed project.
Creating a huge list of tasks is actually what I would call an Agile anti-pattern. It's something that I unfortunately see all the time. User stories are meant to be written from a business or end-user perspective. It's a high level description of what is to be done, for who, and why, not how. I personally don't even like having tasks on the board, just give me a well written story and I'm happy to complete it based on what's written in the acceptance criteria.
How would you implement a database, or operating system, without a big list of tasks?
Some things need to be designed, and don't work terribly well if they're evolved incrementally from user-visible features. Incremental additions can be, but there's also a risk of gradually degrading the architecture of a system through risk-minimized local changes by interchangable resources, I mean developers - which is what I generally see occur under Scrum.
Company and developers are busy and successful, so they hire more, which continues until they are no longer successful.
They remain busy, however, progress on the product is roughly constant. All the additional work capacity is spent on meetings and otherwise organizing the increased worker count.
I believe the above describes every situation in which I have been asked to break down Jira tasks into unnaturally small tasks.
Maybe it was just my particular job, but so much of my work was figuring out how to do xyz, so it was hard to give estimates for something whilst I was still figuring out the scope and complexity of along with how it even works and I was rarely, if ever doing the same/similar thing multiple times. Whenever project managers did push for me to break things down further they’d then immediately complain about too much technical detail.
As far as I'm concerned, "agile" refers to the principles on this page[1], and ONLY the principles on this page. Not Scrum, not Lean, not Kanban, not Xtreme.
The problem is, it's hard to sell a set of principles. To understand people and interactions takes months of working with people: it's much easier to sell them a process or a tool and leave. Customer collaboration requires building a relationship: it's much easier to negotiate a contract for your client and leave. Responding to change requires you to stick around and see what changes happen: it's much easier to sell them a plan and leave. And to arrive at working software you have to get a lot of things right: it's much easier to sell a bunch of documentation for software that doesn't exist, and then leave. Money ruins everything: the reason agile had to explicitly deprioritize the things on the right side of the list in the first place is that all the incentives in a software company push you toward the wrong priorities. The things on the right side of the list are all quick, easy wins that look good on a quarterly report.
There are companies who I've worked with who do agile (the principles) well. There's nothing wrong with looking at Scrum/Kanban/whatever as inspiration, as long as you realize that they're just processes: individuals and interactions matter more. There's nothing wrong with using Pivotal/Jira/Trello/whatever, as long as you realize they're just tools: individuals and interactions matter more.
Call me jaded, but that's probably because the cow stopped giving milk, and they need to come up with a new cult/religion/ideology to continue selling books and conference tickets (and consulting gigs of course).
I think Zed Shaw had the best (or at lest most entertaining) take on being jaded about "agile." Though, fair warning, it is even more vulgar than the URL indicates. http://programming-motherfucker.com/
The terms "agile" and "Agile" have two separate meanings.
Big 'A' - "Agile" is waterfall re-branded - an excuse for corporate empire building and business as usual. It involves lot's of meetings and not trusting those who build software to do the right thing.
Small 'a' - "agile" is the implementation of the manifesto, which basically comes down to smart people figuring out how to work together towards a goal, often by taking small steps.
Until the terminology is sorted out the discussion can't help but be confused.
Hiring a consultant to teach 'Agile' is easy: the change of practices from something completely top-down to something that empowers the people at the bottom is the hard part and some orgs aren't capable of changing. Too many orgs are built around micromanagement, they don't know any other way.
Since businesses are the ones who employ software developers, 'Agile' is the only Agile that matters because that's the definition the people who pay money use.
We can talk all day about principle philosophical differences and what is/isn't 'agile,' but there has been a consensus from businesses in industry that 'agile' is 'Agile.'
Agile has become an excuse for terrible planning and offloading more and more work with ever increasing responsibilities to developers. At some point, enough professionals will reject following these terrible frameworks through different mechanisms. We're definitely not there yet, unfortunately.
In my experience "agile" was a loose rebranding of ideas that had been around for a long while (see all the other non-waterfall approaches that predate it) and "Agile" (i.e. the manifesto) was just one attempt at one of these sort of methodologies.
Once it got popular, "Agile" was co-opted by all the usual players (cf "extreme" before it, to a lesser degree).
The important idea is "agile" vs. waterfall, whether or not that includes anything directly recognizable as "Agile". Or call it something else, doesn't really matter.
Recent history shows you can certainly do things directly recognizable as part of the Agile methodology while demonstrably not being agile, so modulo the no true scotsman fallacy it's a much less fruitful distinction to draw.
True, but they were already branded albeit not as successfully. And this is very much one of those things that is somewhat evergreen - it just gets a new branding every decade or so.
I guess I'm saying "Agile" was/is one of several, and that's ok (good even). It's worth not getting bogged down on the "Agile" part and focusing on the more essential things.
>In my experience "agile" was a loose rebranding of ideas that had been around for a long while (see all the other non-waterfall approaches that predate it) and "Agile" (i.e. the manifesto) was just one attempt at one of these sort of methodologies.
You are correct. The Agile term was probably marketed to the management types as a some breakthrough methodology to _finally_ control the budget for software development. Then it took a life of it's own as many things do.
Every time I see an "agile sucks" post, I take the time to read it and every time (so far) I have found they blame the process for some key part of agile they missed. Quote from the article:
“Way too much of Agile has been not about technology, but about people and about managing things and about getting stuff done — not necessarily getting the right stuff done.”
This is the whole point of agile: progress on iterations, inspect and adapt at the end of each iteration.
Your team might build the "wrong stuff" for an iteration, realize it (inspect), then make a course correction (adapt). If you end up delivering the "wrong stuff" is because you didn't follow this very core principle of agile.
I find it hard to believe these so called "Agile Early Evangelists" can make such a statement. Their background in lean development should have made the familiar with empirical process controls, from where lean and agile come from.
My guess the author quoted them selectively to fit the "agile sucks" narrative of the article.
With these things the reality of agile is within corporate constraints. The Big managers will want to steer the company and dictate deadlines or goals. While the pure form of agile might work the reality of agile causes a lot of friction.
Keep in mind though problems also show up with waterfall or young small startups. Its just which flavor of friction/pain project issue you are willing to deal with
I have been privileged to work in a company that really thought about and worked to prioritize the four values on the left. I would follow those agile coaches to any place they wanted. I have been a staunch defender in real life and online, because I've seen it work very well. I also have been on teams with other processes and seen how much worse it can be.
I will take the values of agile and push for those, and I'll take the lessons learned such as quick feedback loops, continuous integration, relative estimation, automated tests (which came from people like Kent Beck and Robert Martin pushing them so hard alongside agile), and the good stuff.
However, after seeing how badly it can be weaponized against developers, I'm certainly ready to throw out the bathwater, and I think this is what they're talking about. I've seen far too much cargo cult agile and far too much command and control with a light layer of SCRUM.
We have agile "coaches" who have never learned to code! They take a set of color-by-number technical practices but don't understand how or why they matter! I had to correct someone's slide that got the four values wrong, and their consulting group apparently had been copying and pasting them incorrectly from presentation to presentation!
The values and principles of agile are great. The current implementation has some serious debt.
(And while we're at it, we could update it. Too many people misunderstood the documentation part. Continuous attention to technical excellence needs to be upgraded to a value. Delivering frequently today means days instead of weeks.)
> However, after seeing how badly it can be weaponized against developers, I'm certainly ready to throw out the bathwater, and I think this is what they're talking about.
Poor management is a separate issue from agile. Even so, I would rather stay on a poorly managed agile shop than go back to a waterfall shop.
As far as the "values on the left", I like to explain them as a 55/45 split (and adjustable depending on your reality): we still deal with processes and tools, we just take a second to think whether a process is actually needed when we can just talk to someone instead.
Example: on a small team, you might just ask "can someone please approve my changes?" instead of having a whole jira workflow with code reviews and approvals.
The alternative is always Doing What Works. Alot of Agile is superfluous Waste. With time it'll only grow more added layers of inefficiencies. The true costs and risks always get hidden away.
> If so many people have trouble implementing agile maybe it just doesn't work?
The problem with agile and other empirical processes of project control is that it goes against the OCD tendencies of scientifically minded people. They assume that, since programming is all math and logic, software development projects should be as well.
They add micromanaging processes in a futile effort to control what they perceive as chaos and end up trying to fit a square peg in a round hole.
I.e. the old "you are doing it wrong" argument. My experience is that there is no right way that you can blindly apply on any project. There is no set of magical agile dogmas that works everywhere. And there is no substitute for decent engineers doing what needs doing.
Agile was a neat idea 20 years ago and it changed the engineering practices. However, unlike the methodology, engineering practices continued to change and modern software development practices automate away a lot of what agile processes tried to orchestrate.
If you are doing continuous delivery and lean development, you should be conceiving of and shipping software features in time units smaller than a sprint (i.e. continuously). That was very rare 2 decades ago and has become the norm for a lot of tech companies. It requires asynchronous processes and practices. It's enabled by having automated builds, automated tests, and automated deployments. These tools barely existed 2 decades ago and have had a far larger impact on software development then any form of agile.
Lots of large OSS projects and software companies have made the shift from doing feature based software delivery to time based software delivery. E.g. MS famously kept missing its own deadlines with windows vista, windows 7, etc. and shifted to having a more predictable release schedule. Ubuntu's LTS releases appear regular as clockwork in April of every second year. Linux ships a kernel every 2-3 months. Mozilla ships Firefox every month.
Time based releases are basically about releasing an unknown quantity of software at specific intervals and with a high level of quality. It involves having multiple asynchronous tracks of development and instead of planning which of them need to be ready they simply use quality gates to determine which of them are actually ready to ship. It's a shift from what to when and it emphasizes quality (i.e. good engineering) over schedule.
Most agile methodologies are still stuck trying to do feature based planning. It's the project mentality from the nineties where things get commissioned and have to be delivered on a particular schedule. Worse, these things are often under specified and then blow right by their planned deadlines. Just like in the nineties. I've seen a lot of agile projects shipping low quality software doing the wrong things right on time.
Nearly everywhere I've worked the quarterly earnings report is the problem. Due dates mean that time has to be managed so any attempt to use a methodology seems to regress to some not fun way of developing software.
In places I've worked without the pressure of the quarterly earnings report I've seen that developers tended to use whatever methodology worked best. It could be Agile, pair programming, or just walking over to each other's desks and talking on IM.
In my experience "Agile" whatever that means is still the best of all the bad options. Is there really a better methodology? I feel like it works pretty well as long as management doesn't pick and choose the easy parts of it and then add their own layer of BS on top, which is very common.
Agile is the worst form of Software Development Methodoloy, except for all the others.
Which is to say: To all the people jeering for Agile's demise, please provide a superior alternative. I came from a world dominated by Waterfall, and I never want to go back to that. A lot of companies get Agile wrong, and it isn't a panacea, but it is much closer to what developers are naturally inclined towards (e.g. rapid incremental improvements) than Waterfall ever was.
So I challenge anyone who wants to replace Agile, please lean into how developers work rather than trying to mold their work onto your rigid front loaded methodology. Trying to bring in ideas of other industries, like engineering or architecture, that build physical goods and only have one shot at is a folly.
Yeah people have short memories of the pre-agile role demarcation and wasted handover time between them.
It was utterly frustrating to be powerless to improve specification quality and find out you built exactly what was asked but you were asked for the wrong thing.
The silos were horrible. Devs were often as bad as BAs, dev’s just were just crapping on the testers instead. Spare a thought for the production ops person at the very end of the chain. Here comes 3 months of developer work in one weekend and no it doesnt work but we’re still going live because the entire tech org is invested in this release. Now you have smaller squads, you can postpone a release without it looking bad on the top person and thus affecting your career prospects.
In agile models you have the power to fix crap processes that don’t work. You can call out any BS on the retrospective and make it super visible when things are being swept under the rug.
I worked on the customer delivery side of a $750M software project that was a steaming pile of shit, with critical defects that were known for a year with zero effort to address. The integrators were paid to deliver a spec, and it took about a year to get change orders created to diverge from the spec.
The old school waterfall outsourcing models are truly awful, but you don’t really understand how bad unless you’ve lived it. Stupid agile religious practices are dumb for any startup type org, but are probably better than the alternatives in many big enterprise scenarios, where the goal is chunk up the work so marginally qualified people can do something.
> To all the people jeering for Agile's demise, please provide a superior alternative.
The goal of Agile is to let smart people work incrementally towards a somewhat nebulous goal. The ceremony that's arisen around that tends to be put in place by people who feel the need to manage, but don't know how to help their developers achieve that goal.
The alternative, as far as I've seen, is to hire smart, curious people, let them work closely with the end user, and pay them a lot of money. In this situation, the engineers will typically self-organize effectively.
My org was like that when it was 20 people on one floor doing pretty much their own thing in groups of 2-4.
Now it’s 100 people across two countries, expected to collaborate closely with sibling orgs. Management philosophy has shifted to “we will ask a few handpicked experts to write down the best way to do a thing, and then everyone else is a machine for executing that procedure.”
If software engineering worked like that, we would have automated it.
But, see, Engineers do not self-organize effectively, without some kind of frame of constraints to guide them - customers/users, design, time, quality, tech, architecture, etc.
Assembling a talented team that ultimately delivers nothing of value is practically a cliche in this industry.
The problem with Agile is that vanilla Agile almost certainly doesn't fulfill your company's requirements. However, if you put some serious thinking into tuning it for what you need, it will be serviceable.
Agile would work much better if every implementation stated upfront, "Does not work right out of the box".
I think you're debating a strawman there. The argument is not that we need to return to other pre-agile SDLC methodologies but rather take what we've learned and go further. From the article:
> Now, continuous delivery is what’s expected, and the industry is ready for the next thing. But that next thing shouldn’t be another methodology, according to Mary.
> There is no methodology in my field of software engineering that can conceivably last more than five to eight years,” she said. “Everything that is 10 years old is obsolete. Everything that is 20 years old is archaic.”
> Furthermore, she said, methodology requires codification. Beginning with the Capability Maturity Model (CMM) in the ‘90s, software development methodology meant developers had to show they had standards and that they followed them, rather than demonstrating that their standards were constantly in flux depending on consumer needs. That’s the definition of quality standards lean manufacturing practitioners in Japan originally espoused, Mary said, and they’re not compatible with methodology. Instead, they’re all about learning.
> To that end, Mary is excited about all the ways artificial intelligence will allow software engineers to learn better and faster. Automated testing, continuous deployment and cross-functional collaboration are now table stakes, Mary and Tom agreed. Cutting edge companies will discover the next great approach through an engineering mindset and a willingness to learn.
___
Consider that Mary and Tom Poppendieck were responsible for many of the Lean inclusions of the agile movement, which (largely) came from watching plant manufacturing at Toyota. Similarly, much of the DevOps movement was tied to this as well. If you want to talk about what is next, it is likely taking a first principles look at what we're doing today, questioning best practices again, and saying, "if we were going to make a manifesto in 2020, what would it look like?"
You claim I'm "debating a strawman" but then quote the article that literally says they have no alternative in mind (my chief critique) and point towards artificial intelligence to solve the issue somehow.
The only "strawman" here seems to be taking my point about pre-Agile methodologies out of context, and using it to dismiss the entire idea that this article is unconstructive/has no actionable solutions.
Nobody you replied to said they shouldn't be allowed to criticize Agile.
The criticism in this comment thread was that their advice wasn't actionable, not that actionability was a prerequisite in order to criticize Agile at all. There was no false dilemma here because there was no choices provided (false or otherwise), merely a weakness in an argument raised.
The advice they are giving is to stop trusting methods, take the best practices that make sense, and trust your engineers to build a bespoke process from those building blocks.
> “I don't care if it’s Lean or Agile, there’s no silver bullet where if you just follow this formula that somebody else followed, you’re going to be great,” she said. “So today, my favorite word is ‘engineering.’ Just let engineers be engineers.”
Maybe we need to call it the Lego process. SCRUM is Duplo. Waterfall is worse Duplo. Start there if you need, but get to Lego instead. Maybe Duplo is enough for you.
> "Everything that is 10 years old is obsolete. Everything that is 20 years old is archaic.”
Which is not entirely true since some of the best parts of Agile are XP practices that almost everybody does by default now...
Someone once said that the only successful Scrum projects are also doing half of XP (as in, things Scrum doesn't prescribe but are essential anyway). I am still looking for a counterexample.
I honestly don't understand anything that was in this article. Continuous delivery is directly applicable to agile. They are complimentary, but not even really related. From my point of view, the essence of agile development is just continuous improvement. The notion of user stories is a proxy to delivering a small unit of user value from idea to tested and deployed as quickly as possible without having to worry about future requirements.
> To all the people jeering for Agile's demise, please provide a superior alternative.
Give individuals ownership over different parts and then let them self organize, that is all you need if you hire competent people. I'll never work at a place which doesn't work like that again.
To be fair, there are a lot of engineers on the market with a small-but-positive value under a structured process who would be totally useless in an environment like this. I can see why it might be rational to managers.
The problem is that software engineers are paided poorly compared to bankers, executives, and a myriad of other occupations relative to how it used to be 30 years ago. Even doctors aren't paid like they used to be. So way too many smart people go into finance and real estate, which are jobs that really anybody could do, except that our corrupt system makes it worthwhile to pay the smartest people to come up with new ways to scam the public and the government. See: Fed policy, CDOs, sub-prime loans, QE...
The public absolutlely paid the price in the form of bailouts. If those banks had been allowed to fail I would agree with you, but they knew they'd probably be bailed out from the beginning and used CDOs to ensure it.
QE is a direct transfer of wealth from dollar holders to investors and banks. No way around that.
Yeah, this is always what I hear from selfish people. What is "competent"? Even if I know what that is, hiring them is hard and ad-hoc. Agile is a way to orgnize _everyone_ to deliver. Even "non competent ones". Those teams will deliver slower, but will deliver, instead of failing.
There are plenty of examples. The Linux kernel development process comes to mind.
I suspect that most knowledgeable developers rolled their eyes when Agile was "invented". It's a mish mash of a lot of things that were already obvious at the time and some weird kool-aid like pair programming. And just one more in a long line of consultant enrichment schemes.
Agile is the worst form of software development methodology, second only to waterfall. It's still pretty shitty, though.
One of the reasons Agile (with a big A) is so successful is that its peddlers have convinced everyone that there are only two options: Agile and waterfall. When you're up against a strawman, it's easy to win. But it's absolutely a false dichotomy.
There's a quote in the article that I like:
> Find me an actual tech company that talks much about Agile, and I will be astounded.
In my experience, people at tech companies (at least the FAANGs) rarely talk about Agile, although they do talk about things like continuous integration/testing/deployment. They do not obsess over methodologies for how to move post-it notes around a whiteboard (Scrum vs Kanban) or agonize over how to word a user story narrative, or other parts of Agile Theatre.
People at some non-tech companies, especially those supposedly going through a "digital transformation", seem to have fully bought in to the crap that Certified Agile consultants are selling though.
So a superior alternative to both Agile and waterfall is what's in use at a lot of the big tech companies. For example, the engineering culture at Google, which relies on design docs and a very good set of developer tools and infrastructure. It's not perfect, but it's far, far superior to Agile.
And before anyone makes the argument that those non-tech companies are not doing "real Agile", but Google is, let's be linguistic descriptivists and accept that Googlers don't call their methodology "Agile", whereas the Scrum consultants at big corporates do.
This is ultimately the point I think Mary is trying to make. Agile became and end to itself rather than a way to learn how to build software within minimal waste and delay given imperfect information. Obsessing over it is a cargo cult.
That said , plenty of tech companies have inconsistent practices that don’t necessarily lead to great success. To say Google’s methods are “far far superior” to Agile assumes they’re uniform, portable or relevant externally, or that they lead to above average success. That’s debatable given Google’s reputation for abandoned half-built products and an almost comical lack of customer focus.
Yeah. I maintain that most complaints about Agile are actually just complaints about crummy managers. It's really not that bad (in my experience) when the people you're reporting to have reasonable expectations and treat estimates as estimates.
Please do not aim to replace Agile, you'll just be creating new problems. Go back to the basics of the Agile Manifesto, understand where your organization fits in the problems that it aims to fix, then overview the different solutions proposed by various methodologies, pick and choose the ones that address the biggest problems that you're facing and adapt them for your own purpose. Start small, as each solution may introduce some processes and distract from the actual mission. Augment only when necessary. You're Agile.
You don't need to buy entirely into a single methodology. When people say Agile in software today, they really mean the Jira-flavored Scrum. Some now claim that they've abandoned Scrum and you hear more about Kanban. Sometimes they really are switching, other time they're really just doing Scrum with a Kanban board. But again, they're often falling into the same traps of forcing solutions, rituals, and processes they might not really need into their flow.
You want to be Agile? Start small. A simple checklist is a good way to start.
Agile's pretty much only defined by being "not waterfall" (which is defined as "not agile, and worse") and "if it doesn't work, you did it wrong", so that's sort of a tautological statement.
I currently work at a FAANG and we have a chaotic, low-touch approach that honestly seems to work pretty well. There are only two issues I see with it: it requires a high degree of trust in the engineers from management, and it could be possible to skate by doing the bare minimum without it being noticed (it is noticed usually, it's just not publicly shamed in regular standups). This has the major benefit of allowing developers to manage priorities as things change/pop up/assumptions change, and gives them the latitude to take on complex tasks that aren't easily decomposed or estimated a priori.
I think part of the issue is that Agile is meant to be a Process imposed from top down to improve productivity by X%, a way for management and a small army of backlog/task managers to say they are doing Something and having impact (or literally the only thing they actively do). Not only can it easily get in the way of that, it almost always fosters an environment of shaming and lack of trust, because it is too tempting for estimates to be taken too seriously or productivity/performance measured "objectively" by invoking the task-tracking system (in which case you incentivize only taking on very easy, very well understood tasks). It doesn't always devolve to that, but I think at many places if anybody Important up in the management chain starts thinking that way, it will inevitably trickle down. Thus any sufficiently large organization will corrupt Agile.
Perhaps some companies need that level of accountability and visibility, even knowing and disliking the drawbacks, but not all of them. I am honestly not convinced that a rigid process is necessary at all. Yes it makes sense to have a system where you keep track of things that need to be done, but if there's a culture of trust, I don't think the system needs to be gamified or fetishized as much as it usually is.
I agree with the title but find the reasoning hard to follow. It seems like the author keeps changing playing fields and tries to stitch a single narrative.
The main issue my teams have been facing with Agile methodologies is the number of the workflow interruptions built in. The only team members happy with those interruptions were Scrum Masters and PO's but that's because those were parts of their flows, so they had to have them in order to have their job done.
The only long term solution for us was to have days dedicated to Agile ceremonies and days where no team member can be interrupted based on an agile ceremony need (there were few exceptions, like critical bug discovery, etc.). In weekly sprints, we had at least 3.5 days of uninterrupted time and in two-week sprints we had 7-7.5.
That changed a lot the pace of those sprints as people had a lot of time to do their job and everyone was happy.
Everything but the daily standups... We had each day to begin with it, and it was great as it provided a more broad visibility into what is being done, if there are any issues, etc. But every other agile related meeting was a ceremony we have tried to manage as best as possible while not blocking its benefits.
485 comments
[ 3.6 ms ] story [ 326 ms ] threadI'm working with a team that the PM follows Agile like its a silver bullet solution but is obviously terribly flawed.
Our stand-ups are pointless because they're just status updates. Not much of consequence can happen in 10 minutes anyway. I ignore our metrics at the end of each sprint. They're meaningless. Our retrospectives are largely useless other than giving shoutouts to team members. I feel the sprint structure leads to Parkinson's law.
This situation, however, is perfectly normal because Agile advocates "people over processes." My team doesn't give a shit about the Agile process, so it's mostly like forced physical training but without the fitness benefits.
When you throw in the incentives of compensation and career advancement, politics and human nature are bound to corrupt the process further.
This is why codified process will always be better in the long run for most companies, because it's the only tool possible for combatting this
How do you create a paradigm or process that can be amended adapt to a persistent threat without that amendment process itself being used as a tool to corrupt it?
Bureaucracy, heavyweight process, and all rest are ways that systems protect themselves. Wasteful though, which is why new paradigms such as Agile come along.
Now the Agile ecosystem has become corrupted by a process not comparable to all the above. Not solvable by those means either. It’s a toughie.
I think that’s what’s missing in most companies. We have 360 reviews, retrospectives and all kinds of stuff but there is no feedback channel up the chain. At my company whenever there are obvious problems management secludes itself and then a solution will be revealed to the underlings who never got heard. Especially in tech it’s pretty safe to assume that engineers at the bottom of the pyramid are as intelligent and educated as people further up the chain so I think their input is valuable.
It is human nature that when talking to your boss you put things in the most optimistic way that fits your understanding, and your boss hears it even more optimistically. After a few steps up the chain this results in a complete disconnect from reality.
http://www.stamey.info/Humor/shithappens.htm is a humorous but basically accurate take on the dynamic.
Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
I learned this the hard way, maybe 3 months into my first real job. I was testing safety critical systems, but it was still being developed so the tests procedures were also being developed and executed. I was asked for my status, I said: I've tested about 60% of the system (this one was small enough that that sort of statement was actually valid), everything has passed so far. My boss heard: Everything passed. He released the build to the customer, who found a failure almost immediately (about the same time I did, but I had no idea it had been released). I was taken into a conference room and chewed out for making us look like amateurs (we were, when it came to software). I learned several things: 1) I needed a new job; 2) Never use the words "done", "everything", "all" until everything is actually all done, managers only hear those words and nothing else.
> Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
A manager, fortunately not mine I was his peer though he got promoted for his "successes", would always say that his products never shipped late. "How can you say that? I know your last release was 3 months late." "We updated the schedule and we hit the new schedule." "But you didn't update it until the last month. It was 2 months late by then already." "But we hit the final scheduled release date." He just got lucky that the customers weren't loud enough for his boss to realize the spectacular, repeated failure until after he got promoted. The new manager got hit with it instead.
Well, who would have thought :)
Me > Well um.. because I just thought ticket B is a three and this is comparable complexity.
Lead Dev > Yea... this is a two. I mean ticket B.. that can be a two too.
Me > Ok
Product Manager > Next ticket. Let's say a five? Or a four. Everyone hold up your fingers, is this a five or four
Me > (Looks around to see the popular choice. Holds up the same number of fingers)
He then went off on his own to talk to upper management and committed, without our input, to large new pieces of work, and was told what the budgets for those deliverables were.
At the next sprint planning, he was basically telling developers their estimates were wrong because the number of story points we were deciding equated to more money than the budget for those features.
Rather than play along, I simply said "Well why don't you come up with the estimates for us, then?".
To his credit, he did then stop trying to influence the votes after that, but it wasn't long before the product was put into maintenance mode and he was assigned to a different team.
This is not to say that management isn’t useful or necessary, but they are a facilitator of work, not the patron.
There's a lot of companies where delivering what the consumers want/need is not the end goal of the company, at best it's needed to help fulfill the end goal.
A key success of agile was to provide a buzzword that let engineers say no when they were asked for a fixed estimate for a fixed-until-it-changes scope to fit in a Gantt chart.
Take agile away and they'll be straight back to demanding to know how long it's going to take to build something when no one's really sure what the something is.
https://youtu.be/a-BOSpxYJ9M
I said that it would take me 5 minutes to go into Jira and close out all the existing issues if that is what they are worried about?
That's it. The more people are involved, the more complicated the process gets and all of these approaches evolve out of trying to find an agreeable and effective way of doing it so that everyone doesn't have to figure out a new solution every time.
As much of a buzzword hell as it is, I really believe that Scaled Agile Framework (SAFe) is the closest thing to the right balance of trade offs.
I thing “good, fast, cheap; pick 2” is still generally valid.
So I always felt that fast and cheap go hand in hand.
I think the triangle is:
- features
- quality
- cost (=both price & time)
Edit: I forgot to mention that you cannot just throw more engineers at a problem to make it go faster. There is such a thing as an optimal team size.
Another edit (sorry it's Friday evening here;)): The faster you can get it done, the cheaper it will be.
Therefore, if cost must be pegged due to limited time/funds and quality must be pegged due to existential risk then quantity/richness of features must float and be flexible.
All good product owners will learn this truth eventually.
This is unlikely to be true. Bad decisions are your biggest cost.
An employee has a "return multiplier" - for every dollar you spend on them, what is the return to the business? Sometimes it is high. If so, the developer is very cheap. Sometimes is it low, or negative. If so, the developer is expensive.
Why would it be high or low? Part of that is developer skill, but mostly it is how well your organization runs itself and makes decisions.
Organization is definitely a big factor.
But when I said developers are your biggest cost, this is from an accounting point of view.
Or until the ever-growing pile of garbage that agile produces gets too tall and tips over. Then you throw away the codebase and start again :).
I've seen this before. After the second iteration it becomes neither fast nor cheap.
Management was not a huge fan of this piece of paper.
When you finally liberate a team from Agile, it's just breathtaking how much you can focus on delivering working software that gets deployed with quick iterations that's closely aligned with the business and customer's needs. When free from the tyranny of Agile, teams can be effectively self-organize, remove micro-managers, and quickly adapt to changing needs and requirements. My experience is that staff are usually much happier, more productive, and less stressed once agile is gone.
I mean contemporary Agile as pushed by corporations and those awful "coaches" (who never seem to be actual developers) --- if you were you design a system whose end goal was making great developers unhappy, unproductive and locked into a dysfunctional system, Agile would be it.
That's literally the agile manifesto.
"Scrum" is one implementation of what agile was trying to achieve. Maybe that's what you're thinking of?
I know, that's the irony. Agile as pushed today ends up with the compete anithesis of that. It's the difference between "Agile" (Capital A), and "agile"
Read SAFe Distilled or SAFe 4.5 Reference Guide for what happens in large enterprises (god SAFe is a fuck up). It consists of some good ideas, but also some very explicit practices that don't help in every effort (and sometimes hurt). It is literally the opposite of Agile, which is supposed to be about flexibility.
If you want to hear about its "success", just remember it was used for F-35's software...
Big-A Agile is based on the belief that adoption of practices is sufficient, and that deep understanding is unnecessary. This is the realm of cargo cults.
This is the example I use all the time!!
Maybe agile is great when you're making an online shopping cart for a e-commerce website, but the idea of using Agile for complex, engineered systems is laugh-out-loud hysterically absurd. And this does not just mean spaceflight software, it basically means anything more sophisticated than a CRUD app.
The thing people have to forget is the notion of "deliver on two week increments". It's not about delivering a product that can be fielded each increment, in the case of an embedded system, but about delivering something that has some improvement or new testable component. I can't make an embedded radio handle, in two weeks, a completely new message type (well, depends). But I can do things in each two week interval that is verifiable. I can show that I've actually received the new message, that I can send it back out, that I can pass it through the various internal processors (if multiple processors are used) or processes (if a single one). Then I can start transforming it, storing it, changing other things about the radio state based on the message contents. Each of those is independently verifiable and completable within a short period of time. But taken as a whole, it's a 6-month project. The agile way has you make those small things, verify them, and then move on to the next thing. I can deliver (to the test team, to others using the system) the partially-completed system, it just can't be fielded (and that's fine).
And that's not unique to embedded. If you only focus on things that can be fielded in each increment, you'll never develop the more complex tasks, or address the tech debt.
For example, there are days towards the end of the sprint when I’m not allowed to pick up new work and the infinite wisdom of the agile coach is that a ‘clean’ board by the end of the sprint is the what we’re really being paid to deliver (and starting something else would compromise that). This runs alongside serious customer deadlines which are hidden by the dates the Agile program runs by.
It can still be evaluated on the merits but IMO this greatly pollutes the speed at which software devs as a broadly conceived community can come to consensus understanding of this.
Also I think the comparison to lean manufacturing has always been very shallow. I get the metaphor, I just don't think that human resources in engineering can be optimized like manufacturing processes. This quote is the best part of the article:
> "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".
As for the rest of it, I'm not dying to hear what the person who invented Agile thinks we should do next lol.
I have literally heard this pitch.
More generally, it's correlated to it bringing some real or perceived advantage to the person doing the talk.
I mean, have you been watching the news lately?
https://www.bloomberg.com/news/features/2019-05-09/former-bo...
The closest it gets to that is where it say:
We follow these principles:
<snip>
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
<snip>
It basically states a set of attributes your process should exhibit.
Agile as I have experienced it in practice almost never displays those attributes.
The whole thing makes very little sense. I mean read it.
>"We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more."
How the hell do you get from that to where we are today?
What it ought to say is:
"We value short feedback loops that minimize risk and maximize learning.
We value flexibility.
We welcome change.
We don't know what we are doing, only what we intend to do. The outcome is a guess. Our guesses could always be better. We strive to make them so."
First words of an agile consultant.
First words of the next agile consultant.
First words of an agile zealot responding to any complaints or negative comments about agile.
https://i.imgur.com/ecsh9yp.png
Agile has plenty of good ideas.
The problem is the almost religious following and, as you mention, the whole industry that has sprung up around it.
Even the initiators said that it was supposed to be a rough framework, to be adapted to your individual circumstances and teams. It was also a (much needed) counterpoint to the then prevalent waterfall model.
Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")
When you step back, a lot of the ideas make sense, and many teams will even implement similar workflows without having ever heard about "Agile".
Common sense has to prevail though.
It still is and it's barely in the fight still in some corners of the public sector.
If Agile dies (and SAFe might succeed there) then there would be nothing left except Waterfall and people others would call cowboys.
Saddle my horse.
Whether it was right, I'm not sure, but people have been happy with my work, so that allowed me a bit of leeway.
The team/organization has to become a learning organization. That means a number of things, but the critical one here is learning from past failures and successes and incorporating that feedback into their model (ideally continuously, but in Scrums it'd be the end-of-sprint retrospective). You start down a path, and you find it's difficult. You don't press on just because it's the one you selected, that's the way of idiots. You examine the hardships you're facing, and you address them.
> Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")
Yeah. Part of agile is that your process is an adjustible parameter. If you're doing it with a rigid process - any rigid process - then you are not actually doing agile.
Yes. I have found doing agile "properly" to be too rigid. What we need is something more flexible than agile.
Agile and Lean are empirical process controls, they are based on the same concepts. Ken Schwaber explains all this on the first chapter of his book "Agile Software Development with SCRUM":
Defined process control: same inputs always result in same output (manufacturing widgets on a production line).
Empirical process control: same inputs not always result in same output.
Schwaber conceived SCRUM (and was among the founders of Agile) after realizing software development required an empirical process control: give 2 dev teams same specs, 2 different apps will come out (they might do the same thing, but in different ways)
Which bothers me much, much less than selling things like CMMI/PSP certifications or EUP/RUP which are done purely for paper pushing and selling the paper value.
Agile is an improvement on waterfall and you don't need to be certified to do it.
Absolutely. And, having done some agile consulting long ago, I'd say it's more pernicious than that.
The people who truly want to make deep change in pursuit of deep improvement are a small segment of the market. Worse, they don't need a lot of help. In the aughts, I had a few clients who really got it after 3 months of focused work, and then they were off and running.
But a large company that only wants to talk about change and maybe make some 5% improvements if they aren't too hard? That can be milked forever. Well, I can't, because I care about results. But consultants who either don't care or don't notice? They're golden.
And I think this failure of the Agile movement has been obvious for a decade. I gave up on Agile conferences circa 2009, and wrote a long piece about this in 2011: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...
I've seen several director types get promotions promising to get faster and more reliable work out of existing engineers by implementing this new religion... Agile.
The results were always predictable. Layering more meetings and stress on people who already have too much work to do, doesn't help things. But, you're still vice president.
> > "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".
This quote is silly.
To me, agile is just good engineering practice, applied to software. Of course mechanical engineers apply its principles, and have for decades before the term Agile was coined.
And as such, this practice is far older than software.
The Apollo space programme is my favourite example: the ultimate goal remained fixed (man/moon/before end of decade), but all steps of the way were discovered and redefined over the programme's course.
Mission objectives were changed depending on what was learned, often even in flight.
This was a very nice and agile (and sensible) approach, regardless of what it was called.
Of course, even more ironically, it doesn't fit the original agile manifesto:
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
https://agilemanifesto.org/principles.html
I'm not so sure.
What's the official definition? Don't give me the manifesto, that's just an implementation of common sense in development, as applied to software.
As a trained mechanical engineer (maybe that's why the quote about mechanical engineers irked me especially), the agile manifesto just reads like good engineering practice to me.
And indeed the signatories of the manifesto expected to start a huge discussion on what agility meant for any particular team or product, and how to best live up to its principles.
> of course [it doesn't fit]
That was uncalled for.
> and especially any agile consultant's.
I'm (something akin to) an agile consultant though :-)
Also, just to point out: maybe my definition of Agile diverges from canon. That's OK. I don't care to follow the canon, I care to do right by those who I've been asked to support.
> Of course, even more ironically, it doesn't fit the original agile manifesto:
> > Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
I'm not sure how you came to that conclusion.
How is "welcome changing requirements" a bad fit for "changed the mission objectives (as new information emerged)". Clearly the [competitive advantage/likelihood of success] was increased? In fact the entire programme was shaped such that change was explicit part of the plan.
I've stared at this quote for a while now, and I still can't see the contradiction.
I understand why it would be super cool if software worked like that, but I have to iterate on features holistically and sometimes speculatively.
At least for me, software development has never been able to be so cleanly broken down into "first, implement the button" type tasks.
But maybe I'm just dumb.
YES. THANK YOU! I've been trying to tell people this for ages. Maybe other people are somehow capable of perfectly predicting every conceivable edge-case and consequence without even starting to work on something, but I can't. I have to start building something in order to see it clearly. And this ENRAGES certain kinds of managers, I have found.
Here's how you can solve your problem: Make as many tickets as you can think of, it doesn't really matter if they contain the essence of the problem or not, it just matters that there are many tickets associated with the task. Once you have your tickets, estimate them with as much padding as possible. If the manager complains that you're padding a ticket too much, make two tickets and split the estimate.
Once you have a big pile of tickets and estimates, you've bought yourself enough time to do the work. Go do the work, then mark all the tickets as complete. You'll figure out the edge cases as you go, as long as you've made enough tickets and time to do the job at a reasonable pace.
If you have extra time left over, get started on the next thing, improve your tooling, or try to help someone else with their tasks. If you don't have extra time left over, make more tickets next time.
My boss's goal is to try and get everyone to work every waking hour so we can deliver faster. My job is primarily to keep the peace between the executives and the workers. When I fail the workers, it looks like unpaid overtime. When I fail the executives, it looks like undelivered features.
I think the actual purpose of agile, from a developer's perspective, is to create more time to deliver quality engineering work product. From an executive's perspective, agile exists to squeeze developers into completing tickets faster. As a manager, it's my job to keep both parties as satisfied as possible. If engineers have enough time to build and executives get enough deliverables, then the company succeeds.
What's your strategy?
As I noted in a parallel comment, on my team, developers are never creating tasks on their own or in a vacuum. It's done as a team effort during planning meetings.
Perhaps I should add that my boss (senior director) has never asked me about my board, velocity, or anything like that. Discussion is higher level - "is project X on schedule?" or "are you going to miss anything on your roadmap?" not "why is task X or story Y not done yet?" If your director is asking that, he has too much time on his hands.
Asking specifics from higher ups is just smart.
I've reported to two different directors since moving into management. Both were very clear that they'd rather hear about problems early. And both have been nothing but reasonable when I've reported problems. Of course, the entire team needs to be reasonable for this to work, from the product managers to project managers to individual contributors.
And as a manager, nothing annoys me more than an employee who tells me everything is fine then misses a deliverable. Tell me you're having problems so we can find a solution. Please. That's why I'm paid what I'm paid.
Nobody can; that's the primary failure mode of waterfall. Anybody who is telling you to decompose a feature into that many subtasks might claim to be doing agile but they really are trying to do waterfall.
That's the primary failure mode of a straw man software development methodology called waterfall that Scrum practitioners like to use as a bogeyman to appeal to for rhetorical purposes during the inevitable arguments about niggly little details of how to implement Scrum during sprint retrospectives.
I've never actually done real waterfall, but I did study it way back when I was in college, and I have worked in government contracts, which tend to handle the work in a way that is similar to the waterfall I read about in school. One thing I distinctly remember is that it was designed to be a flexible and iterative methodology that would have been difficult to cram into a ticketing system in the first place, let alone cram into Jira while simultaneously carving the work into tiny pieces, up front, all at once.
FWIW, the only place I've ever been asked to do something like that is in ostensible Scrum shops. In fact, the only time I've ever seen anything that looks like the Waterfall of scrum lore is in shops that are trying to do Scrum. I am beginning to suspect that the Waterfall of Scrum lore isn't actually a thing that happens outside of Scrum at all. It's actually what naturally tends to emerge when you try to apply Scrum methodology in a situation where something along the lines of the real, textbook waterfall model would have been more appropriate.
As a concrete example, I currently work at a team that ostensibly uses Scrum, but where QA is a separate department with its own practices that the dev team does not control. They do still want some ability to anticipate work that's coming their way, though, so they're monitoring the scrum board for that purpose. This is the moment where it gets messy, because we're then asked to document things ahead of time, and it's a minor crisis if we keep it flexible during the sprint because any changes to the set of tickets becomes an inevitable hassle as they complain that we've screwed up work product that they generate based on those tickets. It's absolutely something straight out of waterfall horror stories from Scrum lore. But it's only happening because we're doggedly insisting on something that's at least cosmetically similar to scrum. If we weren't doing that, we'd be freer to choose modes of interaction and business artifacts that are better suited to the reality we occupy.
The biggest difference for Waterfall-in-Scrum versus Waterfall is that they've sufficiently (hopefully) decomposed tasks to have short target dates for delivering something to a test team or facility. They don't use it to get feedback from customers (the actual users, not the ones writing the checks). They don't use it to replan when things go wrong. They just use the whip and OT and get back on track.
Out of this chaos, eventually came order. It turns out what mattered most in practice, just like before, was pleasing managers and executives. Effectiveness was generally secondary. So what we ended up dominating is Scrum, the most waterfall of Agile processes. Most shops "doing Agile" these days are still following a top-down, control-oriented, ineffective and unrealistic process, just like before. But now they use Agile labels for their mostly-unchanged process, albeit with a faster release cadence.
* A quarterly release cycle is indeed fast. * 12-18 months between releases (ignoring perhaps bugfix/point releases) is quite common.
You write that "everybody knew that couldn't work" - for some projects.
Definitely agree with your second paragraph - except that Agile labels can be used even without a faster release cadence :-)
It has always been at systems thinking, design and architecture, in that order. Doing it organically is a magic feat rarely done well.
What I do with my guys is I make them decompose the feature, but with the understanding that they'll probably be jumping from one subtask to the other, back and forth, as they understand what it is they'll be building. AND they'll probably come across new subtasks as they work through the feature.
It makes Jira look crappy, but it Reflects Reality (tm) which to me is far more important than keeping Jira clean.
No, nothing is wrong with you. But let me note that the Agile Manifesto doesn't say anything about Jira, or anything prescriptive about how small to make your stories.
If your methodology asks you to do that, I'd argue that your methodology is broken. But it's broken because it's shit, not because it's "agile."
Yeah, it's hilarious how people are being told to do something in the name of "Agile" that is almost completely opposite of the spirit of the Agile Manifesto. sigh
Parkinson's Law doesn't get enough of the blame for bad dev environments. It's so easy to feel like you're doing 40 hours of difficult, important work a week, while actually accomplishing fuck all at best (or generating more work overall at worst).
I bet a big chunk of the problems raised in this thread don't exist if everyone experiencing them cuts their team size in half.
To be able to decompose a feature, you need to learn how to build the software in your head before you go to paper. Consider a large billing page, for example.
Step 1 is to look at the front end. What components do I need to build? (React might use components, or a set of partials and templates in rails, for example.) So, then, I know I need X subtasks, where I will build the pieces I need to expose that behavior to the client to start.
I then look down into the backend. That might be a simple task on the backend, or given 20 minutes of looking, I might see problems that I'll need to tackle. The problems are their own subtasks. I may be able to group some of the front end to a single back end ticket, or I might need a separate subtask for each.
Now, I'm likely to miss something. However, that's why this is an estimate rather than a concrete task list. It's to get started so that we know, at minimum, what it's going to take.
It's not necessarily simple, but it is a skill that can be developed.
Even easier than building the software in your head is actually building the software. I often don't know what the shape of the problem is until I've built something. The users often don't know what they want until they've seen something.
The failure of Agile as implemented is that it's still all about extensive meetings and planning. Software is not like building a bridge where you write out the blueprints and the expensive part is then building the bridge. Software is the blueprint.
To be truly agile is to be able to get software features and changes running and out to users as quickly as possible. You can never get all the information -- if users could provide the details absolutely necessary to build the software on day one then they'd be the programmers. Being experienced helps you naturally scope problems but you'll still be wrong from time time. Being able to iterate on users feedback and dealing with your own failed assumptions is the key.
This is the majority of estimation failures as far as I've seen. You can only "estimate" so far and then leave significant padding for all the other minutia that will likely creep in.
You should still have more granular planning, but it should serve the purpose of aiding the dev in completing the feature. You decide to do more granular planning when, as a dev, you look at a feature and go "The implementation is still ambiguous or I still have questions about how it relates to business goals".
Basically, invert the communication channels and shift responsibility to the devs. They have some loose business goals to hit, and the other staff provide support when they need it.
This solves so many problems I run into in enterprise dev. It kills Parkinson's Law (since the team members that are in the bottleneck are now responsible for the bulk of work generation within the team), it also gets rid of a whole class of poor decisions that are made without the proper context (since now you have a closer relationship between business goals and development).
First off, it means your code is brittle. Second is that you can't trust tests that must change every time your code changes. Might as well not even have them.
Unit tests are supposed to tell you if your refactoring is still working. For the most part, you should only refactor implementation details, not the "API".
There is also a design to uncertainty trade off. The less you know about something, the less time you should spend designing because you can't design something you don't understand. But the more your understand something, the more time you should probably spend designing.
Knowing how much design is too much is subjective and personal. I forget where I read it, XP(Extreme Programming) or Code Complete, but someone worth listening said, to paraphrase, "I've never regretted spending more time on design before I started coding". And I'm pretty sure it was part of XP that said the main purpose of XP(Agile) is to deal with the unknown. The more you know, the less Agile and the more waterfall you should be, saying that for some projects, a 6 month sprint is perfectly valid.
I am in an interesting situation where I am both the programmer and the domain expert. Generally customers come to our team describing a problem that other companies can't figure out. We're pretty much allowed to design everything how we feel like because we're the best at it. We can do virtually any project in a single go of it without feedback from the customer, but we still do agile because it helps with scheduling, planning, prioritization, etc.
We're a "value add" team, which makes things really interesting. None of our work is sold, but the work is important to land large contracts for the companies core services.
No, not at all.
Decomposing a new feature into digestible pieces is hard work and a skill that is learned over time. It's also a skill to know when something is small-enough, or has enough unknowns that further decomposition is wasteful.
Good product owners and managers know this and are good at it themselves. And the decomposition is typically at a story level - "can we take story X, break it into 2 smaller stories, and still deliver something of value?" The tasks are usually pretty self-evident once you break down the stories a few times.
For a lot of my work, if the story is "As a user, I need to do X" the tasks are simply broken into distinctly testable pieces.* Is there a database change? That's a task. Is there an API change? That's another task. Front-end change? Third task. And if you get half-way through and need to start over, that's life and it happens reasonably often. Yeah, there might be some re-testing. Or, a new story for next sprint.
It's up to me, as the manager, to make sure the stakeholders are informed and on-board. At the end of the day, as long as the team is making progress, I'm happy. And if the team isn't making progress, it's probably my fault (and almost never the fault of an individual contributor).
* In my world, tasks don't exist in isolation. They're all subordinate to a user story. Every task (piece of work) we do is done in the furtherance of a well-defined need of the user. And developers are never defining tasks on their own - it's done as a team.
Forget the developer for a moment and take the user perspective. What's the first thing the user wants to do? Log in? Ok, that's a story. What's the next thing the user wants to do? See a list of widget prices? Ok, that's a story. What's the next thing the user wants to do? Change a price? Story.
If you can't sit down and think of a dozen narratives like this, then the fundamental problem that you don't really understand how people are supposed to use your software or what your software is supposed to do.
(Also a user doesn't actually want to log in, they want to do something and you are making them log in)
The task of login is more about security, access control, authorization and tracking, not just giving the user access to a web landing page.
That level of complexity makes it a project in itself.
But sure, you can have follow-on stories like "user is denied access protected resource" or "admin sees resources accessed by user". Enough individually-useless stories and you end up with a useful app. That's kind of the point of agile.
It wouldn't be a complete decomposition, of course, a person watching would see your thinking unfold. I think speculative decomposition would be very interesting, especially in retrospective if we could see the rabbit holes we went down.
Decomposing is easy since we naturally have to do that as we work. Heck, those 500 tasks are probably in your shell, browser and commit histories, mixed in with a bunch of other junk.
The problem is tools like Jira make this heavy-weight, doing it up front makes it nigh impossible, and having others review all this stuff and it becoming promises blows up the LOE significantly. And, I don't think I'd want to be under that much of a microscope as I work.
But if it were very lightweight, where I'm just posting my thoughts and can see them as a quick dependency diagram, and maybe attach notes, commits and URLs to them, and other people can see them, that'd be pretty helpful.
That's precisely the objective of "user stories" and other things. You write a high level version, put it in a backlog with priorities. When you get to it, you realize it's bigger than anticipated ("Oh, I can't just do X, I have to do A, B, and C."). So you turn X into three things, one of which you work on now, and the others in the backlog. Repeat until done.
I don't think Jira is necessarily too heavy-weight, it's the way everyone seems to use it. They want to assign the tasks now, not treat them as backlog items or things that can be modified in the future. Which pushes them back towards big-design-up-front and entirely defeats the objective, you're back to low throughput, high latency development.
https://blog.codinghorror.com/tending-your-software-garden/
The reality is that your ability to break things down depends on experience. I’ve built so many landing pages, for example, that I can tell you an exact breakdown of tasks. Been a part of so many SaaSes, I can tell you exactly what non-core features you’ll need, when, and how much work they are. I can also predict where you’ll hiccup.
But ask me to build something new that’s never been done before (at least by me) and the most I can do is shrug and guess. Maybe give you a rough sketch of an outline of subtasks.
But as an example of early Agile intent, here is how we worked in 2004: http://williampietri.com/writing/2004/teamroom/
You'll note that we never did detailed planning more than a week in advance. We could have, but it would have been a waste, because it would have been speculation on speculation. Instead we'd agree on something small to build, get it working, and then see what we thought.
Here is why it is important:
a) Risk management - if you break a feature down into sub-tasks at any granularity, you are creating an agreement (or at least a conversation) among your peers that this is how something will be implemented, and digging in beforehand to uncover areas which might impede shipping the feature sooner.
b) You're going to have to break down the feature at some point. Being able to think through this ahead of time can be challenging, but often times is the meat of the work you do. You have to get into the hang of it -- think top-down or bottom-up ways of approaching it.
But what if you don't have enough information or understanding yet to do that? Agile is not great for these tasks where you need to take some time learning or experimenting.
I would offer you a couple extra tools here:
- A "spike" ticket -- Agile is all about deliverables and commitments, but sometimes that doesn't work. So create a spike ticket, and define what it is you want to investigate. If you deliver something, great! If not, no worries. The important part is that in the future you've done work that enabled you to learn how to estimate or break down that task in the future.
- A time-boxed ticket -- similar to the spike, but you just make sure you don't spend more than an allotted time on a task.
Basically, you blaze a trail to the most disgusting, kluge-y, and otherwise slapdash implementation that validates your assumptions and satisfies your constraints while making note of everything that needs further work. That lets you bail early if some unknown issue will block success given the current criteria and constraints, and sets you up with the a list of the 80/20 work required to deliver the completed project.
Some things need to be designed, and don't work terribly well if they're evolved incrementally from user-visible features. Incremental additions can be, but there's also a risk of gradually degrading the architecture of a system through risk-minimized local changes by interchangable resources, I mean developers - which is what I generally see occur under Scrum.
Company and developers are busy and successful, so they hire more, which continues until they are no longer successful.
They remain busy, however, progress on the product is roughly constant. All the additional work capacity is spent on meetings and otherwise organizing the increased worker count.
I believe the above describes every situation in which I have been asked to break down Jira tasks into unnaturally small tasks.
Maybe it was just my particular job, but so much of my work was figuring out how to do xyz, so it was hard to give estimates for something whilst I was still figuring out the scope and complexity of along with how it even works and I was rarely, if ever doing the same/similar thing multiple times. Whenever project managers did push for me to break things down further they’d then immediately complain about too much technical detail.
As far as I'm concerned, "agile" refers to the principles on this page[1], and ONLY the principles on this page. Not Scrum, not Lean, not Kanban, not Xtreme.
The problem is, it's hard to sell a set of principles. To understand people and interactions takes months of working with people: it's much easier to sell them a process or a tool and leave. Customer collaboration requires building a relationship: it's much easier to negotiate a contract for your client and leave. Responding to change requires you to stick around and see what changes happen: it's much easier to sell them a plan and leave. And to arrive at working software you have to get a lot of things right: it's much easier to sell a bunch of documentation for software that doesn't exist, and then leave. Money ruins everything: the reason agile had to explicitly deprioritize the things on the right side of the list in the first place is that all the incentives in a software company push you toward the wrong priorities. The things on the right side of the list are all quick, easy wins that look good on a quarterly report.
There are companies who I've worked with who do agile (the principles) well. There's nothing wrong with looking at Scrum/Kanban/whatever as inspiration, as long as you realize that they're just processes: individuals and interactions matter more. There's nothing wrong with using Pivotal/Jira/Trello/whatever, as long as you realize they're just tools: individuals and interactions matter more.
[1] https://agilemanifesto.org/
He subsequently gave a talk comparing agile (and open source and consulting and startups and...) to fascist propaganda. https://www.youtube.com/watch?v=c5Xh2Go-jkM
I'm not promoting it or saying I agree with it (I actually disagree with quite a bit of it), but it is entertaining.
Big 'A' - "Agile" is waterfall re-branded - an excuse for corporate empire building and business as usual. It involves lot's of meetings and not trusting those who build software to do the right thing.
Small 'a' - "agile" is the implementation of the manifesto, which basically comes down to smart people figuring out how to work together towards a goal, often by taking small steps.
Until the terminology is sorted out the discussion can't help but be confused.
Hiring a consultant to teach 'Agile' is easy: the change of practices from something completely top-down to something that empowers the people at the bottom is the hard part and some orgs aren't capable of changing. Too many orgs are built around micromanagement, they don't know any other way.
We can talk all day about principle philosophical differences and what is/isn't 'agile,' but there has been a consensus from businesses in industry that 'agile' is 'Agile.'
Agile has become an excuse for terrible planning and offloading more and more work with ever increasing responsibilities to developers. At some point, enough professionals will reject following these terrible frameworks through different mechanisms. We're definitely not there yet, unfortunately.
Build software in small firms or consultancies who will treat it as a craft.
Maybe more a hope than a solution.
Once it got popular, "Agile" was co-opted by all the usual players (cf "extreme" before it, to a lesser degree).
The important idea is "agile" vs. waterfall, whether or not that includes anything directly recognizable as "Agile". Or call it something else, doesn't really matter.
Recent history shows you can certainly do things directly recognizable as part of the Agile methodology while demonstrably not being agile, so modulo the no true scotsman fallacy it's a much less fruitful distinction to draw.
I guess I'm saying "Agile" was/is one of several, and that's ok (good even). It's worth not getting bogged down on the "Agile" part and focusing on the more essential things.
You are correct. The Agile term was probably marketed to the management types as a some breakthrough methodology to _finally_ control the budget for software development. Then it took a life of it's own as many things do.
“Way too much of Agile has been not about technology, but about people and about managing things and about getting stuff done — not necessarily getting the right stuff done.”
This is the whole point of agile: progress on iterations, inspect and adapt at the end of each iteration.
Your team might build the "wrong stuff" for an iteration, realize it (inspect), then make a course correction (adapt). If you end up delivering the "wrong stuff" is because you didn't follow this very core principle of agile.
I find it hard to believe these so called "Agile Early Evangelists" can make such a statement. Their background in lean development should have made the familiar with empirical process controls, from where lean and agile come from.
My guess the author quoted them selectively to fit the "agile sucks" narrative of the article.
Edit: expanded last 2 paragraphs.
Keep in mind though problems also show up with waterfall or young small startups. Its just which flavor of friction/pain project issue you are willing to deal with
I have been privileged to work in a company that really thought about and worked to prioritize the four values on the left. I would follow those agile coaches to any place they wanted. I have been a staunch defender in real life and online, because I've seen it work very well. I also have been on teams with other processes and seen how much worse it can be.
I will take the values of agile and push for those, and I'll take the lessons learned such as quick feedback loops, continuous integration, relative estimation, automated tests (which came from people like Kent Beck and Robert Martin pushing them so hard alongside agile), and the good stuff.
However, after seeing how badly it can be weaponized against developers, I'm certainly ready to throw out the bathwater, and I think this is what they're talking about. I've seen far too much cargo cult agile and far too much command and control with a light layer of SCRUM.
We have agile "coaches" who have never learned to code! They take a set of color-by-number technical practices but don't understand how or why they matter! I had to correct someone's slide that got the four values wrong, and their consulting group apparently had been copying and pasting them incorrectly from presentation to presentation!
The values and principles of agile are great. The current implementation has some serious debt.
(And while we're at it, we could update it. Too many people misunderstood the documentation part. Continuous attention to technical excellence needs to be upgraded to a value. Delivering frequently today means days instead of weeks.)
Poor management is a separate issue from agile. Even so, I would rather stay on a poorly managed agile shop than go back to a waterfall shop.
As far as the "values on the left", I like to explain them as a 55/45 split (and adjustable depending on your reality): we still deal with processes and tools, we just take a second to think whether a process is actually needed when we can just talk to someone instead.
Example: on a small team, you might just ask "can someone please approve my changes?" instead of having a whole jira workflow with code reviews and approvals.
The only alternative to Agile isn't Waterfall. The right alternative is probably something entirely new.
Which sounds a bit like the "no true scotsman" fallacy.
If so many people have trouble implementing agile maybe it just doesn't work?
The problem with agile and other empirical processes of project control is that it goes against the OCD tendencies of scientifically minded people. They assume that, since programming is all math and logic, software development projects should be as well.
They add micromanaging processes in a futile effort to control what they perceive as chaos and end up trying to fit a square peg in a round hole.
Agile was a neat idea 20 years ago and it changed the engineering practices. However, unlike the methodology, engineering practices continued to change and modern software development practices automate away a lot of what agile processes tried to orchestrate.
If you are doing continuous delivery and lean development, you should be conceiving of and shipping software features in time units smaller than a sprint (i.e. continuously). That was very rare 2 decades ago and has become the norm for a lot of tech companies. It requires asynchronous processes and practices. It's enabled by having automated builds, automated tests, and automated deployments. These tools barely existed 2 decades ago and have had a far larger impact on software development then any form of agile.
Lots of large OSS projects and software companies have made the shift from doing feature based software delivery to time based software delivery. E.g. MS famously kept missing its own deadlines with windows vista, windows 7, etc. and shifted to having a more predictable release schedule. Ubuntu's LTS releases appear regular as clockwork in April of every second year. Linux ships a kernel every 2-3 months. Mozilla ships Firefox every month.
Time based releases are basically about releasing an unknown quantity of software at specific intervals and with a high level of quality. It involves having multiple asynchronous tracks of development and instead of planning which of them need to be ready they simply use quality gates to determine which of them are actually ready to ship. It's a shift from what to when and it emphasizes quality (i.e. good engineering) over schedule.
Most agile methodologies are still stuck trying to do feature based planning. It's the project mentality from the nineties where things get commissioned and have to be delivered on a particular schedule. Worse, these things are often under specified and then blow right by their planned deadlines. Just like in the nineties. I've seen a lot of agile projects shipping low quality software doing the wrong things right on time.
In places I've worked without the pressure of the quarterly earnings report I've seen that developers tended to use whatever methodology worked best. It could be Agile, pair programming, or just walking over to each other's desks and talking on IM.
Which is to say: To all the people jeering for Agile's demise, please provide a superior alternative. I came from a world dominated by Waterfall, and I never want to go back to that. A lot of companies get Agile wrong, and it isn't a panacea, but it is much closer to what developers are naturally inclined towards (e.g. rapid incremental improvements) than Waterfall ever was.
So I challenge anyone who wants to replace Agile, please lean into how developers work rather than trying to mold their work onto your rigid front loaded methodology. Trying to bring in ideas of other industries, like engineering or architecture, that build physical goods and only have one shot at is a folly.
It was utterly frustrating to be powerless to improve specification quality and find out you built exactly what was asked but you were asked for the wrong thing.
The silos were horrible. Devs were often as bad as BAs, dev’s just were just crapping on the testers instead. Spare a thought for the production ops person at the very end of the chain. Here comes 3 months of developer work in one weekend and no it doesnt work but we’re still going live because the entire tech org is invested in this release. Now you have smaller squads, you can postpone a release without it looking bad on the top person and thus affecting your career prospects.
In agile models you have the power to fix crap processes that don’t work. You can call out any BS on the retrospective and make it super visible when things are being swept under the rug.
I worked on the customer delivery side of a $750M software project that was a steaming pile of shit, with critical defects that were known for a year with zero effort to address. The integrators were paid to deliver a spec, and it took about a year to get change orders created to diverge from the spec.
The old school waterfall outsourcing models are truly awful, but you don’t really understand how bad unless you’ve lived it. Stupid agile religious practices are dumb for any startup type org, but are probably better than the alternatives in many big enterprise scenarios, where the goal is chunk up the work so marginally qualified people can do something.
The goal of Agile is to let smart people work incrementally towards a somewhat nebulous goal. The ceremony that's arisen around that tends to be put in place by people who feel the need to manage, but don't know how to help their developers achieve that goal.
The alternative, as far as I've seen, is to hire smart, curious people, let them work closely with the end user, and pay them a lot of money. In this situation, the engineers will typically self-organize effectively.
Now it’s 100 people across two countries, expected to collaborate closely with sibling orgs. Management philosophy has shifted to “we will ask a few handpicked experts to write down the best way to do a thing, and then everyone else is a machine for executing that procedure.”
If software engineering worked like that, we would have automated it.
Assembling a talented team that ultimately delivers nothing of value is practically a cliche in this industry.
Agile would work much better if every implementation stated upfront, "Does not work right out of the box".
> Now, continuous delivery is what’s expected, and the industry is ready for the next thing. But that next thing shouldn’t be another methodology, according to Mary.
> There is no methodology in my field of software engineering that can conceivably last more than five to eight years,” she said. “Everything that is 10 years old is obsolete. Everything that is 20 years old is archaic.”
> Furthermore, she said, methodology requires codification. Beginning with the Capability Maturity Model (CMM) in the ‘90s, software development methodology meant developers had to show they had standards and that they followed them, rather than demonstrating that their standards were constantly in flux depending on consumer needs. That’s the definition of quality standards lean manufacturing practitioners in Japan originally espoused, Mary said, and they’re not compatible with methodology. Instead, they’re all about learning.
> To that end, Mary is excited about all the ways artificial intelligence will allow software engineers to learn better and faster. Automated testing, continuous deployment and cross-functional collaboration are now table stakes, Mary and Tom agreed. Cutting edge companies will discover the next great approach through an engineering mindset and a willingness to learn.
___
Consider that Mary and Tom Poppendieck were responsible for many of the Lean inclusions of the agile movement, which (largely) came from watching plant manufacturing at Toyota. Similarly, much of the DevOps movement was tied to this as well. If you want to talk about what is next, it is likely taking a first principles look at what we're doing today, questioning best practices again, and saying, "if we were going to make a manifesto in 2020, what would it look like?"
The only "strawman" here seems to be taking my point about pre-Agile methodologies out of context, and using it to dismiss the entire idea that this article is unconstructive/has no actionable solutions.
The criticism in this comment thread was that their advice wasn't actionable, not that actionability was a prerequisite in order to criticize Agile at all. There was no false dilemma here because there was no choices provided (false or otherwise), merely a weakness in an argument raised.
> “I don't care if it’s Lean or Agile, there’s no silver bullet where if you just follow this formula that somebody else followed, you’re going to be great,” she said. “So today, my favorite word is ‘engineering.’ Just let engineers be engineers.”
Maybe we need to call it the Lego process. SCRUM is Duplo. Waterfall is worse Duplo. Start there if you need, but get to Lego instead. Maybe Duplo is enough for you.
Which is not entirely true since some of the best parts of Agile are XP practices that almost everybody does by default now...
Someone once said that the only successful Scrum projects are also doing half of XP (as in, things Scrum doesn't prescribe but are essential anyway). I am still looking for a counterexample.
Give individuals ownership over different parts and then let them self organize, that is all you need if you hire competent people. I'll never work at a place which doesn't work like that again.
QE is a direct transfer of wealth from dollar holders to investors and banks. No way around that.
Gambling with unlimited amounts of other people's money is going to be profitable.
https://www.rollingstone.com/politics/politics-features/2008...
I suspect that most knowledgeable developers rolled their eyes when Agile was "invented". It's a mish mash of a lot of things that were already obvious at the time and some weird kool-aid like pair programming. And just one more in a long line of consultant enrichment schemes.
One of the reasons Agile (with a big A) is so successful is that its peddlers have convinced everyone that there are only two options: Agile and waterfall. When you're up against a strawman, it's easy to win. But it's absolutely a false dichotomy.
There's a quote in the article that I like:
> Find me an actual tech company that talks much about Agile, and I will be astounded.
In my experience, people at tech companies (at least the FAANGs) rarely talk about Agile, although they do talk about things like continuous integration/testing/deployment. They do not obsess over methodologies for how to move post-it notes around a whiteboard (Scrum vs Kanban) or agonize over how to word a user story narrative, or other parts of Agile Theatre.
People at some non-tech companies, especially those supposedly going through a "digital transformation", seem to have fully bought in to the crap that Certified Agile consultants are selling though.
So a superior alternative to both Agile and waterfall is what's in use at a lot of the big tech companies. For example, the engineering culture at Google, which relies on design docs and a very good set of developer tools and infrastructure. It's not perfect, but it's far, far superior to Agile.
And before anyone makes the argument that those non-tech companies are not doing "real Agile", but Google is, let's be linguistic descriptivists and accept that Googlers don't call their methodology "Agile", whereas the Scrum consultants at big corporates do.
That said , plenty of tech companies have inconsistent practices that don’t necessarily lead to great success. To say Google’s methods are “far far superior” to Agile assumes they’re uniform, portable or relevant externally, or that they lead to above average success. That’s debatable given Google’s reputation for abandoned half-built products and an almost comical lack of customer focus.
The big drawback to good managers is that they stand up for their teams and don't kiss boot the way bad ones might.
You don't need to buy entirely into a single methodology. When people say Agile in software today, they really mean the Jira-flavored Scrum. Some now claim that they've abandoned Scrum and you hear more about Kanban. Sometimes they really are switching, other time they're really just doing Scrum with a Kanban board. But again, they're often falling into the same traps of forcing solutions, rituals, and processes they might not really need into their flow.
You want to be Agile? Start small. A simple checklist is a good way to start.
"Individuals and interactions over processes and tools"
I think part of the issue is that Agile is meant to be a Process imposed from top down to improve productivity by X%, a way for management and a small army of backlog/task managers to say they are doing Something and having impact (or literally the only thing they actively do). Not only can it easily get in the way of that, it almost always fosters an environment of shaming and lack of trust, because it is too tempting for estimates to be taken too seriously or productivity/performance measured "objectively" by invoking the task-tracking system (in which case you incentivize only taking on very easy, very well understood tasks). It doesn't always devolve to that, but I think at many places if anybody Important up in the management chain starts thinking that way, it will inevitably trickle down. Thus any sufficiently large organization will corrupt Agile.
Perhaps some companies need that level of accountability and visibility, even knowing and disliking the drawbacks, but not all of them. I am honestly not convinced that a rigid process is necessary at all. Yes it makes sense to have a system where you keep track of things that need to be done, but if there's a culture of trust, I don't think the system needs to be gamified or fetishized as much as it usually is.
The only long term solution for us was to have days dedicated to Agile ceremonies and days where no team member can be interrupted based on an agile ceremony need (there were few exceptions, like critical bug discovery, etc.). In weekly sprints, we had at least 3.5 days of uninterrupted time and in two-week sprints we had 7-7.5.
That changed a lot the pace of those sprints as people had a lot of time to do their job and everyone was happy.
Here's an archive link https://archive.is/wip/OFF0F