We can start by recognizing that scrum and agile cannot replace actual development talent. When companies throw additional scrum masters and project managers into a failing project, the project only fails faster.
^ this is agile. at least "people first" - then its process and tools. Tools are last, because there are always new tools and the way you do something should be somewhat consistent within a team/org even if the tools aren't
While scrum/agile does not = more developers, it can be leveraged to actually grow a developer's career and skills when done well.
As a technophile, I always disagree with that sentiment. Michelangelo cannot sculpt the Pieta with a sledgehammer and sticks. These days we need highly specialized tools to do our work. These highly specialized tools require specialized developers. The only alternative to this is low code/no code tools. Low code/no code is great for green field projects, as you can ensure a steady pipeline of such developers.
In a world of continously created saas pursuing bits of walletshare and department budgets endlessly I would say there is a small % of tools that are specialized and uniquely required. There's also a TON of tools that are preference based selection.
Michelangelo sculpted with one set of tools and painted with another. Klimt may have had some overlap with one and none with the other set. Neither was being doggedly pursued by salespeople or having decisions about tools made by their bosses.
One of the first things Scrum says is that you need to have a team with skills that ensures the job done. Same with Agile: literally says about "motivated individuals" and "self-organizing teams". So, arguably, you are not talking about Scrum nor Agile in those failing projects - you just share this naive view I saw in some managers: "we just put scrum master here and thus we have scrum implemented".
I was not referring to "motivated individuals". I was referring to developers with actual technical skills and actual experience with the tools used by the development team. What I often see is management being told we can hire the cheapest developers with zero experience and throw in scrum/agile... and we will achieve the same outcomes. The result is a mountain of tech debt and pagers going off at all hours at the day.
Well, scrum literally says that the team should have skills needed to get the job done, so you can still defend scrum, whereas agile says about putting people's interactions over processes and about pursuing technical excellence, so creating tech debt is still not agile.
So, I'm not sure if we can agree on that, but, arguably, I believe it is not scrum or agile that are failing, but management who doesn't want to be agile or/and follow scrum. This is my experience numerous talk I had on escalated level when my team was rising issues with management.
Why so? No one forbids you to pick a process and/or tools that come close to the perfect process and then change them as necessary. You don't have to reinvent the wheels.
You can use Scrum, Kanban, ... as starting points. Just don't implement them mindlessly like most companies do.
"Individuals and interactions over processes and tools"
"Responding to change over following a plan"
These are 2 of the 4 tenets in the manifesto for agile development [0]. They go on by saying:
"That is, while there is value in the items on
the right, we value the items on the left more."
Essentially, choose tools. Follow processes. Periodically and permanently reevaluate. If following the plan creates an impediment to interactions, or makes the people dissatisfied, respond to these changes. Rinse and repeat.
A previous commenter had it right imho. Its not that the approach isn't working, its that it has been co-opted by management and requires separation to actually succeed. Another note would be that agile itself says:
- non team members shouldn't be at stand ups etc.
- points shouldn't be used for evaluation of performance (good luck with that)
- if management joins anything other than the demo, they should not speak during the meeting (I've had some success here but also always been labeled a curiosity when I push management to be silent)
The framing is wrong. There is no singular alternative that works. Use it as a tool to keep things moving. If it ever feels that you are doing it for the sake of the process, look hard at changing it.
And don't get pulled into success. Most anything can and will work with eager teams at the start. Life has a way of changing without you noticing.
Right, and you should. But a retrospective is just another process. As soon as you find you are doing it for the sake of doing it, consider not for a while.
It is interesting to think about what processes a framework can allow to be changed, and what needs to stay.
That said.
I have never been anywhere where the process does not have room for improvement.
But often improvement seems impossible for structural reasons, for a team that is not empowered, where people are checked out, or not forthright due to whatever reasons. And like you say, people go through the motions for the sake of doing it, leading to wasted time and more frustration.
Eliminating the retrospective is the easy choice to avoid that.
Honestly. Software at Big Co is more like building a house.
If you’re in a high uncertainty environment with loads of market risk, by all means Agile away.
But if you’re designing and developing something fairly well understood… something more oriented towards a better understanding up front with bigger customer delivery increments is completely fine.
As a person who has not actually built a house but has done some renovation and designed and built some temporary structures, the amount of unexpected things coming up is surprisingly similar to software development!
I have a large amount of respect to people who have built a house. And even more for those who have built a bridge or a skyscraper.
> But if you’re designing and developing something fairly well understood…
There in lies the problem; building software is not like building a house.
I've seen many projects at Big Co fail to deliver on time, budget or quality because senior managers believed that everything is "fairly well understood" up front when just wasn't true.
That mistake leads to treating estimates as concrete delivery times, scaling the development team before it's ready and a refusal to fix things that hurt productivity because they're not on the unchangeable timeline.
The author is correct Scrum is usually sabotaged by outside influences trying to turn it into waterfall.
If IBM couldn't make waterfall work in the 1970s it's very unlikely that will work anyone's big project today either.
I don't understand why we're still talking about SCRUM.
People saying they're using SCRUM (by this I mean "literally saying 'SCRUM'") are a dying breed and if they're not then they're cargo culting and I'm not sure why this is more of a big news than management cargo culting waterfall or kanban.
I used to be one of these process experts selling methodologies to whoever was ready to pay, from SCRUM to lean startup.
After doing this full time for 2 years the only conclusion is that none of these processes hold long enough. Why?
1/ every product / team / output is different. some teams should be ticket driven, other exploratory driven, and each of these methodologies apply to one situation only
2/ high turnover in the industry. meaning that new people come and organically change the new flavor of the day, the dynamic of the teams and the throughput of the team
3/ all of the best teams I've worked in the industry (from seed stage to FAANG through late stage startups, and even F50 companies), just do WHATEVER that works for them and don't comply with the flavor of the day (except for the looks of it). They all use a shared core set of values (deserves it's own blog post), and you can find this core set of values in most agile methodologies, buried behind ritualistic behaviors
anyway. I could go on.
Just let it go, and if you're working in one of these last companies applying SCRUM unironically, you're probably not in a high performing team. Time to move
Honestly, this isn’t surprising. Typically external process change struggles to take root because the person isn’t there long enough to really understand the team, the problems, the culture, etc.
I don’t think this is the consultant’s fault, either. Usually the company doesn’t want to pay the money or time to do this.
Speaking as a consultant who's work does hold up, part of the job of a consultant is to make sure clients are aware of what's needed to succeed, and to turn down work that won't succeed. No matter how tempting the fee may be.
(I realize that many of my colleagues don't do this. Fie, I say. Fie!)
you should write that blog post about that core set of values. I think a lot of people have trouble recognizing success when it's not tied to a named concept, brand, or methodology.
My statement wasn't actually directed at you—I don't know you! Sorry it sounded like a criticism.
I was responding to @scott_w saying that process change not taking root isn't the consultant's fault, because companies don't want to pay for good process change. I disagree: I think it's the consultants job to be clear about what's needed to succeed, and not work for companies that don't want to pay for good work.
Regarding methodologies, I agree that successful teams internalize the principles behind their methods and leave the by-the-book methodologies behind. I think you're being overly cynical, though. Beginners need a concrete place to start. It's like saying cookbooks are a sham because expert chefs don't need them.
I didn’t specifically apportion blame. I just said I don’t blame the consultant in the case where it doesn’t succeed due to lack of investment. Yes, you can say consultants should enforce that but many don’t. I can’t say I blame them when they need to pay the bills and there’s a paying customer.
How would you have established the core set of values otherwise? Sometimes teams click, & sometimes an egomaniac inside a team can destroy the product—and end the team. I’ve seen it.
What agile/scrum was supposed to do was kill waterfall, and provide more transparency in how a project it product was actually proceeding with demonstrations of execution or lack thereof.
they do hold up for a while. but it's all ritualistic BS that is not needed. and to be clear, the teams WERE shipping. And were successful. But also quickly abandonned these methodologies in favor of some core values we implemented as a team. I don't think it's very graceful of you to assume my job was just a failure.
What I'm saying is that I was cargo culting too, because I personally believed in these methodologies.
But I realized after seeing many situations, that success is not linked to the usage of these methodologies
Back in 2005, I remember working on startups running on Scrum principles. It worked well at the time, we where able to ship, grow the team, and move forward with a nice few-features-per-week cadence, working remotely, on a small team; less than 10. Tt always worked fine, but very centralized and slow, as all-things-dev were at the time.
I worked with ActiveColab in 2007, Skype 2007, Yammer 2009, Trello 2011, Pivotal Tracker 2013, Trello 2016, Confluence 2022, Slack 2013, Google Meet, and sometimes I think, scrum became _less-relevant_ over the years as more advanced product management tools became the norm and the product manager role matured by leveraging them.
These days, it's not rare to see lead developers manage kanban-like boards very effectively, releasing on time, with grace, without the need of a scrum master to coordinate efforts.
I do like asynchronous scrum daily standups using http://geekbot.com on slack, when on-site or/and distributed and doing sprints. I seen this work well on startups going from pre-seed to series B.
Personally, I am fascinated with team dynamics and how they've changed over the years. We are definitely living the best of times as a developer and I still see sparkles of well-applied scrum every now and then that works nicely.
>These days, it's not rare to see lead developers manage kanban-like boards very effectively, releasing on time, with grace, without the need of a scrum master to coordinate efforts.
Sadly, it's also common to see such kanban teams endlessly winging it and slowly losing sight of what they were trying to accomplish, at the same time burning out their teams on an endless stream of tickets and testing without ever taking time to reflect and course correct on their goals.
True. But given a team of average ability, good process (ie, good habits and conventions) can definitely mean the difference between success and failure.
I'm sorry if I'm splitting hairs a bit here, but I'd argue 'good enough' process is all you need even with average teams. Keep them focused, limit their in flight work, unblock them, iterations with feedback, etc; I just feel some people spend inordinate amounts of time trying to optimize process when process hits diminishing returns pretty quickly.
>I'd argue 'good enough' process is all you need even with average teams.
That's sort of a tautology, right? If it's 'good enough', that implies it's a good process. In my experience, Scrum is a good enough process, with very little wasted overhead. It keeps the team focused, limits their in-flight work, unblocks them and offers regular iterations with feedback.
I'd agree that over-optimization is sometimes a problem, but when something as simple as scrum fails, it's usually down to the basics, like poor meeting practices, or micromanagement, or something outside of the development process entirely, like badly underbidding the project. No amount of process will save a project that was doomed from the start due to poor budgeting of time or money.
I think 'good enough' is just an expression to mean it doesn't need to be perfect or very elaborate. Unfortunately the term scrum these days is far from precise and does not guarantee it'll be lightweight, but the principles I definitely agree with. I've seen all sorts of things, including people over-focusing on the scrum process and nitpicking about all sorts of irrelevant things. I've worked with and without scrum masters in teams as an IC and manager. I think having a scrum master is often unjustified overhead, but having an experienced SM in new teams or teams with an inexperienced manager or lacking in soft-skills, it can help fill the gaps.
And yeah you said it well at the end. There are many other things that I believe are more relevant than the process, but you do need a process teams can follow.
Anything that gets between what me, the designer, and the PM want to do needs to go. There’s an entire industry of bullshit around framing and tracking that fundamental working unit which actually builds software. Project managers, Scrum masters, whatever. If you really need acountability to the bean counters, let them define their own KPIs and give me and the PM an interface to them. Anything else is a total waste of time and effort.
AMZN (amongst numerous other household name brands) ships world class products with 2-pizza teams where a dev, a designer and a product manager are having the conversations that need to be had to ship.
I completely agree with the author, it’s the environment. Scrum was intended to empower developers by allowing them control over when new work got scheduled.
In reality it has been used by management to export their responsibilities onto developers while not ceding control.
With scrum, planning and estimation are now done by developers. But management still reserves the right to reschedule mid-sprint for the all so urgent pop ups they need handled.
The only thing developers can do to make scrum work is be completely devoted to the process and not the business, but that is a career limiting move; scrum masters are put in positions to either be ineffective or be ablative armor for their teams.
I think the blog mentions this, and I'm glad its not just another complaint listing the failings of Scrum as it exists at an unnamed company. But I would put forth that no process can save developers from pathological management with poorly conceived products and even not "having a process" will help. As long as NPD* continues to be a great indicator of success in gathering VC funding and upper management success, developers will be exposed to horrific work environments.
These environments will be label 'Agile', 'Extreme' or whatever label can attract talent. No manner of 'Process X sucks' blog posts will change this. If the Hacker-News Scrum hating community comes up with a "better named process", or a successful manifesto advocating the 'end of processes', organizations led by sociopathic management will quickly adapt these slogans and then distort the environment to their likings.
If management is beating you with sticks because you missed getting your story points done by end of sprint, the problem isn't a process that measures stories by points, and has biweekly sprints where velocity can be measured; its having management that is willing to beat developers with sticks.
I've never been in an environment where Scrum was used to create real pressure.
I've been in environments where it creates fake pressure. Standups where you have to say something for fear that you'll be axed if you say nothing. Questions about whether a ticket is going to get done by Friday. I call these fake pressure because it becomes obvious that there is no teeth to them. People get away with doing nothing and making up a few sentences at standup. People say a ticket is going to be done, but it roles over through two or three sprints.
It's always been a sham for me. You can't point at the Agile Manifesto and say that we're focused on process over interactions. You can't point out that we need a Product Owner who is focused communicating the needs of real users. You can't point out that standups should be for the team's benefit and not status updates to project managers who are barely involved.
Still, I'm a firm believer that the Agile Manifesto makes some good points. I believe that there are elements of Scrum that are really beneficial if implement well. I believe all of it is subverted by management that wants a semblance of control/power. Most of the workers don't care whether it's subverted, they're just there to be paid while doing as little work as possible.
So, who's left to change it? A few righteous agitators? Who are they going to agitate? The managers who have control aren't going to give up the reins. The hoi polloi will ignore anything that takes extra effort.
Saying the scrum master needs to be accountable is true. Everyone should be accountable for something. However, there's simply no way for that to happen in many environments.
I've had scrum masters that were basically just required to gather status updates and report them back to the CTO. I've had contractor scrum masters that were hired because Scrum said they had to have that role but they ignored the part of Scrum that said there was a product owner role. I've had scrum masters that were just engineering managers who ignored their engineering manager roles to be the one who screen shared the Jira during standups, including one who was a Director of Engineering.
All of those scrum masters were installed by the people in power not to be held accountable for anything. Most of those scrum masters, like all the other workers, just want to get by with doing as little as possible while being paid.
I worked at a place that ran sham sprints all the time. If you couldn’t finish something, your ticket would get marked complete, and a new one opened to finish the work (“integrate [component]”) next sprint. This was so the points would be booked and the team stats wouldn’t look bad. I saw one ticket roll over for a good 2 or 3 months like this.
Sprints are bullshit, so people use this method to work around this. It's actually kanban implemented over Scrum.
It's just too common for a task to turn into an "epic" mid-sprint. I only see value in sprints as regular checkpoints but the expectation that all tasks are finished after 2 weeks is just contrary to reality of software development in large organisations.
What are you talking about is problem that you are committing to to much in sprint, thus, your velocity is inaccurate. If I can give you a piece of advice: not finished items should be marked as 0 work done and moved to next sprint, and thus it will affect your team's velocity in future sprints making you commit to less items, which should result in that eventually you will have real velocity of the team and you will know how much work you can complete in each iteration. Of course opposite is also true - with time you can get knowledge or improve codebase in such way that your velocity will go up.
I agree with you, but the "scrum masters" did not want to do that, as it would not "properly account" (their words) for any work done. There was a head bean counter that actually believed the numbers, and would count up all the points and track the completed vs planned. It would cause our stats to look bad if we achieved less work completed. Personally, I did not care about stats, I would rather improve our estimation process. I thought the whole thing was a gamified sham and left.
In my professional experience as developer I'm terribly disappointed with so called "scrum masters" dedicated for the job, the teams that I worked in that implemented scrum correctly were the teams where the team itself took the responsibility to do it right on themselves.
...and we come back to one of agile principles: "the best architectures, requirements, and designs emerge from self-organizing teams". I think the scrum itself says to not confuse the job title with the role and I believe that's what happening in many cases. But it all looks good on paper, we have scrum because we hired scrum master so management is happy whereas in reality it is not true - I saw it many times.
The scrum master on one of my teams literally did nothing. He phoned it in from his kid's basketball games, and tried pushing his responsibilities onto someone else. It was laughable how little work this guy did. He was one contribution was asking if we could break a 5 point ticket into smaller ones.
Because it was. "Waterfall", as described by agile evangelists, was a very rare practice. Good shops had a process that already incorporated most of the best features of agile.
XP is maybe the purest example of the basic problem. If you're good enough to actually practice XP, you don't need a methodology at all. You'll do the right thing in each situation. Methodologies are for those who aren't good enough. And apparently they always find a way to pervert it into its opposite, because people never get tired of exerting power and maintaining an illusion of control.
Yes, if everyone is good you don't need processes.
I do think Scrum took off because it is the opposite of XP, it doesn't tell developers any development practices (Pair programming etc.) so developers didn't feel bullied and said "Yeah, ok, lets do your Scrum". But it backfired.
Yeah I had to suffer through the full-blown training from a "Senior Agile Coach" like the author - taken extremely seriously by the management at the time, so I can see how a bunch of people who took this stuff literally - including the "coaches" - are a bit heartbroken now. My employer who ordered that expensive training collapsed several months after that.
But, trying to distill all the formalistic nonsense into usable points, they actually do have a few. My personal takeaways:
- try to ship faster (try daily for a webapp), but do what feels right for you
- have some task board and scrub it regularly (once every week or two, whatever)
- learn to use ballpark estimates (1 hour, 1 day, 1 week - or whatever else floats your boat. Fibonacci, none of this stuff will ever be realistic - just get the order of magnitude)
- have short meetings about blockers, but schedule stuff for later the moment someone starts banging on for too long
- yeah, and try to break stuff down into smaller, more manageable pieces, the spec, the PR's, pretty much everything
No idea if this is "SCRUM" or not, but this has worked for me so far...
That could also be kanban. Which is actually great - it means that what you described is a set of tools, which can be used for whatever purpose the team decides. That's agile, as opposed to forcing a process on the team.
Using estimates has not and never will be a useful idea. You are in one of two worlds, either your work is predictable and easy to estimate, in which case you could just count the number of tasks/stories and get a sufficient estimate anyway, or your work is very unpredictable in which case your estimates are a lie.
I've seen it work well when practised by small, mature and highly independent teams. Where things start to fall apart is when management gets interested or heaven forbid, the company decides to spin off its own bastardized version of the process (often by bolting on the worst aspects of waterfall).
Scrum is the bane of my existence as a developer. It sucks all creativity and enjoyment out of software development, and replaces it with anxiety, shame and mountains of tech debt. And don't get me started on the velocity metric that so many managers can't seem to live without (see Goodheart's Law). For whatever reason, I'm 10x more productive when I have the autonomy to bounce ideas off my team and proceed in the best way I see fit.
Just like there are companies who say they are Agile, but they are not, there are teams who think they have Scrum, but it is not Scrum. If you say that Scrum encourages tech debt and you had no autonomy then probably it wasn't Scrum.
The other option is that you were using Scrum where it doesn't fit.
By Pareto principle: 80% of times when I see "The tool X sucks" it is because people are using wrong tool or they are using tool wrong.
We can debate the purity of scrum, but in practice, every company I've worked for has modified it to suit management desires, and those modifications have almost always been to the detriment of developers. For example, the "complexity" of a story is inevitably interpreted as "time". And the end of every sprint is often turned into a deadline that leads to tech debt. Creative development is not always a linear process, which scrum somewhat enforces. I think it could work if every story was treated as a spike, and every spike could be pre-empted by new spikes as you discover new things. If it was able to truly flow the same way software devs actually think and work, then it could work. The result of a spike could be a finished work, or perhaps many other spikes. This would model creative development better. But companies don't go for this because they cannot easily estimate time and schedules.
I know what you are talking about, but those are the things I would never agree upon. This is against scrum, against agile. Where I can agree that scrum is not always the way, I truly believe agile is how software development should be done, so I can't even count how many of conversations about it I had with all those managers. Outcomes were two: they agreed to change and project was successful in most of the cases or I quit and project failed in most of the cases, so my experience is only strengthening my views over time.
Once I worked in a team that used scrum almost correctly, and it was a great experience.
Then there was a change in management, and the new manager decided to "introduce scrum" to the whole company. We were told to stop doing what we did previously, and instead to do what the new manager called scrum. Unsurprisingly, it was the same parody of scrum that most companies use. The productivity plummeted, and the developers in our team gradually left the company.
I suspect that scrum works well when it is introduced by the developers, and works horribly when it is introduced by the managers.
The only thing that worked for me when fighting managers as such was to expose where they are wrong about scrum and/or agile. Foursquare, boldly, without emotions. Speak up guys, you are paid to be experts, share your expertise with the org.
I was told this once by a scrum trainer when I told him it sucked. I then challenged him to figure out which part we were doing wrong. He couldn't coz we pretty much did it to the letter - following the directives from another scrum trainer.
I call it the no true scrumsman fallacy.
Doing it "properly" means doing a series of mini waterfalls. Thats just how it was designed.
I'd say it functions as a poor but semi functioning compromise between an intransigent waterfall management and an agile team.
People like you who complain about scrum often complain about things that are not part of scrum. Velocity is never mentioned in the scrum guide for example. What you describe is nothing like scrum.
In my daily work life, academic discussions of "what is scrum (or whatever)" isn't of practical value. What matters is what your employer is requiring you to do. I have yet to work at a company that uses agile methodologies in a manner that doesn't effectively torture devs.
I'm not saying such companies don't exist, only that I've never personally worked for them.
I use "agile" as a filtering term these days. If the job description mentions being an agile shop, I know that job isn't for me. If they use agile principles but don't advertise it, I want to see what the process actually is before deciding to accept the position.
I think you are missing one thing: why do you allow managers to say how to develop software when you are hired to be an expert on that matter?
I know, I know, it is convenient to not to take responsibility and just complain, but that would be so frustrating for me that I couldn't take it. And, yes, in my experience I saw many times where people who are not experts say how something should be done, for example they say "refactor is not needed" or "tests are too expensive". In that case I ask them one simple question: "how many contributions to the codebase you made last month?" The answer is usually silence. If they still oppose I just quit the job and watch the project fail from my colleagues' reports.
Simple as that.
> why do you allow managers to say how to develop software when you are hired to be an expert on that matter?
Because at most companies, you have a simple choice: follow the prescribed procedures, or work somewhere else. I have absolutely quit jobs that were too egregious, but that doesn't change the situation at that company.
Maybe losing valuable people is not affecting company straight away, but it surely affect the project. And with few project failing miserably (as they will if you do corporate waterfall) something might change. If not, then the whole company might fail. Remember, companies compete with each other, it just takes more time for some to fail.
It is important if you want to fix it. If you think scrum is the problem, then changing scrum to something else might appear to make sense. However if the problem is not in your framework but with tangential organizational goals then it won't matter which framework you use. I'd say that's pretty relevant to discuss.
> I'm 10x more productive when I have the autonomy to bounce ideas off my team and proceed in the best way I see fit.
Why would scrum be a hindrance to that? I don’t know if any principle, artifact or event that is there to reduce team autonomy or to prevent the team from bouncing ideas of each other. And it seems scrum is intended to let the team procede whichever way they deem best.
I don't think it's scrum that's the problem. I used to be a big proponent of it when it came about. But in those early days of agile there was excitement that we were gaining this control over our own development process. That was never going to really happen. The company I worked for was doing development work for government and started pushing agile as a way to do it, eventually transforming itself into an agile consultancy. It pains me to look at the profile of people I worked with back then, great people, who now just peddle what I consider bullshit. Anyway for myself I no longer do any of that, just dev work, and I've been on good and bad teams. The process doesn't actually make a difference.
Next time someone tells me at a job interview that they use "scrum", I will ask them two questions:
1) do you have retrospectives, and what typically happens there?
2) have you ever changed anything, as a result of the feedback given by developers at the retrospective? please provide a specific example.
If they say "no" to either of these questions, it is the bad kind of scrum.
If they say "yes", and give any non-bullshit answer to the second one (for example "we had 2-week sprints but then developers changed it to 3-weeks" or vice versa; the exact details are not important), it has a chance to be the good kind of scrum.
If someone says "we don't do retrospectives, because they are a waste of time", they are kinda right, but for the wrong reasons. The retrospective is where developers provide feedback about the process, and can choose to change it. If you have a strictly top-down managed company with zero developer autonomy, in such case having retrospectives is indeed a waste of time, because no matter what the developers might say, nothing ever changes as a consequence. But that doesn't mean that the retrospectives suck; it just means that the company sucks.
Ashamed to admit I was once a Certified ScrumMaster. I’ve grown to realize that it’s too difficult to implement. The gains, if any, aren’t worth the effort and overhead. As the agile manifesto says: “Individuals and interactions over processes and tools”.
Basically high integrity computing processes, very waterfall like, and with quality process verification that most companies will only ever adopt when liability becomes a common legal requirement in software delivery.
It'd be nice to summon someone from that side of things, but I'd venture to say it's just built really differently. Like, how a bursting bathroom pipe in an apartment is different from the same purpose plumbing on SpaceX. Humans actually keep building both, somehow.
I would like to propose a different perspective in Scrum and Agile: they're really actually reasonably good.
Or rather, they do go wrong, but they go wrong for reasons outside of the scope that a development process considers. Instead, I argue the issue is related more with the way that managers and employees interact, and the expectations they both bring to the table.
More explicitly, I am arguing that if the development process changed at a company with a manager running agile processes, the manager won't necessarily suck less if they were already bad. Likewise to the employee.
My suggested action is to treat this as a question around social environment, and tweak accordingly. This would be different for each company I would expect.
I had great success using scrum in central government. It freed us from the whole "get a document approved by lots of external parties before you can do anything" mentality. .
It helped that because it was such a different process, they largely left us alone. They loved the regular retrospectives. They were awed to see feedback from the previous retrospective made real by the next one.
By the end we delivered more for less and it actually did what the stakeholders cared about (not what people thought they wanted when we started and didn't understand it).
Exactly. I saw employees who said they have Agile implemented, but for example they were too weak to speak up, take responsibility and pursuit technical excellence. Same with managers: they say they are Agile, but trusting your employees that they will get the job done was a joke to them.
Having gone through all the management fads for the last 30 years, Scrum in practice might not be agile, but it is definitely better than any other alternative in big corporations.
I for one am glad that there are Scrum masters and POs to deal with all the politics and project nonsense that I don't have any interest in being part of.
Have been using scrum/agile for over a decade now. To most of the companies if not all I worked for, agile/scrum was nothing but just a fancy buzz word to the management team. They just did what they were already doing under the name of agile.
Put all that aside, scrum/agile is actually pretty good on tracking small and simple tasks. So if the whole project can be broken down to nice and small tasks and the estimates were accurate, agile/scrum will work perfectly. Unfortunately all the project I've ever involved were complicated, some were even super large. We humans are pretty bad at estimating and no surprise most estimations were way off. Also, there were huge tasks that could not be broken down. That's exactly where the short coming of scrum/agile. Also, in most projects there were for sure huge, hard and important tasks that needed to be solved as soon as possible. But in scrum/agile, the team tends to pick the small, simple and easy tasks first as it would make the metrics nicer. So when they finally have to pick those important hard and huge tasks, it would be most likely way too late as there were already too much done in the wrong way.
I’m sick and tired of the way software is nuit and how developers are almost always at the bottom of the food chain. We have to accept pressure from the whole org and deliver on things promises to others regardless of reality.
I would like to break the cycle and end my fellow developers’ misery by helping willing teams and orgs work better.
How could I do this? How is it called? I certainly don’t want to be another Agile(tm) peddler. But I also can’t go back to being a code monkey.
And before people here reply that I have to find better orgs to work at: while I believe they exist, they’re so rare that this isn’t possible for 99% of people. In my over 10 years in software, I have never interviewed with or worked at one of these elusive better orgs. I’m also not in the US which might be a factor. Anyway, I’d rather have a different solution to this problem than “work somewhere exceptionally hard to find”
I'm surprised, I've never had problems with org pressure, it was easy for me: either we do agile and you trust our judgement or I quit, because I know that this project will fail and I'm not interested in contributing to projects that are doomed to fail. Do you think I was unemployed for long in the second case?
It exists to give management and PMs control of development again. Without it they can't control what developers do. It might be OK for web development but for most other areas it's a disaster.
When it appeared it was a breath of fresh air: minimal, explainable in 5 minutes, just accumulation of common sense.
You don't even need to declare "We're working in SCRUM" - the important feature is possibility to do "SCRUM in stealth".
SCRUM went wrong when it became "process", when SCRUM masters/evangelists appeared.
They started to teach "right SCRUM practices" and SCRUM became a bureaucratic ritual not distinguishable from RUP.
One big point which hasn't been resolved by scrum in many companies. Manual testing - until it's tested by the QA tested you shouldn't mark it done. However, it is very seldom to finish testing and implementing in one take in exactly 2 weeks. Also business analysts, together with QA testers there is virtually no control in most companies what they do and how they do. All the blame is shifted always on developers because they are at the bottom of SDLC chain but business analysts and QA testers take never responsibility for all inaccuracies and omissions of their work.
Rally a team around a single goal, get out of the way and let them move fast. If you don't trust your people, you have bigger problems that no methodologies will fix.
My firm is making a big deal of "We're going agile/scrum" as of the start of the year.
I'm not sure I quite understand how it makes sense.
From my personal perspective, the sprint model is an okay idea if you're working greenfield, or at least in full control of your deliverables.
I'm wondering how it fits into any sort of dev process where third parties are involved. For the same conceptual change, some will be "send us an email with the ID from the test API call you made, and we'll tell you if it looks right" taking a few hours. Others will demand a huge stack of formal test cases, weekly meetings, and a cycle time of two weeks to even get back what they didn't like about your test requests. How do you possibly slice that into two week intervals?
If you can't do the sprint planning - if you can't decide on the list of things that need to get done during the next two or three weeks - then you obviously can't have scrum.
So it probably means that you will have daily meetings and jira tickets...
Scrum wasn't implemented in Scrum shops for the developers. It was implemented for the executives, so that they have constant visibility into, and control over, all phases of the SDLC.
116 comments
[ 0.19 ms ] story [ 100 ms ] threadPeople first, tools second, and then process.
While scrum/agile does not = more developers, it can be leveraged to actually grow a developer's career and skills when done well.
In a world of continously created saas pursuing bits of walletshare and department budgets endlessly I would say there is a small % of tools that are specialized and uniquely required. There's also a TON of tools that are preference based selection.
Michelangelo sculpted with one set of tools and painted with another. Klimt may have had some overlap with one and none with the other set. Neither was being doggedly pursued by salespeople or having decisions about tools made by their bosses.
IMO: Its not an apt metaphor in the slightest.
So some assortment of reasonable starting points, with their pros and cons described, would be valuable.
You can use Scrum, Kanban, ... as starting points. Just don't implement them mindlessly like most companies do.
These are 2 of the 4 tenets in the manifesto for agile development [0]. They go on by saying:
"That is, while there is value in the items on the right, we value the items on the left more."
Essentially, choose tools. Follow processes. Periodically and permanently reevaluate. If following the plan creates an impediment to interactions, or makes the people dissatisfied, respond to these changes. Rinse and repeat.
[0] https://agilemanifesto.org/
A previous commenter had it right imho. Its not that the approach isn't working, its that it has been co-opted by management and requires separation to actually succeed. Another note would be that agile itself says:
- non team members shouldn't be at stand ups etc.
- points shouldn't be used for evaluation of performance (good luck with that)
- if management joins anything other than the demo, they should not speak during the meeting (I've had some success here but also always been labeled a curiosity when I push management to be silent)
And don't get pulled into success. Most anything can and will work with eager teams at the start. Life has a way of changing without you noticing.
That said.
I have never been anywhere where the process does not have room for improvement.
But often improvement seems impossible for structural reasons, for a team that is not empowered, where people are checked out, or not forthright due to whatever reasons. And like you say, people go through the motions for the sake of doing it, leading to wasted time and more frustration.
Eliminating the retrospective is the easy choice to avoid that.
If you’re in a high uncertainty environment with loads of market risk, by all means Agile away.
But if you’re designing and developing something fairly well understood… something more oriented towards a better understanding up front with bigger customer delivery increments is completely fine.
There in lies the problem; building software is not like building a house.
I've seen many projects at Big Co fail to deliver on time, budget or quality because senior managers believed that everything is "fairly well understood" up front when just wasn't true.
That mistake leads to treating estimates as concrete delivery times, scaling the development team before it's ready and a refusal to fix things that hurt productivity because they're not on the unchangeable timeline.
The author is correct Scrum is usually sabotaged by outside influences trying to turn it into waterfall.
If IBM couldn't make waterfall work in the 1970s it's very unlikely that will work anyone's big project today either.
People saying they're using SCRUM (by this I mean "literally saying 'SCRUM'") are a dying breed and if they're not then they're cargo culting and I'm not sure why this is more of a big news than management cargo culting waterfall or kanban.
I used to be one of these process experts selling methodologies to whoever was ready to pay, from SCRUM to lean startup.
After doing this full time for 2 years the only conclusion is that none of these processes hold long enough. Why?
1/ every product / team / output is different. some teams should be ticket driven, other exploratory driven, and each of these methodologies apply to one situation only 2/ high turnover in the industry. meaning that new people come and organically change the new flavor of the day, the dynamic of the teams and the throughput of the team 3/ all of the best teams I've worked in the industry (from seed stage to FAANG through late stage startups, and even F50 companies), just do WHATEVER that works for them and don't comply with the flavor of the day (except for the looks of it). They all use a shared core set of values (deserves it's own blog post), and you can find this core set of values in most agile methodologies, buried behind ritualistic behaviors
anyway. I could go on.
Just let it go, and if you're working in one of these last companies applying SCRUM unironically, you're probably not in a high performing team. Time to move
I don’t think this is the consultant’s fault, either. Usually the company doesn’t want to pay the money or time to do this.
(I realize that many of my colleagues don't do this. Fie, I say. Fie!)
I've actually done a GREAT job. These teams shipped.
But I realized that they didn't ship because of the actual methodologies but the core set of values we implemented as a team.
ergo, these methodologies are a sham.
I was responding to @scott_w saying that process change not taking root isn't the consultant's fault, because companies don't want to pay for good process change. I disagree: I think it's the consultants job to be clear about what's needed to succeed, and not work for companies that don't want to pay for good work.
Regarding methodologies, I agree that successful teams internalize the principles behind their methods and leave the by-the-book methodologies behind. I think you're being overly cynical, though. Beginners need a concrete place to start. It's like saying cookbooks are a sham because expert chefs don't need them.
The turnover issue we're seeing in the industry definitely doesn't help with seniors leading the way. which ends up being a problem top to bottom
What agile/scrum was supposed to do was kill waterfall, and provide more transparency in how a project it product was actually proceeding with demonstrations of execution or lack thereof.
What I'm saying is that I was cargo culting too, because I personally believed in these methodologies.
But I realized after seeing many situations, that success is not linked to the usage of these methodologies
I worked with ActiveColab in 2007, Skype 2007, Yammer 2009, Trello 2011, Pivotal Tracker 2013, Trello 2016, Confluence 2022, Slack 2013, Google Meet, and sometimes I think, scrum became _less-relevant_ over the years as more advanced product management tools became the norm and the product manager role matured by leveraging them.
These days, it's not rare to see lead developers manage kanban-like boards very effectively, releasing on time, with grace, without the need of a scrum master to coordinate efforts.
I do like asynchronous scrum daily standups using http://geekbot.com on slack, when on-site or/and distributed and doing sprints. I seen this work well on startups going from pre-seed to series B.
Personally, I am fascinated with team dynamics and how they've changed over the years. We are definitely living the best of times as a developer and I still see sparkles of well-applied scrum every now and then that works nicely.
Sadly, it's also common to see such kanban teams endlessly winging it and slowly losing sight of what they were trying to accomplish, at the same time burning out their teams on an endless stream of tickets and testing without ever taking time to reflect and course correct on their goals.
That's sort of a tautology, right? If it's 'good enough', that implies it's a good process. In my experience, Scrum is a good enough process, with very little wasted overhead. It keeps the team focused, limits their in-flight work, unblocks them and offers regular iterations with feedback.
I'd agree that over-optimization is sometimes a problem, but when something as simple as scrum fails, it's usually down to the basics, like poor meeting practices, or micromanagement, or something outside of the development process entirely, like badly underbidding the project. No amount of process will save a project that was doomed from the start due to poor budgeting of time or money.
And yeah you said it well at the end. There are many other things that I believe are more relevant than the process, but you do need a process teams can follow.
AMZN (amongst numerous other household name brands) ships world class products with 2-pizza teams where a dev, a designer and a product manager are having the conversations that need to be had to ship.
Do better.
In reality it has been used by management to export their responsibilities onto developers while not ceding control.
With scrum, planning and estimation are now done by developers. But management still reserves the right to reschedule mid-sprint for the all so urgent pop ups they need handled.
The only thing developers can do to make scrum work is be completely devoted to the process and not the business, but that is a career limiting move; scrum masters are put in positions to either be ineffective or be ablative armor for their teams.
These environments will be label 'Agile', 'Extreme' or whatever label can attract talent. No manner of 'Process X sucks' blog posts will change this. If the Hacker-News Scrum hating community comes up with a "better named process", or a successful manifesto advocating the 'end of processes', organizations led by sociopathic management will quickly adapt these slogans and then distort the environment to their likings.
If management is beating you with sticks because you missed getting your story points done by end of sprint, the problem isn't a process that measures stories by points, and has biweekly sprints where velocity can be measured; its having management that is willing to beat developers with sticks.
* Narcistic Personality Disorder
If it really worked, managers would also be subject to it.
I've been in environments where it creates fake pressure. Standups where you have to say something for fear that you'll be axed if you say nothing. Questions about whether a ticket is going to get done by Friday. I call these fake pressure because it becomes obvious that there is no teeth to them. People get away with doing nothing and making up a few sentences at standup. People say a ticket is going to be done, but it roles over through two or three sprints.
It's always been a sham for me. You can't point at the Agile Manifesto and say that we're focused on process over interactions. You can't point out that we need a Product Owner who is focused communicating the needs of real users. You can't point out that standups should be for the team's benefit and not status updates to project managers who are barely involved.
Still, I'm a firm believer that the Agile Manifesto makes some good points. I believe that there are elements of Scrum that are really beneficial if implement well. I believe all of it is subverted by management that wants a semblance of control/power. Most of the workers don't care whether it's subverted, they're just there to be paid while doing as little work as possible.
So, who's left to change it? A few righteous agitators? Who are they going to agitate? The managers who have control aren't going to give up the reins. The hoi polloi will ignore anything that takes extra effort.
Saying the scrum master needs to be accountable is true. Everyone should be accountable for something. However, there's simply no way for that to happen in many environments.
I've had scrum masters that were basically just required to gather status updates and report them back to the CTO. I've had contractor scrum masters that were hired because Scrum said they had to have that role but they ignored the part of Scrum that said there was a product owner role. I've had scrum masters that were just engineering managers who ignored their engineering manager roles to be the one who screen shared the Jira during standups, including one who was a Director of Engineering.
All of those scrum masters were installed by the people in power not to be held accountable for anything. Most of those scrum masters, like all the other workers, just want to get by with doing as little as possible while being paid.
It's just too common for a task to turn into an "epic" mid-sprint. I only see value in sprints as regular checkpoints but the expectation that all tasks are finished after 2 weeks is just contrary to reality of software development in large organisations.
...and we come back to one of agile principles: "the best architectures, requirements, and designs emerge from self-organizing teams". I think the scrum itself says to not confuse the job title with the role and I believe that's what happening in many cases. But it all looks good on paper, we have scrum because we hired scrum master so management is happy whereas in reality it is not true - I saw it many times.
I do think Scrum took off because it is the opposite of XP, it doesn't tell developers any development practices (Pair programming etc.) so developers didn't feel bullied and said "Yeah, ok, lets do your Scrum". But it backfired.
But, trying to distill all the formalistic nonsense into usable points, they actually do have a few. My personal takeaways:
- try to ship faster (try daily for a webapp), but do what feels right for you
- have some task board and scrub it regularly (once every week or two, whatever)
- learn to use ballpark estimates (1 hour, 1 day, 1 week - or whatever else floats your boat. Fibonacci, none of this stuff will ever be realistic - just get the order of magnitude)
- have short meetings about blockers, but schedule stuff for later the moment someone starts banging on for too long
- yeah, and try to break stuff down into smaller, more manageable pieces, the spec, the PR's, pretty much everything
No idea if this is "SCRUM" or not, but this has worked for me so far...
By Pareto principle: 80% of times when I see "The tool X sucks" it is because people are using wrong tool or they are using tool wrong.
I agree that is the way it is often, but that is to the argument to the poster before: not SCRUM anymore.
Then there was a change in management, and the new manager decided to "introduce scrum" to the whole company. We were told to stop doing what we did previously, and instead to do what the new manager called scrum. Unsurprisingly, it was the same parody of scrum that most companies use. The productivity plummeted, and the developers in our team gradually left the company.
I suspect that scrum works well when it is introduced by the developers, and works horribly when it is introduced by the managers.
I call it the no true scrumsman fallacy.
Doing it "properly" means doing a series of mini waterfalls. Thats just how it was designed.
I'd say it functions as a poor but semi functioning compromise between an intransigent waterfall management and an agile team.
I'm not saying such companies don't exist, only that I've never personally worked for them.
I use "agile" as a filtering term these days. If the job description mentions being an agile shop, I know that job isn't for me. If they use agile principles but don't advertise it, I want to see what the process actually is before deciding to accept the position.
Because at most companies, you have a simple choice: follow the prescribed procedures, or work somewhere else. I have absolutely quit jobs that were too egregious, but that doesn't change the situation at that company.
Why would scrum be a hindrance to that? I don’t know if any principle, artifact or event that is there to reduce team autonomy or to prevent the team from bouncing ideas of each other. And it seems scrum is intended to let the team procede whichever way they deem best.
1) do you have retrospectives, and what typically happens there?
2) have you ever changed anything, as a result of the feedback given by developers at the retrospective? please provide a specific example.
If they say "no" to either of these questions, it is the bad kind of scrum.
If they say "yes", and give any non-bullshit answer to the second one (for example "we had 2-week sprints but then developers changed it to 3-weeks" or vice versa; the exact details are not important), it has a chance to be the good kind of scrum.
If someone says "we don't do retrospectives, because they are a waste of time", they are kinda right, but for the wrong reasons. The retrospective is where developers provide feedback about the process, and can choose to change it. If you have a strictly top-down managed company with zero developer autonomy, in such case having retrospectives is indeed a waste of time, because no matter what the developers might say, nothing ever changes as a consequence. But that doesn't mean that the retrospectives suck; it just means that the company sucks.
What do people writing realtime flight control systems do?
What do quants writing investment strategy code do?
Do they all do the same thing? Would they call their process a ritualised behaviour?
Or rather, they do go wrong, but they go wrong for reasons outside of the scope that a development process considers. Instead, I argue the issue is related more with the way that managers and employees interact, and the expectations they both bring to the table.
More explicitly, I am arguing that if the development process changed at a company with a manager running agile processes, the manager won't necessarily suck less if they were already bad. Likewise to the employee.
My suggested action is to treat this as a question around social environment, and tweak accordingly. This would be different for each company I would expect.
It helped that because it was such a different process, they largely left us alone. They loved the regular retrospectives. They were awed to see feedback from the previous retrospective made real by the next one.
By the end we delivered more for less and it actually did what the stakeholders cared about (not what people thought they wanted when we started and didn't understand it).
I for one am glad that there are Scrum masters and POs to deal with all the politics and project nonsense that I don't have any interest in being part of.
Put all that aside, scrum/agile is actually pretty good on tracking small and simple tasks. So if the whole project can be broken down to nice and small tasks and the estimates were accurate, agile/scrum will work perfectly. Unfortunately all the project I've ever involved were complicated, some were even super large. We humans are pretty bad at estimating and no surprise most estimations were way off. Also, there were huge tasks that could not be broken down. That's exactly where the short coming of scrum/agile. Also, in most projects there were for sure huge, hard and important tasks that needed to be solved as soon as possible. But in scrum/agile, the team tends to pick the small, simple and easy tasks first as it would make the metrics nicer. So when they finally have to pick those important hard and huge tasks, it would be most likely way too late as there were already too much done in the wrong way.
I've come across tasks that were difficult to break down and required a bit of creativity.
Never come across one that was impossible though.
I would like to break the cycle and end my fellow developers’ misery by helping willing teams and orgs work better.
How could I do this? How is it called? I certainly don’t want to be another Agile(tm) peddler. But I also can’t go back to being a code monkey.
And before people here reply that I have to find better orgs to work at: while I believe they exist, they’re so rare that this isn’t possible for 99% of people. In my over 10 years in software, I have never interviewed with or worked at one of these elusive better orgs. I’m also not in the US which might be a factor. Anyway, I’d rather have a different solution to this problem than “work somewhere exceptionally hard to find”
When it appeared it was a breath of fresh air: minimal, explainable in 5 minutes, just accumulation of common sense.
You don't even need to declare "We're working in SCRUM" - the important feature is possibility to do "SCRUM in stealth".
SCRUM went wrong when it became "process", when SCRUM masters/evangelists appeared. They started to teach "right SCRUM practices" and SCRUM became a bureaucratic ritual not distinguishable from RUP.
I'm not sure I quite understand how it makes sense.
From my personal perspective, the sprint model is an okay idea if you're working greenfield, or at least in full control of your deliverables.
I'm wondering how it fits into any sort of dev process where third parties are involved. For the same conceptual change, some will be "send us an email with the ID from the test API call you made, and we'll tell you if it looks right" taking a few hours. Others will demand a huge stack of formal test cases, weekly meetings, and a cycle time of two weeks to even get back what they didn't like about your test requests. How do you possibly slice that into two week intervals?
So it probably means that you will have daily meetings and jira tickets...