Meh... All of these items are more or less misinformed so I'm just going to bring up one.
> Because all product decision authority rests with the “Product Owner”, Scrum disallows engineers from making any product decisions and reduces them to grovelling to product management for any level of inclusion in product direction.
The product owner is part of the scrum team. She or he is responsible for the backlog but if she has trust in the engineers she can let the engineers add their own things to it.
Right, I felt the same way. I think the accurate title for this post is “Why working for, with, and being an asshole is the wrong way to build software.”
Oh I'm with that.
"Why aren't I getting the attention, focus and control I want?"
Especially credit. You get credit in the git logs etc. If you want visibility outside the team go start your own.
I have worked in multiple companies that worked very well with this, in several different teams.
Or to give a much more serious response : this article seems to be very unprofessional, about trying to defend unprofessional behaviour as "normal".
The scrum guide talks about roughly 1hour/week for the scrum team to show stakeholders (such as upper managers and customers) what has been done. Any engineer can at any time reach out to people in the company to demo what has been created, if there's interest.
So I'm not sure what point you're trying to make with starting your own company if you want visibility outside the team.
If OP worked in a “good” place where product managers & engineers trust each other to do those tasks, what value does Scrum bring?
The process is there to account for the places where the people fail. If a requirement of a process is unfailing people then it’s not a valuable process.
The process is there to account for the places where the people fail
That's just wrong. It's there for lots of reasons – to help avoid failures and surprises, to deliver sustainable pace, to help make estimations, to ensure a common understanding of problems and challenges… the list goes on.
Scrum doesn't require infallible people – it's just a tool to help avoid failures, which happen regardless of the skill of your team.
I’ve not found that to be the case. In fact on at least 2 occasions in my career adding scrum practices to a product team essentially destroyed them.
I’ve seen instances of all of the items the article claims being problems in practice. That was across a wide variety of teams & skill levels.
I’ve also seen no evidence in the literature or research that backs up your assertions.
I’ll concede some teams have benefited from scrum practices, but in the face of my anecdotal experience & the lack of data forgive me for being skeptical of a practice that seems largely around to keep process consultants in business.
I’m certainly not receptive to a defense that someone who criticizes it just works at the wrong place or hasn’t seen “real” scrum.
In fact on at least 2 occasions in my career adding scrum practices to a product team essentially destroyed them.
Why was Scrum added to these teams if they were already working? It's a tool to deal with problems. It's not something to use if another process is already working.
I’ve seen instances of all of the items the article claims being problems in practice.
I have found essentially the exact opposite of all the problems that are claimed to exist, and I've equally seen no research to confirm that they somehow destroy teams.
my anecdotal experience & the lack of data forgive me for being skeptical of a practice that seems largely around to keep process consultants in business
My experience directly contradicts that. Every Scrum-ish team I've worked with has had a pretty steady pace of development and produced good software. Some other processes have worked too; the worst I've seen are fake-agile-actually-waterfall processes.
I’m certainly not receptive to a defense that someone who criticizes it just works at the wrong place or hasn’t seen “real” scrum.
There's no such thing as 'real Scrum' so I won't use that defence. A good process is one that works for the team and company. Scrum is not a universal solution; for extremely high-performing teams it's probably overkill; for bureaucratic teams it will crush everything. It's just another good tool to use where appropriate.
Usually scrum seems to be added for the sake of ‘predictability’
But nothing about the process fundamentally change the problems with software estimation. So in cases where teams are delivering rapidly at high quality but unpredictably, scrum acts as a place to hang paperwork & as a process to hide bad project managers. All while lying to management with charts.
You'd be arguing against research in that case. Software estimation is a widely studied subject and no estimation technique we've found has brought projects into the 'predictable' mode.
There are a few different estimation techniques that are better than others but scrum actively avoids those techniques (probably for the best as they are expensive and don't show huge gains over other forms of estimation).
Instead scrum attempts to track a made up velocity number by using any number of known to be inaccurate estimation techniques. It then gets hyper focused on breaking down tasks into small enough tasks that the velocity number stays constant, all while forgetting about the real desire, the quality software the business desires, delivered to paying customers. At that task, scrum has proven to be no more predictable than any other technique.
An enlightening exercise for any project manager is to go to a sales team and explain what you are delivering in a given sprint. Almost always, they won't care at all about that, and will instead ask for when some 'major epic' thing will be done, at which point you won't be able to tell them with any more certainty than if you'd just guessed.
Thats the best case. The worst case is that all that work you put into showing transparency around velocity ends up being used against the teams working on the hardest and most innovative projects in your organization. Their velocity will be all over the map while the maintenance teams will look good. You'll have convinced the leadership of the organization that velocity is a real number to track (cause you spend so much resources tracking it) and the incentive structures will reflect it. You just made it worse for good engineers to join the teams solving your hard problems.
> There are a few different estimation techniques that are better than others but scrum actively avoids those techniques (probably for the best as they are expensive and don't show huge gains over other forms of estimation).
What methods would you suggest for software estimation then?
I wouldn’t do it. Prioritize your backlog, work on the most important things, reprioritize at your planning meetings.
Invest in going fast, reliability & figuring out priorities. There are lots of agile methodologies that abandoned the estimation step for the reasons I mention (fdd,rad,dsdm).
Not sure if you are joking but unfailing people is no requirement of scrum. Scrum requires the people participating in the process to at least have some fundamental grasp of the process though. If the actors can't be bothered to learn this, then of course it won't work.
If you don't know what values scrum brings over for example traditional waterfall, then the scrum handbook is a good place to start to learn.
I've worked on agile projects for 20 years. I was using agile before the manifesto was published. The opposite of 'not scrum' is not 'waterfall'. Further, I don't disagree with anything Ken Schwaber outlines in his writings on scrum. The problem is, on the ground, that is not what scrum is. Ken has been fighting that battle for at least 10 years.
If the process is just as often implemented incorrectly as correctly, that is a failing of the process. And in this case that is the most damning thing you can say about scrum. More often than not, people do it wrong. I'm tired of that excuse.
> The problem is, on the ground, that is not what scrum is.
The companies I have worked on who claim to use scrum does not suffer from the issues OP brought up. So I question your statement. I also have >20 years experience.
> If the process is just as often implemented incorrectly as correctly, that is a failing of the process.
That's bs. That would be like putting a tractor driver in a Cessna and when the plane crash clamr that Cessnas are inherently unsafe to use. Yes, scrum requires buy in and some basic level of competence. If one or both of these are missing in an organization and they still say they use scrum then that can hardly be blamed on scrum.
My team's product owner is as much part of the team as any of the engineers. Her role includes balancing the need to deliver products with the need to build reliable and scalable software, and to make those arguments to other parts of the company on the team's behalf. Since we're all skilled professionals, we're able to trust each other to make good decisions, and if I say "we cannot proceed until we have done <necessary technical work>" then that is taken seriously and work is prioritised as required.
I'm baffled by the awful environments that people apparently work in where this is not the case.
I've never worked at a place where I've been allowed to say no, I think scrum working the way in which you describe is more the exception than the rule.
The trick that apparently gets lost somewhere along the line is that you don't just say "no", you say "not yet, this first" or "can't, but can do this instead" or...
Either give the product owner a reason and a way to work towards what they want, or an alternative that (hopefully) is just as good.
No it's not lost. It gets the same though longer winded results of saying "no". The b.a. product owner complains to your manager that you're not a team player and you get into trouble with your boss.
Even if that didn't happen you're never going to win, because the b.a.'s have no other job than playing this verbal judo, while you also have to spend your time getting conplex technical things to work. You're like someone who goes to a karate class twice a werk fighting against someone who's life is dedicated to training.
A well-functioning team is composed of skilled professionals. If you are being asked to do something, it is because someone else has a problem that needs to be solved. "No" is an especially unhelpful response to that scenario. Other approaches can work:
- "No, but how about this alternative idea?"
- "No, this prerequisite is required. How can we make that happen?"
- "Not without causing these other problems – let's decide if it's worth the tradeoff"
- "No, that idea is fundamentally impossible. Here's why."
If you work in an environment where engineers are never listened to, then it's a bad environment. But similarly, engineering requirements are not the only ones, and it's important to realise that quite often an engineer doesn't have all of the required information to make the correct long-term decision.
I'm highly suspicious that every time there is one of these threads, agile-consulting companies simply monitor the web for them and send a bunch of posters over to make up fictional srories about how agile is going great where they work.
I've never been in an environment where you could say no to the b.a.'s under agile either. And I've tried being more articulate as well and that just results in the same claims to your manager as "no" does that you're not being cooperative, a team player, etc.
I'm pretty sure after working several places that did agile that most of these "worked great for me" stories are made up.
I've never met another dev in real life who did something other than agile and preferred agile.
In the case of scrum, the role of the Product Owner is extremely important. In theory, the PO should have both a solid background in product development and business development so that the backlog can be correctly prioritized, in order to make sure that the development team can keep on working on the right stuff without discussions. In practice this is rarely the case and the PO often is just some non-technical person that is assigned to do stakeholder management, which puts additional stress on the team, contrary to the goal of keeping stress away from the team.
feels like you have not yet had problems with bad management. Bad management can drive into the ground even a great team with very responsible engineers. They will try to fight the restrictions, burn out, and then simply give up by quitting.
This is the story that keeps unfolding again and again in front my own eyes at my work, alas.
I've experienced plenty of bad management just like that which you described. These managers always seemed to be reacting to having to work with a pool of untrusted incompetents they couldn't get rid of for whatever reason. It all comes down to hiring - even bad managers ease up their control as trust increases.
Isn't Scrum Agile? Agile in corporate parlance usually means "Scrum or XP", and since XP appears to have fallen out of favor, "We are an Agile shop" means "we employ Scrum or at least intend/pretend to".
Use an iterative waterfall with shorter iterations, and keep the process overhead very low (have peer reviews, version control, automated testing, etc.). Beyond that, you need to have a good team, you need to trust the team, and you need to have good management. These are not easy, and no kind of process can solve issues with these.
I moved to an organization using Kanban about a year ago and couldn't be happier. I'd been doing scrum for years and had a lot of complaints similar to the author. Sometimes it's better, sometimes it's worse, but overall it seems like a bad system.
We work by having an engineering manager work out requirements with a product owner, and my manager just comes and asks me how things are going and when they're going to be done. Often times, my estimations are off, but some of that is just expected. Honestly, I'm very happy with it and I've never felt like I was doing better work.
Respectfully, I'm not sure the author understands what he's talking about. He's arguing for the no-op process where engineers determine all their own tasks and priorities - and I mean, I'd like that too. But there's a fundamental difficulty in aligning each engineer's personal goals with those of the company, and some amount of management or organizational design has to go towards addressing it. Engineers won't just naturally align with the organization if nobody rewards us for doing so.
> Do managers or Product Owners track and estimate every task they engage in with little or no say in what they work on?
> Are they required to present burn down charts that show that they are on target to finish?
> Are they required to do bi-weekly sell-off meeting to justify their activities?
These things are in fact the primary responsibilities of managers and product owners.
Theoretically, Scrum is a way to say 'work on this, because it's what aligns with the organization'. But, of course, whether it really does or not depends on who is implementing it and how.
When you let engineers run around doing our own thing, we never have to show that the work we're doing is the most productive use of company time. We just have to show that it plausibly could be. Nobody will hesitate to reward the well-liked engineer with a cool sounding project, even if other things would really be more important.
As the author noted, Scrum solves this problem by making number of story points completed the primary tool for a team to measure its productivity. If the cool-sounding project isn't important enough to take other items out of the sprint, there's no way to offer a post-hoc justification for doing it anyway. It has no story points, so work on it is considered non-productive.
It makes it pretty easy for an engineer to understand what is needed to achieve success, communicate about obstacles in order to get assistance, and have transparency about how their work fits into the larger organization.
Well having done RAD/DSDM (including projects presented at the IEE) some of the bizarre rules based things scrum gets up to seem sub optimal (to be kind).
I have even done very successful hybrid projects that used Waterfall for design and built the system using RAD in 12 weeks - including doing a physical delivery.
> Because all product decision authority rests with the “Product Owner”, Scrum disallows engineers from making any product decisions and reduces them to groveling to product management for any level of inclusion in product direction.
Only if your Product Owner sucks. Their job is to communicate with all stakeholders, which includes letting the engineering staff have their say. If the engineers feel left out, the product owner isn't doing their job.
> Scrum, in accounting for all the engineer’s time in a tightly managed fashion, discourages innovation — which typically occurs spontaneously and outside of any schedule or system of good predictability.
It doesn't. That is why there is a 'focus factor' baked in, to allow for their time to not be tightly managed.
> Scrum encourages “least amount of work possible” solutions — to conform to it’s strict predictability requirements.
No, it doesn't. The scope of a task becomes a negotiation point for assigning stories into sprints. Sometimes you do something quicker to make the sprint work. But that shouldn't be the default answer. It should just be a talking point. Most of the time, the engineers dictate the effort required to complete a feature, and if they always pick the quickest easier way.... well, then that is a problem that needs to be resolved with the team.
I could go on for every point. But I'm not seeing a single shred of evidence that this author actually understands Scrum. They seem to have been exposed to poor scrum teams, focusing completely on processes and documents, not the communications, negotiations, and discussions that are truly the heart of scrum... admittedly put within a defined structure. I'm not saying Scrum is perfect. It isn't, but you definitely are going to hate it if you have never even seen it done well.
> > Scrum, in accounting for all the engineer’s time in a tightly managed fashion, discourages innovation — which typically occurs spontaneously and outside of any schedule or system of good predictability.
> It doesn't. That is why there is a 'focus factor' baked in, to allow for their time to not be tightly managed.
Can tap into my experience and say that it does discourage innovation and has nothing to do with focus.
While doing innovation and research, your path is fuzzy and unpredictable and does not fit into a predefined scope that the team has to commit on at the beginning of the sprint.
While doing research or innovating, you never know if you can deliver business value - your research may succeed... or may fail, which is often the case. Meanwhile, in the predefined sprint scope, everybody is clear what tangible business value it will bring, everything is clear and no room for wiggling and straying away.
Edit: I am not talking about occasional research tasks, which seem to fit well into Scrum structure. I was talking about the innovation as a project that requires a big chunk research to be done.
While doing innovation and research, your path is fuzzy and unpredictable and does not fit into a predefined scope that the team has to commit on at the beginning of the sprint.
This is because you are doing it wrong.
Innovation and research is valuable. It has a business cost, and may not result in business value in all cases. That's totally fine! But it's still important to recognise that so that we can understand the costs and benefits that are being realised.
I pretty frequently engage in long-running research tasks within a Scrum team. The approach that works is to be clear about what the deliverable output is. Perhaps this takes the form of a plan of action, or documentation of the research. If the whole scope of that research project doesn't fit into a sprint, then that's a red flag – break it up.
I've had literally no problems with this approach and it's made the process of understanding the business costs of long-running, uncertain projects much clearer.
I've found that "certificated scrum masters", "certificated project managers" and their ilk are the problem. Anyone who says we must follow a specific process because that is how agile development is done is missing the point.
The actual agile manifesto makes a lot of sense IMO.
Welcome changing requirements, even late in
development. Agile processes harness change for
the customer's competitive advantage.
Everyone has deadlines. Either customer imposed ones, or the market running against you. At some point, you'll have to meet them. When do you stop accepting changes and deliver? Can you welcome last minute disruptive changes? Sure. But the market won't wait for you to implement them.
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.
To be fair, I've only seen Scrum. But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
This is just wishful-thinking, the Agile Point. The only remotely close way of doing this is turning everyone into cogs in a machine. No, thanks. Weren't we valuing the individual?
Continuous attention to technical excellence
and good design enhances agility.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
The best architectures, requirements, and designs
emerge from self-organizing teams.
Some of the most successful products and projects (Linux, Python, Apple) revolve around strong leadership.
There are definitely good ideas in this manifesto. Ideas that have been applied for years, before even the manifesto came to be. Some others are just wouldnt-it-be-nices.
Everyone has deadlines. Either customer imposed ones, or the market running against you. At some point, you'll have to meet them. When do you stop accepting changes and deliver? Can you welcome last minute disruptive changes? Sure. But the market won't wait for you to implement them.
What’s the point in delivering something both you and the customer know is wrong, when you can turn around and fix it instead? The point is delivering something of value instead of delivering something you agreed upon half a year ago.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
The text says "couple of weeks to a couple of months", not 2/3 weeks. If a task requires a longer attention span than two months, you should probably break it down and deliver something of value earler on. As you say yourself: "the market won’t wait for you to implement them".
To be fair, I've only seen Scrum. But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
What a nice way of motivating people. Every person in a good team is of big value, and cannot easily be replaced by anyone else. Creating a good team requires thought and planning. If Scrum is about diluting the individual, then Scrum is far more wrong than I originally thought.
This is just wishful-thinking, the Agile Point. The only remotely close way of doing this is turning everyone into cogs in a machine. No, thanks. Weren't we valuing the individual?
When you motivate people, and give them tasks they like, they tend to perform their best. Turning people into cogs in a machine, as Scrum seems to like, is not a good way of motivating them. As the manifesto says: value the individual and motivate them, and they will get the job done.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
And that is why you don’t care about sprints. You do one task, preferably as small as possible, and deliver it when it is done. That’s it. I have never understood why I should spend loads of time planning what I should do the next N weeks, when I can just do it instead.
Some of the most successful products and projects (Linux, Python, Apple) revolve around strong leadership.
Having strong leaders around a self organizing team is not necessarily problematic. Self organizing teams are more about giving the team freedom to work the way they want. A manager or leader can then contribute by removing obstacles and ensure the team has everything it needs instead of micromanaging everything.
The most important part of The Agile Manifesto, in my opinion, is "Individuals and interactions over processes and tools". If you care more about the process than you do of the people in the team, you are doing it wrong.
Everyone has deadlines. Either customer imposed ones, or the market running against you. At some point, you'll have to meet them. When do you stop accepting changes and deliver? Can you welcome last minute disruptive changes? Sure. But the market won't wait for you to implement them.
There is no point in delivering software which doesn't meet customer requirements. If changes to software are required because it doesn't meet requirements, pretending otherwise solves nothing. Agile processes make it easier to cope when these changes happen, and accept that if change is required then other constraints have to move – such as deadlines.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
Most tasks do not require 'longer attention spans', whatever that means. Almost literally every task can be broken into smaller parts with minimal effort, helping to expose implementations to scrutiny at earlier stages.
And while Scrum traditionally has a fixed sprint length, there's no requirement to follow that to the letter. If your constraints are such that it doesn't work, then adjust the process to suit.
But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
No – Scrum is about the team of individuals. Depending on a single individual is obviously stupid regardless of what process you use.
The only remotely close way of doing this is turning everyone into cogs in a machine.
Not at all true. A nice sustainable pace is reasonable for any team of skilled professionals. If you are not achieving it, your team is broken.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
And pretending otherwise does nothing. Make refactoring a requirement for completion, and you will avoid this problem.
> I've found that "certificated scrum masters", "certificated project managers" and their ilk are the problem. Anyone who says we must follow a specific process because that is how agile development is done is missing the point.
If the best argument they can come up with for the process is "because that's how agile development is done" it's because they don't actually understand the process but that doesn't necessarily have anything to do with them being certified or not.
From experience Agile methods require a different and in general more experienced developers - some one fresh out of uni who only studies cs because of the $ is going to struggle
> it's because they don't actually understand the process
That's my point. They paid money to get "certificated", get jobs as project managers or scrum masters but have no idea how development or agile actually works. I've have come across a lot of these types of people.
But being certified or not doesn't have anything to do with it—you can misunderstand the principles of Scrum just fine on your own. Arguably, certified Scrum Masters would be more likely to actually understand them given their training. And there's a huge difference between "project manager" and "Scrum Master", and if you've met someone that claims to be able to fill both roles in the same team then they're either misunderstanding Scrum horribly or they're that one rare unicorn that managed to make it work that one time.
(And, in case you're interested, "certified" is the standard term. In case you're not interested, my apologies for bringing it up.)
The Agile movement was all about dropping the process hoop jumping and focusing on solving problems efficiently, that's why it was considered extreme. Too extreme, as the one true implementation to rule them all is worse than what the agile movement was revolting against. This is how the game is played, over and over and over again; any movement that threatens the narrative is embraced, corrupted and ultimately used as a weapon against its core values.
Yeah this seems to be an excellent list on how not to Scrum. Or an what happens when scrum is used by people that don't understand the spirit list. I actually scrolled down to see if it was a joke list or had a deeper point.
Scrum is actually used to make sure the dev team is part of the decision process and there is no reason not to be innovative But of course the enterprise can kill all things good
Having been a manager, PM, and lead engineer this article really resonated with me. Scrum has a brief window of utility if a team isn't delivering at all, and has lost management trust. It also has a large window of dis-utility where everyone haggles over points, shows velocity as results, and leaves customers and businesses wondering what their engineers are doing. Trust your engineers to do the right thing on their own and they might just surprise you.
What does “trust your engineers to do the right thing on their own” look like to you? And how is that stopped by SCRUM?
For the way we do it, when a stakeholder wants our team to do something, they come to us with user stories and acceptance criteria that matter to them. Then we hash out the rest of it. I don’t really see a useful way we could be trusted more. The Marketing department knows best what feature it is they want on the website, and why. So they specify that part. We estimate complexity and the business weighs the priority of that feature against other stuff in our backlog and it falls into an appropriate sprint based on that and other motivating factors. Then we do the work.
I don’t see a lack of trust? So I’m curious where you see it and feel it should be done better?
If your company isn’t tiny your engineers generally shouldn’t be deciding priority. Unless your company is small enough that the engineers have a clear view of all the priorities of the business all we are capable of is saying “This seems really important to us.” And then that has to be weighed by people who have the bigger picture.
But outside of a very small company to me it’s a bigger crime to waste engineer time with all the meetings necessary to give them that full perspective. When I started at my current company I was the only developer and I worked directly with the CEO and our single marketing person. Back then I helped decide priority. Now the company is too big for that. I don’t have time for all the manager and marketing and business planning meetings to be able to decide priority.
As far as defining features, we can propose them like anybody else but I think I already explained why usually it makes sense for them to come from whoever owns that piece of what you’re doing. But this of course again depends on the structure of your company.
It's curious that this it is interpreted as a lack of trust of engineers.
> Trust your engineers to do the right thing on their own and they might just surprise you.
Yes, they might surprise you. That's potentially quite terrible because there's a whole ton of other people & roles involved in the success of a product... and of course that's assuming that the project is what the customer wants, requires no marketing message for the customer to understand, and sells itself.
> Trust your engineers to do the right thing on their own and they might just surprise you.
I mean - trust but verify. While I largely agree with you it's always demotivating for the team to realize one dev is doing no actual work and getting away with it. Obviously not good for the business either.
Scrum isn't good or bad, but the way the OP (or their company) uses it seems to be a reflection of an inflexible culture.
For example, the OP argues how scrum strips engineers the autonomy they need to get the job done and how it's super inflexible. When we did scrum with my team, the scrum master role was rotated among engineers, growing each of us significantly as professionals. We also made changes to the "core" of scrum after our retrospectives when we decided it was stuff that did not work for us. When management tried to use story points for tracking or forecasting we told them to forget about the whole thing. We added bugfixing sprints after we observed the bugs piling up.
I can go on - we made scrum work in an environment where we were empowered to do what makes sense. The crux of the post seems to be that OP has some very rigid management and devs cannot change the mindset.
Instead of complaining about scrum, I suggest the OP either try changing the culture from the team level. Or if it won't work, just get out and work at a place where there is sane management who does not drink the rigid scrum kool-aid. Luckily, there are more and more places like that these days.
There isn't enough evidence-based science on what produces the best software -- but Scrum can be measured against other software practices, and should be.
I must have heard this at least a dozen times in the last months. OOP isn't good or bad, Exceptions aren't good or bad, Scrum isn't good or bad. It makes me mad. Guys and girls! Software is not a contest in being liberal and tolerant. There are methods and techniques that are objectively better than others in reaching most of the goals that are important. You wouldn't contest a F1 driver on a bicycle - not on a race track. And engineers who really know their stuff usually agree on how to build software in a given domain. I do not know of any success stories using Exceptions on Kernels or RDBMSs or even most server software. But I know people have tried, and even they agree it was a bad idea.
If you want to say Scrum isn't good or bad, depending on how you tweak it so it sucks less, then I say it's definitely bad. It's not a solid method that helps you reliably reach your goals.
If you want to say Scrum isn't good or bad unless metrics are defined (that doesn't seem to be the case here), then you are right, but it's a meaningless statement.
> OOP isn't good or bad, Exceptions aren't good or bad, Scrum isn't good or bad. It makes me mad.
It's all true though. Do whatever you want, all that matters is that you do it well - you just can't simplify a good software engineer down to techniques like these.
No. There are very objective reasons why e.g. exceptions do not work for projects of advanced complexity. In short, they add additional, hard to see code paths and a redundant method of result handling (with its own noisy syntax, and new semantics) that brings grand-scheme structural complexity (now EVERY component must be exception-safe, every added feature's behaviour with respect to exceptions must be validated) that affects the whole project. Once you realize it starts to be a problem you cannot fix it anymore because it affects the whole project. This is why bigger projects use less and less features.
Using only few features might slow development a little in the small, and you might even make more errors in the small while developping. But you can FIX them.
I realize this is not hard science, and I have no motivation to do any on it, like collecting and bending statistics. But as long as all people have to say "XY is not good or bad", "things will get better", "XY has just a very steep learning curve", "you just didn't dance the right dance" and all evidence seems to suggest otherwise, then you've just lost the argument. End of story.
Exceptions work excellently for business-level server software; typically they make the server transaction all or nothing, with transactional semantics. The stateless nature of HTTP, and transactions provided by the database, usually add up to make this easy. Business logic throws an exception, everything unwinds, the transaction rolls back, no state was modified, and the server returns an error code to the client: all good.
When you need partial updates to state in the case of an error, then you're in trouble. When you have multiple transactions to coordinate, but no distributed transaction coordinator, you're in trouble (but you are anyway whether you use exceptions or not). The more state you need to preserve in an error case, the worse exceptions are for your use case. Ideally, exceptions are never caught except at the topmost loop: server request dispatcher, UI event handler, or command line driver. When execution must continue with recovery, exceptions are an inferior tool.
A http request handler is not what I mean by server software. I mean a real server - a long-running process with complex state. Yes, exceptions work for simple programs and short scripts (where you don't handle them at all), or as a plug-in mechanism where there is a catch-all handler at the boundary. I kind-of value exceptions as well for writing python scripts.
To a first approximation, most servers with distinct codebases are HTTP app servers ("request handlers").
And not just simple programs and short scripts; very complex programs, as long as they keep state in a transactional store, and a request has approximately the same lifetime as the transaction.
Sure, if you want to think of the thing that is formed by all combined requests from all clients over the lifetime of the database, as a program - then that might (or might not) be a complex program. But the unit in which exceptions are confined with this architecture is the simple, dirty request handler script.
Which likely doesn't even try to handle any errors, and may actually be a bad, faulty program - the edge cases just don't matter that much for businesses. Maybe an e-mail will be sent out twice in a few circumstances, etc: nobody cares.
Transaction management is implemented in the database server, and very likely that doesn't employ exceptions. And more complexity is implemented by the OS (the "process server" if you will) - if your program dies it specifies how to handle the condition, etc. Another part of complexity is dealt with by the language runtime, for example garbage collection - no exceptions to be seen there.
> Sure, if you want to think of the thing that is formed by all combined requests from all clients over the lifetime of the database, as a program
The thing that serves responses to requests is a program. And it's not just a "simple, dirty request handler script," it's often enormously complex. Entire companies exist with their product exposed via HTTP APIs.
The reason you don't see exceptions in databases, operating systems, etc is because they're often written in C++, which has notoriously wonky semantics around exceptions, or C, which doesn't have exceptions at all.
The thing is, what is the alternative to a scrum process? I see so many shops with zero defined process constantly re-invent project management wheels that were solved a long time ago. Start from a known place like scrum and then adapt it to your team and organization.
We standardize around protocols, frameworks, programming languages, etc because adding constraints and processes make a problem much easier to solve.
Or even the engineering design process. Every ABET accredited engineering program teaches it, and while there are minor variations in the specifics it's always broadly the same. It's worked well in the engineering disciplines for decades & produces reliable products. There's no reason it can't work well for software, only an unfortunate case of "not invented here syndrome".
A fascinating read, thank you for sharing. These gems of knowledge are what I love HN for.
>This paper outlines the work I reviewed that made it clear to me that ‘people’ really are a first-order, and not incidental, project success driver, the characteristics I currently use as predictors.
My personal experience is similar - methodologies can make things incrementally better for a team, but cannot do anything with the ‘people’ factor.
this is a fantastic piece you've linked, i'm impressed how well it holds up after 20 years, and it admits on multiple occasions that it's been all known 30 years before it's been written. amazing.
>Software is not a contest in being liberal and tolerant. There are methods and techniques that are objectively better than others in reaching most of the goals that are important.
Oh please. Dogmatists make the worst software developers. The "objectively better" criteria is highly situational and not nearly as clear cut as it looks most of the time.
There are plenty of methods and techniques in software that work well for some goals and absolutely suck in others and it's very often not obvious that something that is working well or badly for you is doing so mostly because of your situation.
My favorite example of this is microservices - which apparently worked really well for Martin Fowler once or twice (and I'm 90% sure he didn't understand why) and then when he promoted it, it got applied dogmatically by teams all over the world in situations where it was manifestly an idiotic idea.
The same goes for most hot-button issues - type systems, functional programming, nosql, etc.
Believe me, I know that microservices suck for almost anything, because of the added software and infrastructure management complexity.
And so it is with most other hyped technologies. The complexity and loss of control is very rarely worth it. I'm not a dogmatist, but have seriously tried many different approaches. And that's why I do mostly C (well, not for web applications, which I try to avoid as well :>) with only very simple data structures, and I love it, and it works, and it's maintainable, modular, and very performant. Although I know that C has flaws - I have been trying to find ways to improve (and failed so far).
>Believe me, I know that microservices suck for almost anything
woosh
They work really well in large organizational environments where inter-team coordination costs are very high and language flexibility is valuable. Where these benefits outweigh the additional infrastructure headaches it works very well.
I've likewise worked in environments where they were a terrible idea (e.g. startup team of six).
Like you, I used to be dogmatically anti-microservice. I was wrong.
Yes - that may be, so I'm sorry for omitting the context that I don't work in large teams where you can throw lots of money and developers at problems. (I think that's a sad state of affairs, though). Again, I'm not dogmatic, but have actually tried it, and I know (and have described in my comment) the problems they bring.
It seems to me that you are young and just starting your career.
It’s ok to only want to do certain kinds of work (for example C code that runs close to the metal and that is performant and easy to reason about). Other people in the world though enjoy doing different kinds of work and going about it in different ways.
My advice to you is to be humble and watch people as if they had something to teach you. Whether they are doing scrum, waterfall, using this or that language or architecture, there’s always a ton of things you can learn by watching how someone works.
If you pay closer attention, you will see value in what other people are doing and be able to benefit from what they have discovered.
Very true. There's an alchemy between the software you already have, the tools and technologies your team are familiar with, the problem domain you have, and the solution you implement. There are multiple stable equilibria, multiple local optima. In a more complex environment, microservices can let different teams find local optima better suited to their combination of factors.
I'd still be suspicious of people who start out saying they're going to implement a microservice architecture (especially greenfield) without having an understanding of why, though.
I'm suspicious of anybody who starts out with a clear picture in their head of "what the architecture should look like".
I've almost always found that the earlier on in the process I made hard-to-reverse architectural decisions, the worse the architecture ended up being. The more the architecture came about as a series of gradual, iterative refactors that took in to account the various trade offs that became evident, the better it ended up being.
I'm an anonymous architecture astronaut, so usually I have those "should look like" high-level pictures in my head.
But that's just a fucking picture. A plan. That usually does not survive contact with the changing requirements and implementation reality.
Anyone incapable of admitting that there will be changes should be taken upstate to a nice farm, then shot behind the barn. Usually these people are on the client's team. :(
>> Scrum isn't good or bad
>I must have heard this at least a dozen times in the last months.
The thing about any agile methodology is that they are meant to be adapted to the situation at hand, not to be a fixed set of rules that you can just apply to get consistently good results. But many are trying to turn scrum into that - a fixed set of rules that allows micromanagement. I would say that that is definitely bad, but, is that really scrum?
> I do not know of any success stories using Exceptions on Kernels or RDBMSs or even most server software.
I think you need to get out more.
Exceptions are actually pretty standard in the database world. ScyllaDB, arguably the top dog in terms of DB performance, has the "throw" keyword strewn throughout its code base. Oracle PL/SQL's error handling is done entirely through exceptions...
Is this about the database engine itself? Or just the query parser? Or even just exceptions in the query language itself?
FWIW, Scylla is not an RDBMS. I don't know about its architecture and am not interested in digging more. And that thing works reliably? I don't know - not my cup of tea. Why am I reminded of Node.js? Not cherry-picking here...
future<file> sstable::open_file(component_type type, open_flags flags, file_open_options opts) {
auto f = new_sstable_component_file(_read_error_handler, filename(type), flags, opts);
if ((type != component_type::Data && type != component_type::Index)
|| get_config().extensions().sstable_file_io_extensions().empty()) {
return f;
}
return f.then([this, type, flags](file f) {
return do_with(std::move(f), [this, type, flags](file& f) {
auto ext_range = get_config().extensions().sstable_file_io_extensions();
return do_for_each(ext_range.begin(), ext_range.end(), [this, &f, type, flags](auto& ext) {
// note: we're potentially wrapping more than once. extension mechanism
// is responsible for order being sane.
In the 60s or so, people came up with this thing called scheduler. It allowed them to run things (virtually) at the same time, while not needing to make a mess of their code... They also invented cooperative multitasking, where the scheduler only kicks in when explicitly called.
f = open_file_or_schedule(filepath);
read_or_schedule(f, buf, size);
I think this is a common misunderstanding: Scrum is not just a tool, it's a toolbox. You don't use the whole toolbox for every job, you pick the tools you need and leave the rest.
BTW, Exceptions and OOP are used conceptually in most kernels, RDBMS, and any bigger piece of software, it just doesn't make sense to use languages which force their use if a project has too many edge cases.
I agree: A few explicit function pointers / vtables and well-considered use of longjmp or cooperative multitasking (coroutines, switching call stacks) is the way to go in some situtations. The point is "explicit".
>Scrum isn't good or bad, but the way the OP (or their company) uses it seems to be a reflection of an inflexible culture.
It's not the company. This inflexibility is a reflection of scrum's culture. I've seen this inflexibility on multiple teams across the world - what you've experienced is very much the exception and what the OP described is basically the norm.
>We also made changes to the "core" of scrum after our retrospectives when we decided it was stuff that did not work for us.
This is great and it's exactly how I'd want to work but I think almost every single scrum consultancy not only advocate strongly against doing this, they would likely blame anything that went wrong on your "rash" decision to not adhere to core scrum principles.
>We added bugfixing sprints after we observed the bugs piling up.
This is actually something I've heard explicitly called out by consultancies.
>Instead of complaining about scrum, I suggest the OP either try changing the culture
I don't agree. I suggest slagging scrum off until it dies, because the inflexibility which you decry is so deeply embedded in its culture and will only die when scrum does.
Sure, what you did worked great - it just wasn't "scrum".
The whole scrum isn’t good or bad thing annoys me. It’s like the people who argue in favor of communism and when you point out its immense failure rate they say, “well that’s not really communism”. Scrum seems to have a very poor implementation rate and seems to draw the ire of most engineers.
Scrum seems to have a very poor implementation rate and seems to draw the ire of most engineers.
Engineers are as responsible as any other actor for poor Scrum implementations.
Scrum, or any agile method, isn't a silver bullet. It doesn't compensate for bad teams. It's something that teams can use to deal with some common software development issues. It's also not the only tool that can be used for that purpose. Developers pouring ire on "scrum" when they aren't practicing it in the first place are attacking the wrong target.
Process, design, and yes... governing is a series of trade offs. You can certainly talk about the merits of the trade offs and whether they are appropriate for a given context, but a critique that fails to understand the trade offs and speaks in strict good/bad interpretations comes across as terribly naive.
I work for a good company but the team I was on a few years ago was using scrum. It really does promote pushing debt down the line because the product owner only cares about what they can see or measure. The debt isn't part of that set so the developer has to fight to address it. After the first battle many will stop fighting because they are measured by how well the product owner thinks that developer is implementing their roadmap and on track with the schedule they got approved.
Products with heavy debt tend to get rewritten rather than fixed. So it makes sense to keep the schedule through shortcuts and push as many features as possible. Then throwaway the product and rewrite in something newer and more modern.
For developers the benefit is your resume will have the latest tech and you can write sloppy non-abstracted code and not worrying about having to support the product later. Management gets increased visibility into each task with an accurately followed schedule.
Not a surprise it has become so popular. If I could change one thing I would make the scrum master in charge of technical documentation. That one change would force the master into understanding the code at a lower level. That would make them the most knowledgable person on the project thus it would remove the need for daily standups because team members could go to the master at any point in the day with questions or to coordinate effort.
SCRUM does not push for bad, tech
debt laden products any more than any other development system. Managers who want you to push code out the door ASAP will do it just as much no matter the system you’re using.
The solution is not to blame whatever particular form of agile you’re using; the solution is to replace the bad managers if you can’t educate them on why that’s short sighted.
The development team is responsible for technical implementation. Also, the priorities are a negotiation with the product ownern.
It is also in the product owners best interest that the tech debt is not piling up.
In the end the PO is screwed when the product comes to halt because of the tech debt.
I just put my teams on scrum and they love it. Immature scrum that focuses on metrics and jira boards can be a very tough transition, but mature scrum that focuses on collective ownership and continuous improvement is heavenly.
> It assumes that engineers cannot conduct a meeting effectively without a facilitator (Scrum Master)
Scrum isn't perfect but this is certainly not true. If the team is well-functioning there doesn't need to be a facilitator of any kind present. Sounds like OP is either in a bad team or with a bad Scrum master—or both.
Only if you misunderstand me. ;) What I meant was that Scrum doesn't dictate that there has to be a facilitator present but if your team isn't well-functioning it's probably a good idea to have one. I.e. if the Scrum master has done his job exceptionally well he's rarely needed.
We do scrum and this doesn’t describe how it works for us at all. Our scrum master isn’t our manager and neither is our product owner, for example. And we have gone some time with minimal direction from the product owner at all, when things were going well.
My main complaint about scrum was how much of my time was spent in meetings vs working.
Thankfully I dont do agile any more, but I'm close to several teams that do. My fondest memories are:
No over-arching design (Implement feature after feature however the hell anyone likes)
Once said features are implemented, taking the time to "refactor" (read: completely rewrite because the code was so bad) becomes a really hard sell.
Thus, new features get progressively harder to implement, as you touch a deeper cross-section of the code to try and fix stuff semi-covertly, without a formal refactor.
"Tech Debt" was done in sprint scheduled after phase of dev sprints. After, say, a year of development, lets fix all the problems in 3 weeks... When you're dealing with procedural php masquerading as OO, MVC with business logic everywhere it shouldnt be, and abuse of inheritance such that you cant throw an error in a command line because you're not a logged in browser user, who's had their language queried from a database, and doesnt fall back to default when the db isnt there... 3 weeks isnt gonna cut it!
Introducing peer review was supposed to sort that, but bad or inexperienced devs didnt know they were doing shoddy work (it met the criteria after all..), and good devs became impediments because so many stories needed redoing. Demoralising.
We were probably "doing agile wrong". But it's like a religion: open to interpretation. In my experience it's a tool used to extract maximum customer benefit at the expense of the developer's soul, under the pretence of "self managing team" - ie. "You brought it on yourself".
I agree, which is why I pushed for it. But it does depend on the quality of the peers, and should go beyond whether the story met the criteria or not.
Feeding that back through the process is pointless when it inflates the complexity of future stories, because they are less likely to be prioritised, and the rot festers.
> But it does depend on the quality of the peers, and should go beyond whether the story met the criteria or not.
Peer review is a good way to improve the quality of peers, either though the natural education/knowledge sharing, or simply the transparency that helps you remove low quality peers and find better ones. It's not a cure all, but it's hard to imagine a successful team that wouldn't benefit form it.
Some turn up in the morning, however, work on their task for the day and go home. If they reimplement a chunk of the codebase somewhere else, because they feel they need to be writing more code to complete their task, fine, the story still passes. "I didn't know that was there" is not an excuse, it was probably covered in planning but they forgot. It's not going to get refactored, because it works! Peer review in this case goes one of two ways:
1. Bad quality peer sees criteria are satisfied and approves
2. Good quality peer sees all the duplication, some other bad practice thrown in for good measure and disapproves
In 2. say the story has already carried, pushing more work back, and hard deadlines are looming. They will be forced to begrudgingly approve the work, knowing they've just allowed a good chunk of cruft to enter the codebase. They will try and educate (assuming they've not lost the will to live - this might be the 100th time for eg), to prevent it happening again. But it will still happen, because bad developer is bad.
Forgetting the looming hard deadline: If the story carries over so what? We'll retro and add some bloat to the process to try and avoid it happening again.
What isn't explicitly said is "this dev is committing crap and the competent people are getting demoralised", it's the process that gets "refined" not the people. Agile is papering over the cracks and nurturing the codemonkey's behaviour.
But none of this matters, as the deliverable has been delivered. Now on to the next story!
Personally, I dont, I escaped into server admin. Part of that was trying to get the crap to actually run well on the systems we had!
But the organisation is slowly carving our monolith into microservices (Because buzzwords will save the day! <sigh>). I do feel microservices lend themselves better to agile, as the programmatical boundaries a good dev should naturally code within become rigidly defined. And people can effectively do their own thing within their "box".
IMHO it shifts the problems to the layer above, papering over the cracks. (Badly written code performing badly? Whack it behind a gateway and spin up some more containers!)
Even with microservices you're gonna have to define that API at some point (and now we can have as many as we like behind the scenes!), why not at the start with a holistic view... like, a design!
"As a user I can see a login page" - ok here's a static html page with two input boxes and a submit button.
"As a user I can log in" - implement that slice of the backend, all in one hit with no eye on what other things might need from it (Dont get me started on YAGNI... YMNI, imho and you'll miss it when it isn't there! Especially when there's "no time" because it's not a feature with perceived customer benefit which take priority)
And it goes from there. What I would prefer is more like "before we can even think about showing a login page to a user we should have..." which is more bottom up, and doesn't look good to a customer (or management).
> Once said features are implemented, taking the time to "refactor" (read: completely rewrite because the code was so bad) becomes a really hard sell.
This should be a hard sell. The code in production works. Why does it in need a rewrite? The fact it needed a rewrite in the first place doesn't really reflect on scrum but on poor engineering and management practices.
This might be what I have the biggest problem with. To a customer, it works. The customer is happy, they pay their money, which makes management happy, and we all get paid. So the site takes 10 seconds to load on certain pages, who cares! It works, right?
Behind the scenes, there's a fleet of web servers burning CPU so add some more! A database that's overwhelmed, throw some more CPU/mem and faster disks at it! Maybe optimise some queries if you can get it into sprint. But a fundamental problem with the implementation itself cant be fixed, because "it works". Want to leverage some slave dbs for reads? Sorry, no time for that because unpicking the backend would take too long!
It's backwards. In my idealised view: Good design first = better code = less resource hungry = less cost for servers = support more users = more revenue = less time spent working around crap = more time to implement features! = more revenue = better wages. Everyone wins.
If you really have a fleet of web servers burning CPU and you arne't tracking the costs, then I guess "it works" until someone figures out there isn't any money left in the coffer.
There is a lot of value in working software, and unless your engineering infrastructure expenses seriously outweigh your engineering payroll costs (which does happen, but is hardly commonplace), CPU efficiency isn't the be-all to end-all and focusing on it may well prove to be counter productive. If no one actually cares about your page load times, it may very well be that it doesn't matter. You can spend a near infinite amount of time & resources trying to get optimal performance (talk to someone who does high-frequency trading to get a sense) well past the point where the costs pay off.
There is tremendous value in "it works" that engineers generally struggle to fully factor into their thinking (I come by this wisdom honestly). People have written about this extensively (e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-...). That said... the notion that you can't fix actually serious and counter-productive problems with the product because "it works" sounds not only counter to Scrum, but really counter to any software development methodology I've ever heard of.
> If you really have a fleet of web servers burning CPU and you aren't tracking the costs, then I guess "it works" until someone figures out there isn't any money left in the coffer.
What if the costs are fixed? (own servers, unmetered bandwidth) The problem shifts down the line when you want to add a new product and the first product is consuming all your resources. Refactoring then becomes a necessity, but there's no capacity for it because the company is focused on shiny product #2, while also adding shiny features to product #1. Now fire a few people to save money, leaving the remainder short handed, who get stressed and demoralised. But that's ok, because "it works!". Why cant we design first, lay down a good foundation, then add features. The business makes more money, people keep their jobs, instead of spit-balling "user stories" at an application which becomes a giant spit-ball.
> CPU efficiency isn't the be-all to end-all and focusing on it may well prove to be counter productive
> You can spend a near infinite amount of time & resources trying to get optimal performance
I used CPU as a broad example. What happened in my case was there was no performance. At release it was unusable. What stuck in my side was I knew it was going to happen, and had been calling for refactors. But if you cant get them prioritised because you cant justify the customer benefit (it's not released yet, so nothing has fallen over yet) vs features! features! features! it then becomes a bodge job /after/ release. eg. Instead of "refactor this code to make it do fewer queries" it's "lets shove the results of these queries into a memory cache." This satisfies agile, you've iterated and made things "better" - and it fit into a 2 week sprint! - but it feels backwards.
I was going to go on and talk about quantity vs quality, but a quick fact-check google revealed the evangelists with their "Agile parables" (it even sounds cultish), which are in no way perpetuating Agile to sell support tools, say I'm wrong.
I need to thank you for making me realise it's time for a career change. After all this time Agile has failed to convince me, I'm inclined to conclude it never will. But should the next methodology of the moment come along that emphasises quality over quantity, I reserve the right to shout "I told you so!"
> But should the next methodology of the moment come along that emphasises quality over quantity, I reserve the right to shout "I told you so!"
Again, your problem isn't coming from the methodology. The priority on quality comes from the context, not the methodology. I have a friend who does presentations on agile all the time and his team prioritizes quality pretty much over anything simply because it allows them to develop faster.
This is the biggest barrier to making things better. Eventually it leads to fear and risk-averse stasis. You never do anything the best way the first time, but if you get locked into a shitty implementation because "fuckit, it works", that's a rot that will poison your codebase and team.
> No over-arching design (Implement feature after feature however the hell anyone likes) Once said features are implemented, taking the time to "refactor" (read: completely rewrite because the code was so bad) becomes a really hard sell.
Can I ask why do you deliver stories before refactoring, if that's what you're saying? We'd tend to do necessary refactors as part of features or bugs when we touch related code and rarely have pure refactor chores.
Also, why do you have to _sell_ code quality? At least where I work there is a pretty strong divide between what to build and how to build it. There is two-way trust between the product manager and engineers that they know their domain and when they say "this is important right now", it's probably the right thing. After that it's just a matter of communication and prioritisation between the the two sides.
I could easily see how this might fall apart if either side was not trusted or not doing a great job, but I guess it would probably fall apart no matter what development process you followed if that were true.
This post confuses many things but firstly it doesn’t properly talk about Product Management. PM is customer centric and marketing heavy discipline and just knowing how to code and good intentions of devs may not work on competitive market. Secondly Scrum is project management framework and says nothing about software development practices and leaves them completely to the judgment of the team (including developers and PO because they collaborate heavily in Scrum).
> 7. Scrum is very management heavy. Typical teams have Product Owners, Scrum Masters, and Team Leads. Innovative, motivated teams operate better with less management, not more.
> 8. Scrum is typically implemented with HORRIBLE task management tools (Jira, tfs, etc…) that enforce very bureaucratic interpretations of Scrum that waste huge amounts of developer time. Additionally, they effectively lock you into one mode of operation, no matter how ineffective.
Scrum is not just management heavy. It's micromanagement heavy. If someone asked me for an account of what I did for a whole six (assumed to be productive) hours yesterday and how many story points have been burned, and wanted this every single day, I can only imagine that the person has no experience with software development whatsoever. There are productive days when a lot of stuff gets done and there are unproductive days when almost nothing gets done because it's hard to focus or because of distractions. But the daily standup meeting that's enforced would make any unproductive days as something to get publicly shamed about.
On the tools, yes, whatever I've seen in Scrum dictates a lot of time spent updating the tools with estimated hours, hours spent, remaining hours, everyday. That data is rarely useful for any further estimations or to figure out what's going on.
Lastly, the "story points not related to effort but related to effort" conundrum confuses developers (especially the ones who are considered to follow instructions and code whatever the user story's tasks say). Add to this a really weird concept called velocity, which is not just a measure of how many story points are completed in a sprint, but is used as a range — too low and you're not working hard enough; too high and your estimates are wrong. The only way to satisfy the Scrum master in many cases is to lie and make the velocity conform to expectations.
I have no idea who was tying to prove what to whom when they came up with Scrum and Scrum certifications. The certified Scrum masters I've seen don't even know how to write a good user story or to break it into tasks.
Give me low process overhead iterative waterfall any day over these schemes. Repeatedly trying to turn software developers into predictable manufacturing style widget makers is not a good idea.
The daily stand-up is simply about "what did I do yesterday, what will I do today and do I have any impediments". It has nothing do with productivity evaluation.
You have given an example of a shitty company using a shitty process. It's unfortunate that it's nominally 'Scrum', but that's kind of irrelevant.
If someone asked me for an account of what I did for a whole six (assumed to be productive) hours yesterday and how many story points have been burned, and wanted this every single day, I can only imagine that the person has no experience with software development whatsoever.
This is wrong. If this is happening, you are following a broken process.
On the tools, yes, whatever I've seen in Scrum dictates a lot of time spent updating the tools with estimated hours, hours spent, remaining hours, everyday.
No it doesn't. This is wrong and broken.
Lastly, the "story points not related to effort but related to effort" conundrum confuses developers
If a software engineer can't immediately grasp this abstraction then they are not good software developers.
too low and you're not working hard enough; too high and your estimates are wrong
Velocity should not be used as a stick. It's a measure of the amount of work being done which can be used for planning. If it fluctuates, that can indicate problems that need to be dealt with.
Give me low process overhead iterative waterfall any day over these schemes. Repeatedly trying to turn software developers into predictable manufacturing style widget makers is not a good idea.
"Low process iterative waterfall" is basically Scrum.
I work two-week sprints. There are a total of four hours of meetings for those two weeks. 5% overhead seems pretty reasonable.
The problem, as I see it, is having Scrum certifications and certified Scrum Masters who follow a rigid process with many conditions (which is antithetical to the agile mindset).
I know “low process overhead iterative waterfall” sounds agile, but somehow the word “Scrum” seems to make people want a certified Scrum Master (these people usually frown on waterfall-like processes even if those are better suited for delivery).
A lot of this article is a mix of a little bit of truth with a lot of what is just bad management.
But the first few and final example are good warnings that can be applied anywhere, Scrum or not. Paraphrasing / quoting the article:
"[If] all product decision authority rests with the “Product Owner” ... engineers [are disallowed] from making any product decisions and reduces them to grovelling to product management for any level of inclusion in product direction."
"[If] all the engineer’s time [is accounted for] in a tightly managed fashion, [that] discourages innovation — which typically occurs spontaneously and outside of any schedule or system of good predictability."
"[You] encourage “least amount of work possible” solutions [when you expect engineers] to conform to ... strict predictability requirements."
"[If you] divide every task into small items that can theoretically be completed by anyone on the team .. [you] discourage engineers from taking pride in and/or ownership of their work. This lack of ownership results in ... Poor design [and] Lack of motivation ('It isn’t my thing', 'It was broken when I start working on it')"
"[Don't] ignore the fact that any task that has been done before in software does not need to be redone because it can be easily copied and reused. So, by definition, new software tasks are truly new territory and therefore very hard to estimate."
This is a list of things they are doing wrong not things wrong with Excel, Visual Studio, JIRA, The Scrum Methodology or any other tool they have poorly implemented.
They should seek external help in getting their usage of these simple tools back on track.
I’ve seen people create horrible abominations using Excel, that doesn’t mean that the tool itself is broken or that all the effective usages of Excel are wrong. It’s the same with Scrum. If you use it right it’s awesome, if you use it wrong it’s just a different name for the same old bad project management you always had.
173 comments
[ 2.9 ms ] story [ 224 ms ] threadBut keep telling us it's for product quality.
> Because all product decision authority rests with the “Product Owner”, Scrum disallows engineers from making any product decisions and reduces them to grovelling to product management for any level of inclusion in product direction.
The product owner is part of the scrum team. She or he is responsible for the backlog but if she has trust in the engineers she can let the engineers add their own things to it.
I think OP just works in a bad place.
Or to give a much more serious response : this article seems to be very unprofessional, about trying to defend unprofessional behaviour as "normal".
The scrum guide talks about roughly 1hour/week for the scrum team to show stakeholders (such as upper managers and customers) what has been done. Any engineer can at any time reach out to people in the company to demo what has been created, if there's interest.
So I'm not sure what point you're trying to make with starting your own company if you want visibility outside the team.
The process is there to account for the places where the people fail. If a requirement of a process is unfailing people then it’s not a valuable process.
That's just wrong. It's there for lots of reasons – to help avoid failures and surprises, to deliver sustainable pace, to help make estimations, to ensure a common understanding of problems and challenges… the list goes on.
Scrum doesn't require infallible people – it's just a tool to help avoid failures, which happen regardless of the skill of your team.
I’ve seen instances of all of the items the article claims being problems in practice. That was across a wide variety of teams & skill levels.
I’ve also seen no evidence in the literature or research that backs up your assertions.
I’ll concede some teams have benefited from scrum practices, but in the face of my anecdotal experience & the lack of data forgive me for being skeptical of a practice that seems largely around to keep process consultants in business.
I’m certainly not receptive to a defense that someone who criticizes it just works at the wrong place or hasn’t seen “real” scrum.
Why was Scrum added to these teams if they were already working? It's a tool to deal with problems. It's not something to use if another process is already working.
I’ve seen instances of all of the items the article claims being problems in practice.
I have found essentially the exact opposite of all the problems that are claimed to exist, and I've equally seen no research to confirm that they somehow destroy teams.
my anecdotal experience & the lack of data forgive me for being skeptical of a practice that seems largely around to keep process consultants in business
My experience directly contradicts that. Every Scrum-ish team I've worked with has had a pretty steady pace of development and produced good software. Some other processes have worked too; the worst I've seen are fake-agile-actually-waterfall processes.
I’m certainly not receptive to a defense that someone who criticizes it just works at the wrong place or hasn’t seen “real” scrum.
There's no such thing as 'real Scrum' so I won't use that defence. A good process is one that works for the team and company. Scrum is not a universal solution; for extremely high-performing teams it's probably overkill; for bureaucratic teams it will crush everything. It's just another good tool to use where appropriate.
But nothing about the process fundamentally change the problems with software estimation. So in cases where teams are delivering rapidly at high quality but unpredictably, scrum acts as a place to hang paperwork & as a process to hide bad project managers. All while lying to management with charts.
I'd argue quite the contrary.
> scrum acts as a place to hang paperwork & as a process to hide bad project managers. All while lying to management with charts.
So weird. The "paperwork" tends to make bad project managers all too transparent.
There are a few different estimation techniques that are better than others but scrum actively avoids those techniques (probably for the best as they are expensive and don't show huge gains over other forms of estimation).
Instead scrum attempts to track a made up velocity number by using any number of known to be inaccurate estimation techniques. It then gets hyper focused on breaking down tasks into small enough tasks that the velocity number stays constant, all while forgetting about the real desire, the quality software the business desires, delivered to paying customers. At that task, scrum has proven to be no more predictable than any other technique.
An enlightening exercise for any project manager is to go to a sales team and explain what you are delivering in a given sprint. Almost always, they won't care at all about that, and will instead ask for when some 'major epic' thing will be done, at which point you won't be able to tell them with any more certainty than if you'd just guessed.
Thats the best case. The worst case is that all that work you put into showing transparency around velocity ends up being used against the teams working on the hardest and most innovative projects in your organization. Their velocity will be all over the map while the maintenance teams will look good. You'll have convinced the leadership of the organization that velocity is a real number to track (cause you spend so much resources tracking it) and the incentive structures will reflect it. You just made it worse for good engineers to join the teams solving your hard problems.
What methods would you suggest for software estimation then?
Invest in going fast, reliability & figuring out priorities. There are lots of agile methodologies that abandoned the estimation step for the reasons I mention (fdd,rad,dsdm).
If you don't know what values scrum brings over for example traditional waterfall, then the scrum handbook is a good place to start to learn.
If the process is just as often implemented incorrectly as correctly, that is a failing of the process. And in this case that is the most damning thing you can say about scrum. More often than not, people do it wrong. I'm tired of that excuse.
The companies I have worked on who claim to use scrum does not suffer from the issues OP brought up. So I question your statement. I also have >20 years experience.
> If the process is just as often implemented incorrectly as correctly, that is a failing of the process.
That's bs. That would be like putting a tractor driver in a Cessna and when the plane crash clamr that Cessnas are inherently unsafe to use. Yes, scrum requires buy in and some basic level of competence. If one or both of these are missing in an organization and they still say they use scrum then that can hardly be blamed on scrum.
Nailed it.
My team's product owner is as much part of the team as any of the engineers. Her role includes balancing the need to deliver products with the need to build reliable and scalable software, and to make those arguments to other parts of the company on the team's behalf. Since we're all skilled professionals, we're able to trust each other to make good decisions, and if I say "we cannot proceed until we have done <necessary technical work>" then that is taken seriously and work is prioritised as required.
I'm baffled by the awful environments that people apparently work in where this is not the case.
Either give the product owner a reason and a way to work towards what they want, or an alternative that (hopefully) is just as good.
Even if that didn't happen you're never going to win, because the b.a.'s have no other job than playing this verbal judo, while you also have to spend your time getting conplex technical things to work. You're like someone who goes to a karate class twice a werk fighting against someone who's life is dedicated to training.
A well-functioning team is composed of skilled professionals. If you are being asked to do something, it is because someone else has a problem that needs to be solved. "No" is an especially unhelpful response to that scenario. Other approaches can work:
- "No, but how about this alternative idea?"
- "No, this prerequisite is required. How can we make that happen?"
- "Not without causing these other problems – let's decide if it's worth the tradeoff"
- "No, that idea is fundamentally impossible. Here's why."
If you work in an environment where engineers are never listened to, then it's a bad environment. But similarly, engineering requirements are not the only ones, and it's important to realise that quite often an engineer doesn't have all of the required information to make the correct long-term decision.
I've never been in an environment where you could say no to the b.a.'s under agile either. And I've tried being more articulate as well and that just results in the same claims to your manager as "no" does that you're not being cooperative, a team player, etc.
I'm pretty sure after working several places that did agile that most of these "worked great for me" stories are made up.
I've never met another dev in real life who did something other than agile and preferred agile.
Show me a better system instead.
This is the story that keeps unfolding again and again in front my own eyes at my work, alas.
https://thehftguy.com/2017/12/06/scrum-vs-kanban-arent-they-...
We work by having an engineering manager work out requirements with a product owner, and my manager just comes and asks me how things are going and when they're going to be done. Often times, my estimations are off, but some of that is just expected. Honestly, I'm very happy with it and I've never felt like I was doing better work.
Scrum is a framework. Kanban is a method of implementing a system.
Here's Kanban creator David J. Andersen's own words:
http://www.scrumcrazy.com/David+Andersen+-+There+is+no+kanba....
Also Kanban has certification:
http://leankanban.com/team-kanban/
And consulting services around it:
http://leankanban.com/project/services/
This is: https://en.wikipedia.org/wiki/Kanban_(development)
Since 2005ish.
About 10 years after Scrum was formalised at OOPSLA 95'
> Do managers or Product Owners track and estimate every task they engage in with little or no say in what they work on?
> Are they required to present burn down charts that show that they are on target to finish?
> Are they required to do bi-weekly sell-off meeting to justify their activities?
These things are in fact the primary responsibilities of managers and product owners.
Hoes does Scrum reward the engineer for such alignment?
edit: grammar
How does Scrum reward the engineer, in a way that it does it better, and more profoundly, than other methodologies?
As the author noted, Scrum solves this problem by making number of story points completed the primary tool for a team to measure its productivity. If the cool-sounding project isn't important enough to take other items out of the sprint, there's no way to offer a post-hoc justification for doing it anyway. It has no story points, so work on it is considered non-productive.
I have even done very successful hybrid projects that used Waterfall for design and built the system using RAD in 12 weeks - including doing a physical delivery.
Only if your Product Owner sucks. Their job is to communicate with all stakeholders, which includes letting the engineering staff have their say. If the engineers feel left out, the product owner isn't doing their job.
> Scrum, in accounting for all the engineer’s time in a tightly managed fashion, discourages innovation — which typically occurs spontaneously and outside of any schedule or system of good predictability.
It doesn't. That is why there is a 'focus factor' baked in, to allow for their time to not be tightly managed.
> Scrum encourages “least amount of work possible” solutions — to conform to it’s strict predictability requirements.
No, it doesn't. The scope of a task becomes a negotiation point for assigning stories into sprints. Sometimes you do something quicker to make the sprint work. But that shouldn't be the default answer. It should just be a talking point. Most of the time, the engineers dictate the effort required to complete a feature, and if they always pick the quickest easier way.... well, then that is a problem that needs to be resolved with the team.
I could go on for every point. But I'm not seeing a single shred of evidence that this author actually understands Scrum. They seem to have been exposed to poor scrum teams, focusing completely on processes and documents, not the communications, negotiations, and discussions that are truly the heart of scrum... admittedly put within a defined structure. I'm not saying Scrum is perfect. It isn't, but you definitely are going to hate it if you have never even seen it done well.
> It doesn't. That is why there is a 'focus factor' baked in, to allow for their time to not be tightly managed.
Can tap into my experience and say that it does discourage innovation and has nothing to do with focus.
While doing innovation and research, your path is fuzzy and unpredictable and does not fit into a predefined scope that the team has to commit on at the beginning of the sprint.
While doing research or innovating, you never know if you can deliver business value - your research may succeed... or may fail, which is often the case. Meanwhile, in the predefined sprint scope, everybody is clear what tangible business value it will bring, everything is clear and no room for wiggling and straying away.
Edit: I am not talking about occasional research tasks, which seem to fit well into Scrum structure. I was talking about the innovation as a project that requires a big chunk research to be done.
This is because you are doing it wrong.
Innovation and research is valuable. It has a business cost, and may not result in business value in all cases. That's totally fine! But it's still important to recognise that so that we can understand the costs and benefits that are being realised.
I pretty frequently engage in long-running research tasks within a Scrum team. The approach that works is to be clear about what the deliverable output is. Perhaps this takes the form of a plan of action, or documentation of the research. If the whole scope of that research project doesn't fit into a sprint, then that's a red flag – break it up.
I've had literally no problems with this approach and it's made the process of understanding the business costs of long-running, uncertain projects much clearer.
The actual agile manifesto makes a lot of sense IMO.
http://agilemanifesto.org/
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
Everyone has deadlines. Either customer imposed ones, or the market running against you. At some point, you'll have to meet them. When do you stop accepting changes and deliver? Can you welcome last minute disruptive changes? Sure. But the market won't wait for you to implement them.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
To be fair, I've only seen Scrum. But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
This is just wishful-thinking, the Agile Point. The only remotely close way of doing this is turning everyone into cogs in a machine. No, thanks. Weren't we valuing the individual?
Continuous attention to technical excellence and good design enhances agility.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
The best architectures, requirements, and designs emerge from self-organizing teams.
Some of the most successful products and projects (Linux, Python, Apple) revolve around strong leadership.
There are definitely good ideas in this manifesto. Ideas that have been applied for years, before even the manifesto came to be. Some others are just wouldnt-it-be-nices.
What’s the point in delivering something both you and the customer know is wrong, when you can turn around and fix it instead? The point is delivering something of value instead of delivering something you agreed upon half a year ago.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
The text says "couple of weeks to a couple of months", not 2/3 weeks. If a task requires a longer attention span than two months, you should probably break it down and deliver something of value earler on. As you say yourself: "the market won’t wait for you to implement them".
To be fair, I've only seen Scrum. But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
What a nice way of motivating people. Every person in a good team is of big value, and cannot easily be replaced by anyone else. Creating a good team requires thought and planning. If Scrum is about diluting the individual, then Scrum is far more wrong than I originally thought.
This is just wishful-thinking, the Agile Point. The only remotely close way of doing this is turning everyone into cogs in a machine. No, thanks. Weren't we valuing the individual?
When you motivate people, and give them tasks they like, they tend to perform their best. Turning people into cogs in a machine, as Scrum seems to like, is not a good way of motivating them. As the manifesto says: value the individual and motivate them, and they will get the job done.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
And that is why you don’t care about sprints. You do one task, preferably as small as possible, and deliver it when it is done. That’s it. I have never understood why I should spend loads of time planning what I should do the next N weeks, when I can just do it instead.
Some of the most successful products and projects (Linux, Python, Apple) revolve around strong leadership.
Having strong leaders around a self organizing team is not necessarily problematic. Self organizing teams are more about giving the team freedom to work the way they want. A manager or leader can then contribute by removing obstacles and ensure the team has everything it needs instead of micromanaging everything.
The most important part of The Agile Manifesto, in my opinion, is "Individuals and interactions over processes and tools". If you care more about the process than you do of the people in the team, you are doing it wrong.
EDIT: formatting.
Everyone has deadlines. Either customer imposed ones, or the market running against you. At some point, you'll have to meet them. When do you stop accepting changes and deliver? Can you welcome last minute disruptive changes? Sure. But the market won't wait for you to implement them.
There is no point in delivering software which doesn't meet customer requirements. If changes to software are required because it doesn't meet requirements, pretending otherwise solves nothing. Agile processes make it easier to cope when these changes happen, and accept that if change is required then other constraints have to move – such as deadlines.
Some tasks require longer attention spans. This is the classic technical debt accumulation choke point. Now, iterations + deadlines make sense. But does it have to be a fixed 2/3 week period, always? Where the flexibility in that?
Most tasks do not require 'longer attention spans', whatever that means. Almost literally every task can be broken into smaller parts with minimal effort, helping to expose implementations to scrutiny at earlier stages.
And while Scrum traditionally has a fixed sprint length, there's no requirement to follow that to the letter. If your constraints are such that it doesn't work, then adjust the process to suit.
But this point is in direct contradiction. Scrum is about diluting the individual. Everyone is a replaceable resource.
No – Scrum is about the team of individuals. Depending on a single individual is obviously stupid regardless of what process you use.
The only remotely close way of doing this is turning everyone into cogs in a machine.
Not at all true. A nice sustainable pace is reasonable for any team of skilled professionals. If you are not achieving it, your team is broken.
And takes time. Ops, sprint's time is over. I guess that refactoring can wait.
And pretending otherwise does nothing. Make refactoring a requirement for completion, and you will avoid this problem.
If the best argument they can come up with for the process is "because that's how agile development is done" it's because they don't actually understand the process but that doesn't necessarily have anything to do with them being certified or not.
That's my point. They paid money to get "certificated", get jobs as project managers or scrum masters but have no idea how development or agile actually works. I've have come across a lot of these types of people.
(And, in case you're interested, "certified" is the standard term. In case you're not interested, my apologies for bringing it up.)
Scrum is actually used to make sure the dev team is part of the decision process and there is no reason not to be innovative But of course the enterprise can kill all things good
For the way we do it, when a stakeholder wants our team to do something, they come to us with user stories and acceptance criteria that matter to them. Then we hash out the rest of it. I don’t really see a useful way we could be trusted more. The Marketing department knows best what feature it is they want on the website, and why. So they specify that part. We estimate complexity and the business weighs the priority of that feature against other stuff in our backlog and it falls into an appropriate sprint based on that and other motivating factors. Then we do the work.
I don’t see a lack of trust? So I’m curious where you see it and feel it should be done better?
But outside of a very small company to me it’s a bigger crime to waste engineer time with all the meetings necessary to give them that full perspective. When I started at my current company I was the only developer and I worked directly with the CEO and our single marketing person. Back then I helped decide priority. Now the company is too big for that. I don’t have time for all the manager and marketing and business planning meetings to be able to decide priority.
As far as defining features, we can propose them like anybody else but I think I already explained why usually it makes sense for them to come from whoever owns that piece of what you’re doing. But this of course again depends on the structure of your company.
> Trust your engineers to do the right thing on their own and they might just surprise you.
Yes, they might surprise you. That's potentially quite terrible because there's a whole ton of other people & roles involved in the success of a product... and of course that's assuming that the project is what the customer wants, requires no marketing message for the customer to understand, and sells itself.
I mean - trust but verify. While I largely agree with you it's always demotivating for the team to realize one dev is doing no actual work and getting away with it. Obviously not good for the business either.
For example, the OP argues how scrum strips engineers the autonomy they need to get the job done and how it's super inflexible. When we did scrum with my team, the scrum master role was rotated among engineers, growing each of us significantly as professionals. We also made changes to the "core" of scrum after our retrospectives when we decided it was stuff that did not work for us. When management tried to use story points for tracking or forecasting we told them to forget about the whole thing. We added bugfixing sprints after we observed the bugs piling up.
I can go on - we made scrum work in an environment where we were empowered to do what makes sense. The crux of the post seems to be that OP has some very rigid management and devs cannot change the mindset.
Instead of complaining about scrum, I suggest the OP either try changing the culture from the team level. Or if it won't work, just get out and work at a place where there is sane management who does not drink the rigid scrum kool-aid. Luckily, there are more and more places like that these days.
There isn't enough evidence-based science on what produces the best software -- but Scrum can be measured against other software practices, and should be.
I must have heard this at least a dozen times in the last months. OOP isn't good or bad, Exceptions aren't good or bad, Scrum isn't good or bad. It makes me mad. Guys and girls! Software is not a contest in being liberal and tolerant. There are methods and techniques that are objectively better than others in reaching most of the goals that are important. You wouldn't contest a F1 driver on a bicycle - not on a race track. And engineers who really know their stuff usually agree on how to build software in a given domain. I do not know of any success stories using Exceptions on Kernels or RDBMSs or even most server software. But I know people have tried, and even they agree it was a bad idea.
If you want to say Scrum isn't good or bad, depending on how you tweak it so it sucks less, then I say it's definitely bad. It's not a solid method that helps you reliably reach your goals.
If you want to say Scrum isn't good or bad unless metrics are defined (that doesn't seem to be the case here), then you are right, but it's a meaningless statement.
It's all true though. Do whatever you want, all that matters is that you do it well - you just can't simplify a good software engineer down to techniques like these.
I recommend reading this piece from a guy who studied software development methodology for over 20 years: http://alistair.cockburn.us/Characterizing+people+as+non-lin...
Using only few features might slow development a little in the small, and you might even make more errors in the small while developping. But you can FIX them.
I realize this is not hard science, and I have no motivation to do any on it, like collecting and bending statistics. But as long as all people have to say "XY is not good or bad", "things will get better", "XY has just a very steep learning curve", "you just didn't dance the right dance" and all evidence seems to suggest otherwise, then you've just lost the argument. End of story.
When you need partial updates to state in the case of an error, then you're in trouble. When you have multiple transactions to coordinate, but no distributed transaction coordinator, you're in trouble (but you are anyway whether you use exceptions or not). The more state you need to preserve in an error case, the worse exceptions are for your use case. Ideally, exceptions are never caught except at the topmost loop: server request dispatcher, UI event handler, or command line driver. When execution must continue with recovery, exceptions are an inferior tool.
And not just simple programs and short scripts; very complex programs, as long as they keep state in a transactional store, and a request has approximately the same lifetime as the transaction.
Which likely doesn't even try to handle any errors, and may actually be a bad, faulty program - the edge cases just don't matter that much for businesses. Maybe an e-mail will be sent out twice in a few circumstances, etc: nobody cares.
Transaction management is implemented in the database server, and very likely that doesn't employ exceptions. And more complexity is implemented by the OS (the "process server" if you will) - if your program dies it specifies how to handle the condition, etc. Another part of complexity is dealt with by the language runtime, for example garbage collection - no exceptions to be seen there.
The thing that serves responses to requests is a program. And it's not just a "simple, dirty request handler script," it's often enormously complex. Entire companies exist with their product exposed via HTTP APIs.
The reason you don't see exceptions in databases, operating systems, etc is because they're often written in C++, which has notoriously wonky semantics around exceptions, or C, which doesn't have exceptions at all.
We standardize around protocols, frameworks, programming languages, etc because adding constraints and processes make a problem much easier to solve.
If not scrum, what? Where do you start?
There are plenty of methodologies and tools that can replace or complement Scrum.
>This paper outlines the work I reviewed that made it clear to me that ‘people’ really are a first-order, and not incidental, project success driver, the characteristics I currently use as predictors.
My personal experience is similar - methodologies can make things incrementally better for a team, but cannot do anything with the ‘people’ factor.
Oh please. Dogmatists make the worst software developers. The "objectively better" criteria is highly situational and not nearly as clear cut as it looks most of the time.
There are plenty of methods and techniques in software that work well for some goals and absolutely suck in others and it's very often not obvious that something that is working well or badly for you is doing so mostly because of your situation.
My favorite example of this is microservices - which apparently worked really well for Martin Fowler once or twice (and I'm 90% sure he didn't understand why) and then when he promoted it, it got applied dogmatically by teams all over the world in situations where it was manifestly an idiotic idea.
The same goes for most hot-button issues - type systems, functional programming, nosql, etc.
And so it is with most other hyped technologies. The complexity and loss of control is very rarely worth it. I'm not a dogmatist, but have seriously tried many different approaches. And that's why I do mostly C (well, not for web applications, which I try to avoid as well :>) with only very simple data structures, and I love it, and it works, and it's maintainable, modular, and very performant. Although I know that C has flaws - I have been trying to find ways to improve (and failed so far).
woosh
They work really well in large organizational environments where inter-team coordination costs are very high and language flexibility is valuable. Where these benefits outweigh the additional infrastructure headaches it works very well.
I've likewise worked in environments where they were a terrible idea (e.g. startup team of six).
Like you, I used to be dogmatically anti-microservice. I was wrong.
It’s ok to only want to do certain kinds of work (for example C code that runs close to the metal and that is performant and easy to reason about). Other people in the world though enjoy doing different kinds of work and going about it in different ways.
My advice to you is to be humble and watch people as if they had something to teach you. Whether they are doing scrum, waterfall, using this or that language or architecture, there’s always a ton of things you can learn by watching how someone works.
If you pay closer attention, you will see value in what other people are doing and be able to benefit from what they have discovered.
I'd still be suspicious of people who start out saying they're going to implement a microservice architecture (especially greenfield) without having an understanding of why, though.
I've almost always found that the earlier on in the process I made hard-to-reverse architectural decisions, the worse the architecture ended up being. The more the architecture came about as a series of gradual, iterative refactors that took in to account the various trade offs that became evident, the better it ended up being.
But that's just a fucking picture. A plan. That usually does not survive contact with the changing requirements and implementation reality.
Anyone incapable of admitting that there will be changes should be taken upstate to a nice farm, then shot behind the barn. Usually these people are on the client's team. :(
The thing about any agile methodology is that they are meant to be adapted to the situation at hand, not to be a fixed set of rules that you can just apply to get consistently good results. But many are trying to turn scrum into that - a fixed set of rules that allows micromanagement. I would say that that is definitely bad, but, is that really scrum?
I think you need to get out more.
Exceptions are actually pretty standard in the database world. ScyllaDB, arguably the top dog in terms of DB performance, has the "throw" keyword strewn throughout its code base. Oracle PL/SQL's error handling is done entirely through exceptions...
Is this about the database engine itself? Or just the query parser? Or even just exceptions in the query language itself?
FWIW, Scylla is not an RDBMS. I don't know about its architecture and am not interested in digging more. And that thing works reliably? I don't know - not my cup of tea. Why am I reminded of Node.js? Not cherry-picking here...
In the 60s or so, people came up with this thing called scheduler. It allowed them to run things (virtually) at the same time, while not needing to make a mess of their code... They also invented cooperative multitasking, where the scheduler only kicks in when explicitly called.BTW, Exceptions and OOP are used conceptually in most kernels, RDBMS, and any bigger piece of software, it just doesn't make sense to use languages which force their use if a project has too many edge cases.
It's not the company. This inflexibility is a reflection of scrum's culture. I've seen this inflexibility on multiple teams across the world - what you've experienced is very much the exception and what the OP described is basically the norm.
>We also made changes to the "core" of scrum after our retrospectives when we decided it was stuff that did not work for us.
This is great and it's exactly how I'd want to work but I think almost every single scrum consultancy not only advocate strongly against doing this, they would likely blame anything that went wrong on your "rash" decision to not adhere to core scrum principles.
They literally invented a whole new phrase to emphasize just how much they look down on what you do: https://www.scrum.org/resources/what-scrumbut
>We added bugfixing sprints after we observed the bugs piling up.
This is actually something I've heard explicitly called out by consultancies.
>Instead of complaining about scrum, I suggest the OP either try changing the culture
I don't agree. I suggest slagging scrum off until it dies, because the inflexibility which you decry is so deeply embedded in its culture and will only die when scrum does.
Sure, what you did worked great - it just wasn't "scrum".
Well, it's either good, bad or ho-hum. If it's bad or ho-hum we shouldn't be doing it. If it's good we should be doing more of it.
Now, it's true that a technique can be good but people practice it badly.
But first we need to know positively that the technique is good -- before we insist people follow it more carefully.
So saying "X isn't food or bad, it's the way you use it" don't help.
Saying "X is good, but you're using it wrong" would indeed be helpful (if true).
Engineers are as responsible as any other actor for poor Scrum implementations.
Scrum, or any agile method, isn't a silver bullet. It doesn't compensate for bad teams. It's something that teams can use to deal with some common software development issues. It's also not the only tool that can be used for that purpose. Developers pouring ire on "scrum" when they aren't practicing it in the first place are attacking the wrong target.
Products with heavy debt tend to get rewritten rather than fixed. So it makes sense to keep the schedule through shortcuts and push as many features as possible. Then throwaway the product and rewrite in something newer and more modern.
For developers the benefit is your resume will have the latest tech and you can write sloppy non-abstracted code and not worrying about having to support the product later. Management gets increased visibility into each task with an accurately followed schedule.
Not a surprise it has become so popular. If I could change one thing I would make the scrum master in charge of technical documentation. That one change would force the master into understanding the code at a lower level. That would make them the most knowledgable person on the project thus it would remove the need for daily standups because team members could go to the master at any point in the day with questions or to coordinate effort.
The solution is not to blame whatever particular form of agile you’re using; the solution is to replace the bad managers if you can’t educate them on why that’s short sighted.
You don't need everything to be "inside the project/inside the process" and even then, more things fit there than people might think.
Scrum isn't perfect but this is certainly not true. If the team is well-functioning there doesn't need to be a facilitator of any kind present. Sounds like OP is either in a bad team or with a bad Scrum master—or both.
This is a tautology.
My main complaint about scrum was how much of my time was spent in meetings vs working.
Thankfully I dont do agile any more, but I'm close to several teams that do. My fondest memories are:
No over-arching design (Implement feature after feature however the hell anyone likes) Once said features are implemented, taking the time to "refactor" (read: completely rewrite because the code was so bad) becomes a really hard sell.
Thus, new features get progressively harder to implement, as you touch a deeper cross-section of the code to try and fix stuff semi-covertly, without a formal refactor.
"Tech Debt" was done in sprint scheduled after phase of dev sprints. After, say, a year of development, lets fix all the problems in 3 weeks... When you're dealing with procedural php masquerading as OO, MVC with business logic everywhere it shouldnt be, and abuse of inheritance such that you cant throw an error in a command line because you're not a logged in browser user, who's had their language queried from a database, and doesnt fall back to default when the db isnt there... 3 weeks isnt gonna cut it!
Introducing peer review was supposed to sort that, but bad or inexperienced devs didnt know they were doing shoddy work (it met the criteria after all..), and good devs became impediments because so many stories needed redoing. Demoralising.
We were probably "doing agile wrong". But it's like a religion: open to interpretation. In my experience it's a tool used to extract maximum customer benefit at the expense of the developer's soul, under the pretence of "self managing team" - ie. "You brought it on yourself".
Feeding that back through the process is pointless when it inflates the complexity of future stories, because they are less likely to be prioritised, and the rot festers.
Peer review is a good way to improve the quality of peers, either though the natural education/knowledge sharing, or simply the transparency that helps you remove low quality peers and find better ones. It's not a cure all, but it's hard to imagine a successful team that wouldn't benefit form it.
Some turn up in the morning, however, work on their task for the day and go home. If they reimplement a chunk of the codebase somewhere else, because they feel they need to be writing more code to complete their task, fine, the story still passes. "I didn't know that was there" is not an excuse, it was probably covered in planning but they forgot. It's not going to get refactored, because it works! Peer review in this case goes one of two ways:
1. Bad quality peer sees criteria are satisfied and approves 2. Good quality peer sees all the duplication, some other bad practice thrown in for good measure and disapproves
In 2. say the story has already carried, pushing more work back, and hard deadlines are looming. They will be forced to begrudgingly approve the work, knowing they've just allowed a good chunk of cruft to enter the codebase. They will try and educate (assuming they've not lost the will to live - this might be the 100th time for eg), to prevent it happening again. But it will still happen, because bad developer is bad.
Forgetting the looming hard deadline: If the story carries over so what? We'll retro and add some bloat to the process to try and avoid it happening again.
What isn't explicitly said is "this dev is committing crap and the competent people are getting demoralised", it's the process that gets "refined" not the people. Agile is papering over the cracks and nurturing the codemonkey's behaviour.
But none of this matters, as the deliverable has been delivered. Now on to the next story!
But the organisation is slowly carving our monolith into microservices (Because buzzwords will save the day! <sigh>). I do feel microservices lend themselves better to agile, as the programmatical boundaries a good dev should naturally code within become rigidly defined. And people can effectively do their own thing within their "box".
IMHO it shifts the problems to the layer above, papering over the cracks. (Badly written code performing badly? Whack it behind a gateway and spin up some more containers!)
Even with microservices you're gonna have to define that API at some point (and now we can have as many as we like behind the scenes!), why not at the start with a holistic view... like, a design!
"As a user I can see a login page" - ok here's a static html page with two input boxes and a submit button. "As a user I can log in" - implement that slice of the backend, all in one hit with no eye on what other things might need from it (Dont get me started on YAGNI... YMNI, imho and you'll miss it when it isn't there! Especially when there's "no time" because it's not a feature with perceived customer benefit which take priority) And it goes from there. What I would prefer is more like "before we can even think about showing a login page to a user we should have..." which is more bottom up, and doesn't look good to a customer (or management).
This should be a hard sell. The code in production works. Why does it in need a rewrite? The fact it needed a rewrite in the first place doesn't really reflect on scrum but on poor engineering and management practices.
This might be what I have the biggest problem with. To a customer, it works. The customer is happy, they pay their money, which makes management happy, and we all get paid. So the site takes 10 seconds to load on certain pages, who cares! It works, right?
Behind the scenes, there's a fleet of web servers burning CPU so add some more! A database that's overwhelmed, throw some more CPU/mem and faster disks at it! Maybe optimise some queries if you can get it into sprint. But a fundamental problem with the implementation itself cant be fixed, because "it works". Want to leverage some slave dbs for reads? Sorry, no time for that because unpicking the backend would take too long!
It's backwards. In my idealised view: Good design first = better code = less resource hungry = less cost for servers = support more users = more revenue = less time spent working around crap = more time to implement features! = more revenue = better wages. Everyone wins.
Imho, agile doesn't help make this happen.
If you really have a fleet of web servers burning CPU and you arne't tracking the costs, then I guess "it works" until someone figures out there isn't any money left in the coffer.
There is a lot of value in working software, and unless your engineering infrastructure expenses seriously outweigh your engineering payroll costs (which does happen, but is hardly commonplace), CPU efficiency isn't the be-all to end-all and focusing on it may well prove to be counter productive. If no one actually cares about your page load times, it may very well be that it doesn't matter. You can spend a near infinite amount of time & resources trying to get optimal performance (talk to someone who does high-frequency trading to get a sense) well past the point where the costs pay off.
There is tremendous value in "it works" that engineers generally struggle to fully factor into their thinking (I come by this wisdom honestly). People have written about this extensively (e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-...). That said... the notion that you can't fix actually serious and counter-productive problems with the product because "it works" sounds not only counter to Scrum, but really counter to any software development methodology I've ever heard of.
What if the costs are fixed? (own servers, unmetered bandwidth) The problem shifts down the line when you want to add a new product and the first product is consuming all your resources. Refactoring then becomes a necessity, but there's no capacity for it because the company is focused on shiny product #2, while also adding shiny features to product #1. Now fire a few people to save money, leaving the remainder short handed, who get stressed and demoralised. But that's ok, because "it works!". Why cant we design first, lay down a good foundation, then add features. The business makes more money, people keep their jobs, instead of spit-balling "user stories" at an application which becomes a giant spit-ball.
> CPU efficiency isn't the be-all to end-all and focusing on it may well prove to be counter productive > You can spend a near infinite amount of time & resources trying to get optimal performance
I used CPU as a broad example. What happened in my case was there was no performance. At release it was unusable. What stuck in my side was I knew it was going to happen, and had been calling for refactors. But if you cant get them prioritised because you cant justify the customer benefit (it's not released yet, so nothing has fallen over yet) vs features! features! features! it then becomes a bodge job /after/ release. eg. Instead of "refactor this code to make it do fewer queries" it's "lets shove the results of these queries into a memory cache." This satisfies agile, you've iterated and made things "better" - and it fit into a 2 week sprint! - but it feels backwards.
I was going to go on and talk about quantity vs quality, but a quick fact-check google revealed the evangelists with their "Agile parables" (it even sounds cultish), which are in no way perpetuating Agile to sell support tools, say I'm wrong.
I need to thank you for making me realise it's time for a career change. After all this time Agile has failed to convince me, I'm inclined to conclude it never will. But should the next methodology of the moment come along that emphasises quality over quantity, I reserve the right to shout "I told you so!"
Again, your problem isn't coming from the methodology. The priority on quality comes from the context, not the methodology. I have a friend who does presentations on agile all the time and his team prioritizes quality pretty much over anything simply because it allows them to develop faster.
Quality got worse.
In this scenario it's the fault of agile, as the reason for doing agile was to increase quality.
It's a tool/methodology that can help you increase quality.
If quality isn't a priority, a methodology that helps to better align your work with your priorities isn't going to improve quality.
Can I ask why do you deliver stories before refactoring, if that's what you're saying? We'd tend to do necessary refactors as part of features or bugs when we touch related code and rarely have pure refactor chores.
Also, why do you have to _sell_ code quality? At least where I work there is a pretty strong divide between what to build and how to build it. There is two-way trust between the product manager and engineers that they know their domain and when they say "this is important right now", it's probably the right thing. After that it's just a matter of communication and prioritisation between the the two sides.
I could easily see how this might fall apart if either side was not trusted or not doing a great job, but I guess it would probably fall apart no matter what development process you followed if that were true.
> 8. Scrum is typically implemented with HORRIBLE task management tools (Jira, tfs, etc…) that enforce very bureaucratic interpretations of Scrum that waste huge amounts of developer time. Additionally, they effectively lock you into one mode of operation, no matter how ineffective.
Scrum is not just management heavy. It's micromanagement heavy. If someone asked me for an account of what I did for a whole six (assumed to be productive) hours yesterday and how many story points have been burned, and wanted this every single day, I can only imagine that the person has no experience with software development whatsoever. There are productive days when a lot of stuff gets done and there are unproductive days when almost nothing gets done because it's hard to focus or because of distractions. But the daily standup meeting that's enforced would make any unproductive days as something to get publicly shamed about.
On the tools, yes, whatever I've seen in Scrum dictates a lot of time spent updating the tools with estimated hours, hours spent, remaining hours, everyday. That data is rarely useful for any further estimations or to figure out what's going on.
Lastly, the "story points not related to effort but related to effort" conundrum confuses developers (especially the ones who are considered to follow instructions and code whatever the user story's tasks say). Add to this a really weird concept called velocity, which is not just a measure of how many story points are completed in a sprint, but is used as a range — too low and you're not working hard enough; too high and your estimates are wrong. The only way to satisfy the Scrum master in many cases is to lie and make the velocity conform to expectations.
I have no idea who was tying to prove what to whom when they came up with Scrum and Scrum certifications. The certified Scrum masters I've seen don't even know how to write a good user story or to break it into tasks.
Give me low process overhead iterative waterfall any day over these schemes. Repeatedly trying to turn software developers into predictable manufacturing style widget makers is not a good idea.
If someone asked me for an account of what I did for a whole six (assumed to be productive) hours yesterday and how many story points have been burned, and wanted this every single day, I can only imagine that the person has no experience with software development whatsoever.
This is wrong. If this is happening, you are following a broken process.
On the tools, yes, whatever I've seen in Scrum dictates a lot of time spent updating the tools with estimated hours, hours spent, remaining hours, everyday.
No it doesn't. This is wrong and broken.
Lastly, the "story points not related to effort but related to effort" conundrum confuses developers
If a software engineer can't immediately grasp this abstraction then they are not good software developers.
too low and you're not working hard enough; too high and your estimates are wrong
Velocity should not be used as a stick. It's a measure of the amount of work being done which can be used for planning. If it fluctuates, that can indicate problems that need to be dealt with.
Give me low process overhead iterative waterfall any day over these schemes. Repeatedly trying to turn software developers into predictable manufacturing style widget makers is not a good idea.
"Low process iterative waterfall" is basically Scrum.
I work two-week sprints. There are a total of four hours of meetings for those two weeks. 5% overhead seems pretty reasonable.
I know “low process overhead iterative waterfall” sounds agile, but somehow the word “Scrum” seems to make people want a certified Scrum Master (these people usually frown on waterfall-like processes even if those are better suited for delivery).
But the first few and final example are good warnings that can be applied anywhere, Scrum or not. Paraphrasing / quoting the article:
"[If] all product decision authority rests with the “Product Owner” ... engineers [are disallowed] from making any product decisions and reduces them to grovelling to product management for any level of inclusion in product direction."
"[If] all the engineer’s time [is accounted for] in a tightly managed fashion, [that] discourages innovation — which typically occurs spontaneously and outside of any schedule or system of good predictability."
"[You] encourage “least amount of work possible” solutions [when you expect engineers] to conform to ... strict predictability requirements."
"[If you] divide every task into small items that can theoretically be completed by anyone on the team .. [you] discourage engineers from taking pride in and/or ownership of their work. This lack of ownership results in ... Poor design [and] Lack of motivation ('It isn’t my thing', 'It was broken when I start working on it')"
"[Don't] ignore the fact that any task that has been done before in software does not need to be redone because it can be easily copied and reused. So, by definition, new software tasks are truly new territory and therefore very hard to estimate."
Detail how it needs to look, how it needs to scale ... Ill decide how we make that happen and Ill tell you how long its going to take.
Its not my job to decide the product, if you need an engineers input maybe you are not a good product manager.
I dont need any one outside of engineering to help make implementation decisions.
I just want to code and go home. Stop it with this kum by ya , 20% , everyone gets a say bs. Are you running a business or a daycare?
They should seek external help in getting their usage of these simple tools back on track.
I’ve seen people create horrible abominations using Excel, that doesn’t mean that the tool itself is broken or that all the effective usages of Excel are wrong. It’s the same with Scrum. If you use it right it’s awesome, if you use it wrong it’s just a different name for the same old bad project management you always had.