I agree that almost all of the orgs I encountered that were Doing Agile were pretty horrible.
On the other hand, I've also been in orgs that were very successful in developing software in a way that we would recognise as being agile. We just didn't make a big deal out of it. And we didn't do standups (most of the time), or 2 week sprints, or retrospectives. We didn't even pair consistently: we split up on trivial stuff, paired on hard stuff, as needed and available.
In one org we started without TDD, and even without having much in the way of automated tests at all. But when we discovered testing and TDD, boy did we ever embrace it. Not because anyone told us to (mid-to-late 90s, so no Agile Manifesto), but because the difference for us developers was night and day.
So develop software. Concentrate on what helps you develop software. In my experience, you will probably discover something recognisably agile all by yourself. So when you discover that, help yourself to what helps you. Don't make a big deal out of it.
Sounds a lot like TPS, I assume Toyota did a lot of the stuff before any books were written. Also, my personal experience, the more companies talk about Lean and Six Sigma the less they do it.
Similar with "culture". Companies with a great culture don't talk much about it, they just get shit done without much fuss and typically have some fun doing it. Because getting shit done is fun.
On the other hand, if the company all about how fantastic the culture is...run! Or at least tread very, very carefully.
Totally agree. Talking about culture or hanging up "inspirational" posters is usually a warning sign. Good organizations live their values and don't feel a need to constantly talk about them.
What you are describing is exactly what Agile should be. You have a team of people who care about the craft and they slowly improve processes based on real world feedback and experience. If something doesn't work you drop it and try something else. This happens in environments where people respect each other.
Management does not want to hear that software development is a craft. They want it to be a trade, with fungible workers operating in a predictable process with fungible components. The great tragedy of Agile is that it was co-opted as the vehicle of this transformation. It was seized by management, weaponized to other purposes, and then aimed right back at us.
Yeah, I've come to similar conclusions about Agile. It can be a great way for an already well-disciplined organization to think about the work they're doing. But many, many undisciplined organizations thought that Agile would be a catalyst for them to become disciplined. But the nitty gritty of "doing Agile" required EVEN MORE discipline than what all these organizations were already capable of exercising, so it just highlights all the frustrations everyone already has.
That does seem to be the downside of Agile. It's a collection of maybe a dozen different techniques and practices. But if one of those practices falters, whether it's the TDD, or the business side still wants a certain deadline, or you don't bother to demo at the end of a sprint, then the whole house of cards falls like dominoes..... checkmate!
You also need to give people the chance to make mistakes and learn from them. I don’t envy a lot of the young developers who get micromanaged all the time and never have a chance to try something.
Am tired, just dead bones tired of all of it. It's all just this or that money making scheme with guys like me caught in the middle. I despised Waterfall, fell into scrum and in the past ten or so years kinda sorta saw one project where agile basically worked right. The rest has been mostly waste. I've worked on billion dollar systems on down to just little nothings. Doesn't matter, companies are struggling to make anything work. I think there's just too much complexity and they all believe IT solutions can manage that complexity. It's failing like never before while expectations have never been higher.
Message to the Fortune 500: Just get rid of anyone with the title "scrum master" they're dead weight. You were fooled, deal with it. I am trying to move my team over to Kanban right now, but we have all this reporting crap up the chain all designed around CA Rally, the worst productivity tool ever made.
Got any insight as to why TDD is such a great thing?
From how Ive heard it described it seems to be a way for management to impose rules on how developers can approach writing a feature and not necessarily a benefit to the devs.
If I understand it properly, it's essentially write a test so it fails then fill in the feature gap. The whole write first bit is just so you don't have to retest later to verify that your test didn't hit a false positive, right? Something a careful person would check anyways...
TDD was discovered as technique by developers, for their own benefit. Management imposing it can be a bit of an anti-pattern, though it may be well-intentioned. I have certainly pushed teams to TDD, though primarily by example and by the positive experience you quickly get.
Anyway, TDD has nothing to do with management imposition. It is really diametrically opposed, at most orthogonal.
2. "test didn't hit a false positive"
Yes, that's a good thing, though hardly the main point. You write that a careful person does this anyway. I'd rather have a machine do that for me, and have less of a requirement on me to be careful. Computers are much better at being careful, and I want them to do the tedious stuff.
I also am wary of claims of "you just need to be careful". Probably lots of bugs ahead!
3. So why is TDD so great?
First, it's a design technique. Writing the test first forces you to think about how your new production code will be called. It also forces you to be clear about what the code is supposed to do, precisely, before you write the code. Writing the test tends to be significantly less difficult than writing (correct) code to solve the problem. You then write only the minimal code that is required to pass the test, which surprisingly also tends to be fairly trivial. Also, you know when you're done: when the tests are green again.
So you've transformed one difficult step into two fairly trivial steps. Which sounds weirdly magical, but actually works in practice. Also, since you only wrote the minimal code that was required for the tests to pass, you know that you have good test coverage.
After you've done a few, you're in a place to detect duplication and eliminate it by refactoring. This tends to be somewhat more intellectually challenging. However, you are truly refactoring, that is not changing the functionality, so you have a great safety net with the good test coverage you achieved before.
So you get this small steps that are fun and easy, and you tend to not regress.
Psychologically, it's also very pleasant, compared to the usual flow, which goes somewhat as follows:
1. I have a rough understanding of the problem,
2. I code a solution, which is at the limits of my understanding
(because everything is up in the air)
This part feels very good, "I am the master of the universe"
3. Then I should write some tests, but the incentives are a bit negative
-> if the tests all pass, nothing happens, so why did I write the tests?
also if nothing happens, how do I know I actually tested something?
(not by "being careful")
-> if the tests do show a failure, it's a distinct downer from step 2
(and so I probably subconsciously write my test so they don't fail)
4. I don't really want to refactor, because my solution from step 2 was already awesome!
TDD:
1. I write a test. It fails appropriately. That makes me happy.
2. I write minimal code. Tests go green. That also makes me happy.
3. I refactor. The code now is much nicer and tests still green. More happiness.
So with most normal development flows you get that initial rush and then a lot of downers. With TDD, you don't get quite the same initial excitement, but instead you get these nice regular dopamine hits without really much downside.
The best way I've seen agile described is "designing code in a way that acknowledges change.' I think there's a lot of value in understanding that you may have to go back to the drawing board when designing parts of your program. I have little regard for the entire industry that has risen up around agile, however.
You don’t hear the name Joel Spolsky much any more, but he was pretty influential in software process thinking in the 90’s - not really for being particularly insightful or original, but more because he was one of the first people who thought of writing a blog about software design. One of his early “observations” about software project management was that “you wouldn’t buy a pair of jeans without knowing how much they were going to cost, why would you expect your manager to sign off on a software project without knowing how much it was going to cost?” The utter blithering stupidity of this perspective highlights how managers look at software design: the purpose of software methodology is to put a fixed cost on every software project. As long as that’s what they’re looking for, software methodology is going to continue to be a lost cause.
Nothing, inherently, except perhaps the fallacy of comparing a pair of jeans to the rather chaotic and unpredictable world of bespoke systems development. One is inherently known (a pair of jeans you've presumably already manufactured), the other is one big unknown, basically.
He compared to jeans because they're an extremely trivial purchase that someone would give little thought to, but still demand to know what they cost. The point is that in all cases you need that information, with a major software project you simply need it far more.
And that's the core of the problem (which, ironically in the context of the article, agile methodologies were supposed to address): wanting or needing something to be true/exist doesn't make it so.
I think maybe the point they're trying to make is that comparing a software development project that may include a number of unknowns to buying a pair of jeans where there are probably very few if any unknowns is fallacious and a waste of time.
That said, I don't think it's black and white. Some projects, making a simple web site that follows whatever Squarespace template is popular and filling it with some relevant content can be a pretty simple project, even to the point where the buying a pair of jeans comparison makes more sense.
The problem, I suppose, is when you believe strongly that every project can be accurately priced up front. Not every project is a moonshot, but some truly are and it's not always obvious up front which ones are.
And every engineer likes to pretend that their project is a moonshot.
If you are inventing something incredible,why are you someone's employee instead of getting capital investment or self-financing from your last achievements?
"Sweet, easy enough, I'll just use X, Y and Z, no problem."
"Oh but sorry you have to use A, B, and C instead. Oh and it'll need to connect to D for reporting, and we'd like it to sync with E which we've never done but they have an API so it can't be too hard, right? Oh and it needs to fit this design we've already approved that's about as far from native to the platform you're developing for as it could be and has a ton of UX issues that require fundamental changes to fix, which you'll uncover as you work. That's about as easy, right?"
"...."
[EDIT] oh and also one "minor" feature we're going to toss in later is actually big enough to build an entire company around so you'll have to waste a bunch of time talking us down from that while we quietly lower our opinion of you, and we've accidentally described a few things which are basically impossible by egregiously violating e.g. CAP theorem or being what's effectively a highly-available distributed filesystem that needs to fit without our timeframe and budget, which is 1/20 what it would take to maybe build that and that alone.
Writing software is design, not manufacturing [1]. Do companies know accurately in advance how much it will cost to design a new nuclear power plant, aircraft carrier, or jet engine?
There are times when something truly original is being built and it's a valid argument for estimate uncertainty. But a significant portion of the industry is engaged in building CRUD app #237 or Ho-Hum SaaS #17, and a significant percentage of estimates end up wrong because some developer who was bored with his job decided to use a blingy new Javascript framework he had no experience with because he wanted a challenge. Money gets lit on fire again and again this way and it's frankly selfish and irresponsible.
If the software is truly that unoriginal, use something off the shelf. The issue is that in literally every non-toy product I've every been involved in the product owners will always turn CRUD app #237 or Ho-Hum SaaS #17 into something significantly more custom and much more complex.
>because some developer who was bored with his job decided to use a blingy new Javascript framework he had no experience with because he wanted a challenge.
That definitely happens, but I think it's a different problem.
> The issue is that in literally every non-toy product I've every been involved in the product owners will always turn CRUD app #237 or Ho-Hum SaaS #17 into something significantly more custom and much more complex.
YES! The cost of custom-everything design and bespoke gee-whiz is vastly under-appreciated, IMO. CRUDy Android and iOS apps using built-in widgets and simple color theming? Relatively easy to estimate, and pretty damn fast. Custom-everything, we want both to look pretty similar (so, probably Material-ish given current trends, which you'd think would come for free at least on Android but... doesn't), custom animations everywhere, oh I hate that default date picker on this particular Samsung phone (and sure, it's god-awful) can we customize it, and the designer came up with this layout for this form that requires all kinds of twisty-bendy manipulation to reproduce versus something more straightforward but that's what the "stakeholders" approved so let's do that.
Et c. et c. and pretty soon the app's 10x as expensive (no exaggeration!) because you couldn't live with the easiest-for-the-user-anyway default styles with some custom coloring.
And the Web's at least as bad. Often you're also making your page way less usable and breaking it on certain browser/device combos due to the extra effort you put in to make it "pretty", at great cost.
[EDIT] in fact I've in the past suggested providing design points for "pretty gee-whiz versus not-actually-bad straightforward implementation" to make product owners choose between getting e.g. three normal (and OK-looking but not "pretty") features or one pretty feature in a sprint, to make this cost more visible and give owners on a budget more flexibility in shipping features, but it never caught on—in particular designers hate, hate, hate the idea, I think because it might expose that their work is sometimes (often...) more expensive than it's worth (and vastly more expensive than they're being paid, since it also eats huge amounts of developer time) given you've got some developers with even a hint of aesthetic and UX sense on a project.
I wonder if you looked at time estimates from developers of common process applications (like CRUD) in say, the 1980s, who used COBOL or FORTRAN, vs say those using Pascal or C - if those developers of code using extremely well known processes and libraries had better estimates than those using the "newer bling"?
"Yeah, yeah. It doesn't have to be perfect analysis, just a rough idea for scheduling. Not written in stone, ha ha."
"Uh, a week?"
"Okay, great, so if I say 8 days you should have it done by then?"
"Yeah, I hope so."
"Great, thanks."
Day 3: actually doing X requires unforeseen Y and Z which will each take a month.
"We need to add Y and Z to the schedule, which will each take a month."
"There's no room for that in the schedule."
"Then I can't do X"
"We've already costed X and committed to it. We've got to do X. We need to make X happen. When can you get back to me with a path to green for X? And also, let's schedule a post-mortem to figure out why we missed on X."
At the post-mortem, PM doesn't show up: "Uh, turns out we missed on X because I estimated it too quickly and the PM committed to those estimates. I didn't know about Y and Z when estimating, and these are things I could have done to detect them as requirements if I had known to do so."
Email from PM: "Hey, could you give me an estimate for X'? Doesn't have to be perfect, ha ha."
I've solved this by not panicking about software someone else owns. Worked out OK so far. They can fix their processes and expectations (I'll help!) to get more value out of me, or choose not to. Not my problem. Environment gets too toxic, well, see ya, and good luck (you'll need it).
"A week" is a terrible estimate for complex things that have components that you can't foresee. If you lose "a day" on some problem outside of your control (compiling starts to fail and it turns out someone checked in some code into your branch by mistake), you've basically wasted 20% of your estimate.
Never say a week when there's the chance something will require a month because you didn't understand it.
You need to learn this lesson. A week is a blink of an eye in software, and only things you understand take a week.
"I don't know why, but I don't feel comfortable guessing less because I don't fully understand X."
"Okay, that's not a good way to estimate. Why don't you spend some time to come up with a plan for X and we can estimate it after that. When can I have the plan?"
> "How long will X take?"
>
> "I don't know, I've never done X before."
I fail to see how your example makes any case on how estimating how many resources a project needs is stupid.
At most your example argues that asking inexperienced and clueless devs to estimate stuff they know nothing about produces highly unreliable information.
The main difference between hacking away at a code base and software engineering is identical to the difference between construction workers and civil engineering. Sure, estimating is hard. Yet of you want reliable estimates you need to check with experienced professionals.
I've worked about 10 years at FAANG companies and I'm yet to meet the experienced and expert devs who are good at estimating things. I think your characterization of "inexperienced and clueless" devs is off.
The reason I think my example illustrates a problem is it's moving away from agile. I shouldn't estimate X without understanding it? Okay, I'll take the time to figure out, look around corners, etc. I realize I need Y and Z. I've got to estimate those, to estimate X, so I start working with other teams to figure these things out, and drawing up plans and schedules and who's doing what when to get all this stuff delivered. Now we're basically at the waterfall model where we're planning everything out before doing it.
The other problem is that all the process is genuinely a waste. If we need X then let me work on it rather than do all the planning and scheduling nonsense. The schedule will be wrong anyway. If I have problems, need help, etc with X then I can raise those concerns and management can react in an agile way.
If you look through the managers who are weighing in on this thread, you’ll notice a common perspective: they don’t trust you to actually work on something. That’s why I have so little hope for any software methodology - even if it starts from something positive (like XP, which was the precursor to “agile”, did), it will be turned into “I know all of my programmers are stealing from me, how can I stop them?”
In part because we still measure effort in hours instead of in wear and tear. That guy answering comments on Hacker News is probably trying to recharge, not steal from you.
I learned recently that the crane operators that unload cargo ships at some major ports can’t work more than four hours at a time. The movements are too precise and take a great deal of attention. I think you can work two shifts per day but there’s a mandated gap of some number of hours between them.
> At most your example argues that asking inexperienced and clueless devs to estimate stuff they know nothing about produces highly unreliable information.
Wow, so much hubris in such as small post.
Unless you have been doing the exact same thing forever(in which case you are at a risk of being replaced), new work _always_ comes with a degree of uncertainty. Be it new technologies, new business requirements or what have you. I have never worked in two identical projects in my life.
The correct answer would be a 'spike' or proof of concept. Either that, or hire those magical developers you seem to have who know everything there is to know.
This would've worked if you'd taken your initial estimate (1 week) - bumped it to the next unit (1 month) - then doubled it (2 months). Maybe you would've been given 9 or 10 weeks, so you'd have the week for X, and 1 month each for Y and Z.
I understand this is a contrived example, but it's strange that, when you take a real project that "overran" based on the initial estimate(s), and you apply the formula - just how often the formula is actually really close to the actual amount of time.
Your argument here is against "business". In a dream world, where we could build for building's sake, and create "more perfect" things with no considerations for cost, I'd agree with you, but businesses (in the abstract) are a math function: Does it cost us less to make this than we charge for it? If so, we are a profitable business, if not, we are an unprofitable business.
Managers are there to manage costs, to protect the "business".
I think every owner of a casino knows there is luck involved in their profits, and that the odds are stacked slightly in their favor.
I don't think any owner of a business thinks that there is luck involved in creation of their software systems. They absolutely look at the $10 million per year they are spending on "IT" and look at the $26 million per year in value that spending is unlocking in absolute dollar terms.
But they know that the sum of all cards drawn will be profitable, because their practices guarantee it. Normal businesses cannot guarantee that the sum of all their projects will be profitable.
The subtler point is that if the developer won't make an estimate for how much time a feature will take, than the manager will. And the manager's estimate will tend to be even worse than the developer's, so we should all prefer that the developer makes the estimate even though it will be a bad estimate in an absolute scale.
Note that the manager's estimate may be implicit or explicit, conscious or unconscious, but they will make one in the absence of the developer's. An implicit estimate would look like the manager doing a "gut call" when deciding whether or not to give the go-ahead to a feature, with no numbers attached to time and cost.
The problem is that if the developer gives an estimate, the manager might hold him accountable to it. If the developer can't even meet his own estimate, that proves to the manager that he's a bad developer. (He's not; he's just bad at estimating development time, as is everybody else.)
And that's what Scrum and story points are for: to help you make long term estimates once part of the work has been done. You use the feedback on how abstract story points turn out in practice, to get a rough long term estimate of how long the work is actually going to take. Without being so explicit that anyone can be held accountable to them.
That's one of the big reasons that I've become a proponent of Donald G. Reinertsen's approach to product development.
He emphasizes a variation of the cost benefit formula + urgency, called Cost of Delay for prioritizing that basically boils down to "Value / Time Remaining" because your cost is the time remaining. It naturally prioritizes work based on shortest time to value because, unless the value proposition is enormous, dividing by estimated time is almost always going to favor the shorter tasks.
This will of course vary by team, but in general it puts a focus on responsiveness and getting the little stuff out of the way first (including tech debt). When you have a formula that doesn't let the little stuff build up, it gets easier to focus on the big stuff too. It's also make it easy to pivot because it totally disregards prior time investments.
It meshes better with what agile is supposed to be better than any other methodology that I've encountered to this point.
Tech Debt is not little, or we wouldn't carry it around, rather than devote time (and risk of revealing new unknowns) to fixing it. My agile experience has always favored conquering little stuff first, since ppl recognize stories more than points (which vary in value, over time). This, unfortunately, sometimes leads to the hilariously subversive act of splitting a story up into as many tiny pieces as possible so that some people may inflate that "What have you done for me lately" list while more tangible coding is done by others.
Cost of Delay metrics do a good job of explaining how the cost of that debt can grow over time the longer it goes unfixed. Some are smaller than others, but uniformly it tends to get harder to fix the longer you wait to fix it.
He also doesn't place any type of value in points or other, bluntly, useless metrics for measurement. The entire focus is around delivered value. You can break it into however many stories you want, the value is the same. The measurement of how many things you delivered or how much time they took is not important to the company.
What's important is how much value was been delivered to the company. It's the only metric that matters.
splitting a story up into as many tiny pieces as possible
Honestly I think that's the intended outcome from the business's perspective. The smaller the piece of work, the easier it is to assign a cost and a benefit to it, and the easier it is to hand off to another developer.
>... and the easier it is to hand off to another developer.
I have never seen this work out in practice. And it's like a 1%-of-the-time thing anyway.
On the flipside, I've seen quite a few hours wasted adding fake subdivisions to tasks, probably close to 30% of the time. Outside of small, known bug fixes or tech debt cleanup, it's rare for a thing to wholly fit within a sprint.
As a consumer, I can buy an already manufactured pair of jeans or an already programmed Adobe Photoshop for a fixed price.
As a producter, the cost of building a jeans factory does not seem to be inherently more predictable than the budget of a software development project.
The recent Foxconn plant in Wisconsin that may or may not be happening is a great example. They committed earlier, and claim now that their initial estimates were incorrect and that a plant in that area will not be profitable for them. They have now stopped the plan entirely and reworked what it is they are going to build completely from scratch. This despite having already committed to building the factory.
Even at scale people can pull out of complex agreements if their estimates turn out to be faulty.
You should be able to put it in a range otherwise you probably don't know enough to build it. You don't want to spend 100 millions on something that will save 1 million a year.
managers and business leaders make decisions based upon those estimates.
Imagine if that cable guy coming your house said between 1 hour and 2 weeks. Good luck making anything approaching a good decision surrounding the cable guy's visit.
Agile never gave organizations a holistic, viable alternative to Waterfall. Because there’s a difference between theory and practice. Product work is more about practice. When we complain about “AINO” (Agile In Name Only), we’re not being honest with ourselves.
I agree with most of the article. Specially the keep-learning part.
All Agile did was put software development teams unfairly under a microscope.
I believe Agile has been tremendously beneficial for the industry globally, especially in some subtle ways. For example, Agile says you have to communicate a lot, if you want to get software done. Here is the subtlety: if, today, you stop telling the average programmer to communicate, they will stop and go back to silo mode, "naturally". At least some of them. If you think about software at the industry scale, you have to think about a wide population which require processes.
I was once working in a nice little company where, one day, they introduced agile. It was so much beneficial. Before, the bosses thought that talking was simply a lost of productivity. You're a programmer right? So code, don't talk. So we would develop software without talking to each other. After we had daily meetings, scrums things and stuff. Our velocity has sky-rocketed.
You have to realise that before Agile, a fair portion of all software development projects that were started would simply bust and never get shipped. The code is a complete monster or the budget is nuked. When I started in the company I just mentioned, I started working on a codebase that was the worse code I've ever seen in my life. You would touch one line and everything would stop working. It was a condensed piece of spaghetti with hacks on top of hacks on top of hacks. Software architecture? That requires some talking and thinking, forget about that, not permitted.
Now I agree with you, agile is not much useful for a hacker who consistently get software done and understand deeply what's happening.
edit: not that I'm an agile guru or anything. Actually I only know the thing superficially. I show up at the meetings, when I'm ask how hard something is, I answer, then I mess around with my office friends. Still I can appreciate it works much better than any process the bosses can come up with.
> You have to realise that before Agile, a fair portion of the software development projects that were started would simply bust and never get shipped. The code is a complete monster or the budget is nuked.
The difference is that of ivory tower planning and the following phases of development, testing and a “big bang” release (waterfall) vs working with an MVP with the purpose of releasing as soon as possible and then work in iterations based off of actual feedback and demand (agile). If you manage to nuke your budget or create a monster of a code base already at the MVP stage no methodology is going to save you.
I really do believe agile (or at least "not waterfall") has reduced the rate of major software flops. But I'd be fascinated to know how often it turns failures into successes, versus revealing failures earlier in the project cycle. Both are valuable, obviously. It's just interesting that part of Agile's value is in revealing engineer dysfunction or fundamentally bad ideas at the MVP stage instead of the completion stage.
That’s a point, and I agile development would not be possible without the internet. But working incrementally based off of customer/user input is only possible after the first release. What was observed here was the huge number of projects that would previously fail without ever getting that point.
I have the subjective sense that the rate of massive, irreparable flops in software has gone down a lot. (Very anecdotally, the latest "permanent failure" on Wikipedia's list of major failed software projects is from 2014. That list is worth a read regardless.)
Projects still get cancelled, budgets and timelines get blown out, features and testing get compromised. But at least in dedicated software companies, it's pretty rare for something to get through 90% (or 150%) of its allotted development time and then be completely discarded as unsalvageable. (For technical reasons; market changes are a different beast.) Projects in technical crisis are either apparent sooner or have usable elements, rather than failing outright at the end of a waterfall.
Of course, there's a lot of room to debate what changed and why. If Agile led projects to fail sooner or less dramatically, that's still important, but less notable than if it changed them into successes. And a lot of technological advances have helped too; things that might have been outright failures as shrinkwrapped software can become late or overrun projects in the era of digital distribution.
I was interested in comp sci precisely because I didn't have to talk to people, an area where every interaction is a performance. I want to be alone - what's so wrong with that?
There is nothing wrong with that, but careers under capitalism are about _signaling value_. I hope you find a place (or have found a place) where you're appreciated. :)
I'm not sure why you're being downvoted but some of the best career advice I've heard is someone saying you should assume when you go to work that you're working under a communist dictatorship. I.E. Toe the party line, make your boss look good, (pretend to) eat up the company propaganda, assume the leader(s) (C-suite) will do whatever they want, when they want, especially giving themselves bonuses regardless of company performance.
It's a weird contrast to normal life, considering a lot of us live in democracies, but once I understood this and started acting accordingly, it helped me handle my work life.
It is probably good advise for most cases but individually depends on the shop and culture. Like if you really should always wear a three piece suit to interviews regardless of what they request. Some are flatter, others are more hierarchical (ironically practice can fail to line up with org chart structures).
Some are all about the politics, others don't have the luxury of self delusion or actually value the sort of brutal honesty to say "I have looked at the new framework - while trendy it is buggy inefficient crap." The ones who lack the luxury tend to be smaller but small size is no guarantee that they'll just say no to the flavor aid.
I don't know if that's great advice, but its certainly advice that will help get you promotions and salary bumps.
Like anything there's a grey zone here. Understanding the political realities of a business is highly beneficial and will help you move forward with the company and let you know when to pick your battles. But I've watched people who do nothing but this lose the trust of those at their level. And that trust is crucial for agile development. I also personally feel like pushing back on the company when appropriate can indeed provide a lot of value to the organization. "Why are we doing this meeting?" "We need another week for testing" "I'd like to see the roadmap you are planning". Just don't push back all the time.
As far as your career - salary and promotions and recognition - you can't just sit in a corner and be quiet. People have to recognize that you're good at what you do and it's hard for others to see when you only communicate in code check-ins.
You have to start attending meetings, speaking at meetings, and being useful beyond the code.
In all areas of life we judge those around us not by the true facts, but on our limited knowledge. Working to improve that knowledge can result in a change in judgment. It seems completely reasonable for there to be a person who does good work but who isn't know for doing such, and as such is viewed worse than they should be. By working to increase the knowledge of the work they do, their evaluation in the eyes of others will improve.
Of course, it isn't a straight forward or simple in practice. There are those who lie and misrepresent, and if you are to obvious about your intent you will be viewed as manipulative.
I don't see why this deserves either the label of BS or a ban from HN?
Perhaps some thing think it's uncomfortably close to the 'slur' of 'virtue signalling' that people throw around. While I'm sure the intention was nothing but pure in how it was used here, it did make bit second-read the comment to assess whether it was being used in a negative way, and to really understand the point being put across.
I had a similar sentiment - As a programmer, I did not want to be communicating with a client, I wanted to be left alone in a sense that I only ever had to interact with people that understand how coding works.
I think it's doable; say you are the sole developer of a massively used library. The programmers being your only users - they read the API docs, submit an occasional feature request/bug report.
Heh. I have a reverse sentiment. As a programmer, I want to communicate with the actual users who'll be using my product. But without two layers of intermediaries on both our and customer side, who end up turning this whole thing into a game of telephone, ostensibly in the name of all the other important stakeholders.
The programmer users are arguably "clients" in this context too. Much the same sorts of issues, different labels.
One need only look at the bug tracker on the average "massively used library" to see that you will be communicating with clients a great deal. Your dream of a small group of perfect programmers filing neat and accurate feature requests or bug reports doesn't match any reality I've ever seen. To expect them all to "understand how coding works" is also likely a pipe-dream - users of all ability levels are out there writing code, filing bug reports for things that aren't bugs, asking questions in forums that aren't meant for those questions, mad at you because the direction the project has taken isn't the one they or the company they work for want/need, etc etc...
Users of a library are still users. You can write a library in an ivory tower, just like you can write applications in an ivory tower, but that kind of software is written for yourself and only incidentally for anyone else. People might still use it if there are no viable alternatives, but don’t expect them to be very happy about it. Especially because scarcity of communication also implies scarcity of documentation.
Feel ya. Surprisingly enough, I'd say that I've not had to attend so many boring meetings in my career but certainly I'm just lucky. Also, once I did get a manager who was an "agile guru" and he was annoying. He talked to me like he's the jedi of software development and I'm his padawan. Thankfully, he got fired.
I had a professor at university who acted that way about Agile. He thought he was literally Uncle Bob incarnate. But when you looked at his credentials, he had never once worked in a real production environment, he had only ever taught.
Nothing is wrong with that, but don't expect to keep a job where you don't have someone to do all the business and project planning and coordination for you, at the expense of your salary. If your computer science talent is good enough, that might work for you.
You're abdicating your responsibility of developing working software. Your job is to solve problems, using code where appropriate. Your job is to understand the entire system and explain how the corporate system above interacts with the technology you are tasked to develop.
If you can't do that, what is the difference between you and a group of outsourced employees making pennies on the dollar?
you’re presupposing he wants to fit into that holistic model, as a senior person. maybe he is fine plugging away at code, the task defined, like a monk transcribing texts.
the world needs both kinds, and a range in between.
If you are writing software just for yourself, there is nothing wrong with that. If you are writing software for others, be prepared to be steamrolled by people who know how to write good code AND how to communicate with consumers of the software (i.e., paid customers, OSS devs, enterprises, etc., that one will depend on what kind of software you are writing).
I think that a lot of developers with a mindset similar to yours tend to underestimate how important good/valuable feedback (and communication in general) is. And I am saying that as someone who initially went into comp sci for reasons similar to yours. Some of the best engineers I ever worked with had amazing communication skills, and it acted as a x10 multiplier to their technical skills and overall productivity.
Ultimately, these days programming is table stakes - IMHO you get zero credit for knowing how to code. The best developers are those who are able to step up and actually interact with people to articulate and understand problems. Whether that's with fellow developers about code, with managers/stakeholders about technical things, or with end-users/clients about users needs.
You're more than welcome to want to sit in isolation and code away, but I would suspect you would find it difficult to do this and be happy. I've never worked in a team that would work with someone like that.
> > All Agile did was put software development teams unfairly under a microscope.
Most managers crave control over their teams; this is not something that Agile introduced. If anything, Agile let them use the sort of control they were already demanding towards more useful and productive goals, by introducing bazaar-like practices to centralized software development (release early, release often; shorten feedback loops as much as possible; make extensive use of refactoring, software testing and XP principles); while at the same time not being altogether incompatible with self-organizing development teams (these were mentioned in the original Agile manifesto, after all).
Agile is shit. It hasn't done anything but increase the cost of software and overhead for technical teams. It's ridiculous how much time is wasted just to "be agile". It's a scam, total scam sold to huge companies because it provides a high level overview of the work being completed.
Eh? I'd compare it to No True Scotsman, not AI: I mostly hear people insist that anything which doesn't work isn't Agile.
I'm extremely grateful for the changes agile wrought on the software industry, and while I think many of the best insights have become commonplace, I don't think formal Agile methods are outdated or exhausted yet. But I do think formal Agile is extremely hard to do right, and has very common failure modes of knowingly-unrealistic planning, unproductive meetings, excess design changes, and tech debt neglected to ship MVPs.
One measure of a methodology is how much value it provides when it's used right, and I agree that Agile shines here. But it's also worth asking how easy a method is to get right, and how gracefully it devolves when things aren't perfect. My experience is that agile advocates commonly neglect those parts, dismissing widespread frustration with 'bad agile' on the basis that if it had been implemented perfectly, those issues wouldn't have come up.
> You have to realise that before Agile, a fair portion of all software development projects that were started would simply bust and never get shipped.
I have plenty of gripes with Agile, but there's definitely a "victim of its own success" aspect to the whole thing.
"Agile in name only" is frustrating compared to a good system, but in many cases it's still far better than what came before it. Basic ideas like "we should expect requirements to change" and "if the programmer doesn't know what their code will be used for, something is wrong" weren't necessarily accepted prior to agile. Projects that are chaotic, mismanaged messes under AINO might well have been orderly deathmarches or cancelled outright in the past. Some of this progress is technological (source control, post-release patching, digital distribution), but some of it really does owe to Agile.
It reminds me a bit of the scientific method. The elaborate eight-step thing taught in school feels like silly boilerplate, but it's shocking to realize that "ideas are tested by experiment" was a genuine breakthrough from a past that made major choices like disease treatments and scurvy cures based on 'reasoning' without testing them at all.
Agile is not science-based, it is an ad hoc bag of techniques that seem to have merit. As such you have to be able to reason about what your practices are and you have to relate everything to concrete business goals.
Your goal should never be to become "more agile". You should be looking to improve efficiency, to make sure you are building the right thing, to improve communication so that everyone is on the same page, etc. "Agile" is never going to magically improve anything.
Uncle Bob wrote about this in The Tragedy of Craftsmanship (https://blog.cleancoder.com/uncle-bob/2018/08/28/Craftsmansh...). His perspective is Agile lost its way once the Project Managers stepped in. It seems to me the problem with Agile in practice today is there's too much focus on process and not enough focus on, you know, the actual software being delivered. Add to that a host of less-than-desirable ideas that have taken hold (don't even get me started on emergent architecture) and you realize modern Agile has become a cesspool.
What to do if you're on an Agile team? There's value in the Lean Methodology. Remember, the whole point was to deliver software your customer needs to fulfill their business objectives. Continually assess your Agile practice and see if what you're doing still makes sense for the project you're working on right now and for the current stage that project is in. This continual assessment with a focus on the end goal of delivering software can go a long way to addressing Agile's modern ills.
I had not heard of "emergent architecture" until you implied you hated it, and so I thought I would Google it.
I have no idea what emergent architecture is in the agile context, but the idea of emergence is fascinating to me, and has been ever since I read "Emergence: The Connected Lives of Ants, Brains, Cities, and Software" by Steven Johnson.
We see the idea of emergent design everywhere in nature and society. I'm not suggesting everything can be built and designed "emergently", but certainly some things can be. See the book (and concept of) "the cathedral and the bazaar". The idea of something useful coming into existence without central planning is an enticing one, and so I can see the draw.
Thank you for that book recommendation - I just bought it and am looking forward to reading it :)
I also had not heard the term "emergent architecture" in relation to software development. I'm intimately familiar with the concept of emergent order, though, and my grokking it was part of my "political awakening" in my 20s. It led me from being a libertarian-leaning conservative to an anarcho-capitalist. To put it another way, I believe that the system create by independent actors acting in their own self-interest naturally create the most efficient system possible within the constraints of the environment within which they operate.
There's a name for this concept when applied to company management, but I can't recall it at the moment :(. I want to say the word has a Greek root, and it reminded me of "autarchy" ("self organized") but that wasn't what it was called.
You'd probably like Milton Friedman's ideas, and I'd be surprised if you've not already come across his work and YouTube lectures.
There is something to be said for small government, but also can go too far: actors acting in their own best interest can obviously cause harm to society or to the larger group as a whole (See "tragedy of the commons").
Related to emergence is rapid iteration, and it's importance to analysing complexity, which is caputured nicely in "Boyd's law". See https://blog.codinghorror.com/boyds-law-of-iteration/, and then see if you can find the referenced paper by Roger Sessions. (Sessions seemed to caputure the essence of "Lean Startup" before it was cool...)
I don't know any description of Agile better than meaningless buzzword. Buzzwords are not without value. They are a tool for selling a change without describing all the nitty gritty.
"Agile" is a lie that before 2001 nobody at all had ever done iterative software development, and all software process ideas that came after 2001 are by definition "agile."
"Agile", as with any other buzzword, is not for people who actually understand the details of the change; it's for someone who needs a handle onto concepts that would fly over their heads without a buzzword. Lets stop litigating the term, and move on.
Organizations have to find their own way of working. It's going to change based on the size of the organization, its goals, and its people. With the exception of 1.5 years in a software company, most of my organizations have been internal IT. What I find is that user areas, who initiate projects with us, often don't have a clear view of what they want but think it's going to help the organization. Our job is to build models quickly so that they can validate an idea. Only after iterating through a few models can they reliably say the benefit/cost of the idea.
A very good engineer within my organization said to me, "Our job is to get something in front of them quickly so they can figure out what they don't want." Our successful projects have been ones in which we kept to this maxim.
I think this is what the author was trying to say but I had a hard time following his writing style.
"Thou shalt not be negative" (HN) - well I'm sorry. Just some example text:
> both Agile and Waterfall are focused on building. Design is about validating.
or
> So…what’s the way out? It’s a smart focus on clear outcomes, not output, with roadmapped outcomes replacing planned milestones, with trusted product teams, not project teams, empowered to vet assumptions and discover the minimal path to value.
Satire? Enough words to fill a hot air balloon or two. "Smart" focus, "clear" outcomes, "outcomes" vs. "milestones"(?), "trusted" product teams (as opposed to... what?), "empowered" etc.?
Far more interesting than the contents of many links that make it to the HN homepage is discovering the fact that they do so.
Honestly that statement is the nail on the head for me. There's a huge distinction between an output and an outcome and a huge distinction between a project plan and a road map.
> between an output and an outcome and a huge distinction between a project plan and a road map.
I mention neither of those.
I'm not complaining about your reply, I think it shows how those texts are (to be?) used: Just like art. You project your own experiences into it, it's more about getting you to think. So naturally different people are going to react differently and read different things into the same text. That's perfectly alright - you get something out of it (only) if you put something in. If there is nothing to begin with, which may just be lack of experience (new programmer), you'll get less out of it. If you have two decades large project experience in various companies and teams you'll get a lot out of it. Let's attempt to make the best out of it then, out of any text, if we are presented with that opportunity and the headline got us to click on it :-)
In the spirit of that second paragraph - focusing on outcomes, not specific implementations - here's a rewrite:
Original:
> So…what’s the way out? It’s a smart focus on clear outcomes, not output, with roadmapped outcomes replacing planned milestones, with trusted product teams, not project teams, empowered to vet assumptions and discover the minimal path to value.
Rewritten:
> So… what’s the way out? It’s a focus on clear outcomes, not output. There should be dates for when specific outcomes should happen. Teams should be empowered to find the minimal amount of work needed to achieve the outcomes.
It's a really wordy way to say what any good leadership class would tell you. Focus on what you want more than how you get it, and let your subordinates decide how to get it. Both good product teams and project teams are capable of working that way.
This kinda sums up my problem with endless agile vs. waterfall vs. whatever discussions. The interesting parts aren't really new ideas and the extra buzzwords add nothing.
Interesting, I’d interpret “roadmapped outcomes replacing planned milestones” to mean there should not be predetermined dates and deliverables (planned milestones), but rather, a sequence of value-delivering outcomes (roadmapped outcomes).
A relative order of outcomes could also work, yeah. You'll have a date in there somewhere, though, even if it's not driven off the amount of work. Could be "and if we can't achieve this outcome by X date we give up on this whole outcome".
I have a theory that commercially successful software development methodologies are like diets: they have to be almost impossible to follow. This ensures that when you fail to lose weight/achieve bug free software, you blame yourself for not following the rules exactly, rather than the rules for not working.
Then maybe there should be one rule (as was my most successful dieting effort): Is what you're doing helping make or save money?
I know such a simple rule is probably causing a lot of you to cringe, but I can very easily fit just about any activity of value you or your team embarks on into that question, and it's a rule the entire company can get behind.
That's probably a very apt metaphor, but there's an upside too: many diets actually do at least temporarily help people in spite of their irrational basis, just because $random_diet causes the dieter to observe and reflect on their eating habits more carefully in general. Even a non-sense diet like "eat only foods colored yellow or red" probably does a lot of people a lot of good, compared to their usual state of being completely unconscious of their awful habits. Same goes for software development practices.
I think with this analogy you're presuming that people are exposed to a variety of foods and there is some positive correlation between action and awareness. However, for someone who only ever goes to McDonald's to eat, they'll just switch to french fries and ketchup for your nonsense diet.
At the very least, I think it's common to judge methodologies by "how well does it work when it's done right?", when we should be using a three-part test.
1. How much value is there if it goes right?
2. How easy it is to get right?
3. How gracefully will it break down if it's not done perfectly?
Methodologies don't have to shine on 2 or 3. NASA-style clean-room coding is the gold standard of hard-but-effective; you have to suck it up and commit entirely with a great team, but the results are unbeatable. Waterfall was "good, bad, bad", but used by lots of less-exceptional teams with hideous results.
Agile, I think, improved #3, but not #2 so much. It's commonly sold by appealing to how much it can potentially help, but I think the more honest pitch is "it's tough to do right and it'll sting if we get it wrong, but it hurts a lot less than getting waterfall wrong."
Not everyone. Some are very successful with it. The problem is that everybody wants to copy that success and tries to cargo cult themselves through it without understanding it.
"let’s stop pretending Agile was some sort of cure all."
That's certainly good advice. Agile is more of an attitude than a solution, and attitude is not going to solve all your problems. Even if you are as Agile as you can be, you still need to solve your problems. It's just that being Agile might make you more flexible and effective at solving those problems.
And Lean is certainly more comprehensive than Agile. And I don't think they contradict each other. In fact, they're very similar; they're both about empowering the people who are doing the actual work.
"management thinks the focus should still be scope and deadlines and efficiency, ignoring that the deadlines are arbitrary and the requested time estimates are a form of waste."
Then management is not Agile and hasn't properly learned what it is. That is sadly common. Agile is not magically going to solve bad management, but Agile does emphasize keeping management at a distance and empowering developers.
"Agile actually tends to mask the core problem, which is a systemic, bidirectional lack of vertical trust."
Does it mask that? I think it exposes it. Without that trust, Agile cannot work. Agile insists on that trust. I suspect that every case of broken Agile is caused by a lack of trust. Fix the lack of trust (or use a system that works without trust if you absolutely must).
"Did you know story points were actually invented to obscure time and help alleviate this problem? That backfired too, didn’t it?"
Yes I did and no it didn't. Every single team I've worked in over the past 6 years knew this and kept repeating it every time someone comes along and tries to match story points to actual hours.
This first half of the article suggests the author has mostly worked in very toxic management cultures.
My last company (before going self-employed) had an agile environment and it was mostly a good system to work in.
I don't recognize the fairly strictly Agile (not AINO) environment I worked in in that article. Not perfect, but it worked and better than a lot of places that had no good methodology.
IMO focusing more on systems than people in management is a mistake.
To the extent your work isn't repetitive, the only job process can do is set a cadence on communication & decisions. Yes 2-week scrum does this but in my experience it distracts companies from the real goal of management -- attaching subject matter experts to projects, i.e. concentrating time & resources towards a goal.
Process can get rid of bad people and share around your best people, but unless you have an assembly line, beware of any other claim.
I was prepared for our weekly "I Hate Agile" post, but this one is actually really great. It's a lot of the arguments I make to Agile haters. The fundamental problem that drives most agile failures isn't in the team's execution, it's in the business' expectations. One side is signed up for incremental delivery, and one side is set up for a fixed scope and deadline and the result is misery. I think this article makes a lot of good points. Scrum isn't the complete answer, but it's a big step in an organizational transformation. I think it's suffered a lot from being oversold.
“One side is signed up for incremental delivery, and one side is set up for a fixed scope and deadline and the result is misery.“
That's just one example of the general value I like to ascribe to "AINO": it gives you a useful mental model of the delta between what you have and a known good process, making it quite easy to name the missing pieces. This usually doesn't make it easier to actually fix their absence, but at least you know where to start with damage mitigation. When waterfall fails, you just throw your hands in the air and say "more of the same".
I often sit between the business and tech orgs, so the way I explain it to the business is this: I can get you detailed status reports and metrics, but they will slow progress and be expensive. So think about why you need that information: if there are legitimate business reasons with dollars attached, go for it. If it’s just to soothe your anxiety about the timeline, a therapist will be far cheaper and more effective.
> One side is signed up for incremental delivery, and one side is set up for a fixed scope and deadline and the result is misery.
This is a brilliant summary, thank you.
The best 'agile' experiences I've had are situations where the 'clients' are directly involved, often within the same organization. Instead of a hard scope or deadline, there's just a shared interest in producing a valuable product efficiently, and the users are on-hand throughout the process for feedback and reevaluation.
The worst experiences have been waterfall contracts, developed by an internal simulation of agile. The software team does frequent "releases" to business or management, who provide feedback and feature requests, but the actual recipients are uninvolved outside of occasional demos, or contacted only indirectly by non-programmers. The result is almost always thrashing, with time and effort spent pointlessly satisfying the forms of agile even though the real timeline and customer feedback are unyielding.
In large enough orgs, internal clients can end up being just as bad as external clients, or even worse since they have a direct line to your PM, can track your feature board, etc. yet there is basically no sense of camaraderie or shared goals.
I'd say generally IME they are still preferable, but occasionally can be more painful.
Actually, I think Scrum is just a first step to give the development team some autonomy to create a bubble in which it can play by its own rules. AFAIK, mediating between the development team and the expectations of the management is the job of the Scrum Master [1]:
> The Scrum Master serves the organization in several ways, including:
> ...
> - Helping employees and stakeholders understand and enact Scrum and empirical product development;
So, I am not saying that it is an easy job. Many managers just hear what they want to hear, so it can be quite difficult. But if your team has such issues, be sure to support your Scrum Master with good arguments to help him make managers understand what it means to use Scrum (and become Agile).
That would be more helpful than the far more common response of just "you are doing it wrong" argument with absolutely no indication of how to do it right.
> So find a good booklist. Follow some good blogs. Here’s a start: If you haven’t read Sense & Respond, Lean Enterprise, A Seat at the Table, and Everyone Is a Change Agent, I suggest you do so pronto. Your leaders too.
I'd add Peopleware to that list.
> Start reading posts by John Cutler, Melissa Perri, Bob Marshall, Allen Holub, Laura Klein, Erika Hall, Neil Killick, and branch out from there.
Anyone care to add their own handful of books and bloggers to these recommendations?
Any software development methodology or process is going to come up short if it ignores the fundamental nature of software development, which is: We do not know what we're doing until we do. Any process or methodology that tries to extract promises out of the software development organization about things that are still in the we do not know what we're doing phase is going to result in disappointment at best and serious harm to the overall effort at worst.
And it turns out that it's not unusual for software developers (and consequently software development organizations) to spend more time in the we don't know what we're doing phase than the we do know what we're doing phase. That sucks from a management point of view, but no amount of process is going to make the problem go away. The best you can do is manage the risk. We don't need good processes or methodologies nearly so badly as we just need management enlightenment about how software development actually works.
That's the truth, except, I can't tell clients that "I'll know how long it will take when I'm halfway done." The best I can do is look at prior work and multiply by the LarryDarrell Constant of 1.5.
I usually do 2x my expectation. I keep track of my prior estimates and compare against my real time. I normally miss by about 30% - 50%, this leaves me a fair margin. It’s also been improving with time, which means I’m getting better at assessing my skills and problems.
For me, I tend to work quicker than other contractors / employees, so I still often beat out the other quotes (time or money). That’s not always the case for others, so they margin of safety may not be there. In which case, multiply by a smaller constant.
I kind of lean towards double the schedule and cut the scope in half and hope that maybe we can deliver a little extra credit work.
It's important to note that padding schedules is not just done in software, it's a common practices in all sorts of engineering. It's not bad that we do this, it's bad that management often doesn't understand that we do this and doesn't understand why. The understanding why part is actually really important, because that means management can be expected to do their part in managing the uncertainty or at the very least they can be expected to be careful about not making the situation worse.
I think one of the top skills for management / senior dev is to be able to translate between software estimate uncertainty and the business reality surrounding the schedule.
For example, if you ask, "How long will this project take?"
You could just say, "I don't know. At least two weeks, but it might take as long as six months." This may be accurate but it's not helpful.
Or you could say something like, "In a half day I could put together a preliminary design, and identify the most uncertain parts of the project. From there we could figure out what is needed to tighten the schedule or adjust the requirements."
And I think that one of the more common problems with managers in larger orgs is that they can't tell the difference between somewhat arbitrary deadlines (we want this for Q3 to look good in our org, but we can break this project into smaller milestones and push it back if necessary) and deadlines that reflect the reality of business (we want this for Q3 because this will have cascading effects on other parts of our business, and if there are problems we need additional resources or an adjusted scope).
Any time there is a deadline, the manager should have a clear understanding of the consequences of missing that deadline, and be able to communicate that to the engineers. The efforts to come up with a schedule and milestones should be collaborative. Too often there are mandates that have unclear consequences for missing them, and when schedules slip you have no idea what you are supposed to prioritize.
Funny thing. If you do this organizationally, then you eventually end up making larger and larger estimates. IMO it is better for management to understand the inherent risks associated with engineering estimation and build the padding on their end.
It's a tough nut to crack. Think about fields like architecture and engineering, where the historical knowledge is well over three thousand years. Yet they still have failures, go over estimates, and generally suffer the same issues that we do, but at much greater costs.
I love this talk by Dave Thomas, one of the people whose names are on the original Agile Manifesto.
Among other things he speaks about the simplicity of the Manifesto, about the cultish movement and consulting services which rose around it and the cultish, distorted, complex vortex of BS that has formed around a super simple concept.
In my experience, these discussions boil down to anecdotes at twenty paces.
I've seen software development done very poorly and done very well, sailing under a variety of flags. But on an eyeballing there are far more bad experiences than good that get labeled "Agile".
Whether this is because of causal differences or because it's easy to affix any label to anything (and most experiences suck regardless of the label) is hard to tell without a control-group universe were the Agile Manifesto was never written.
I have a theory here... culture beats process, but companies without the culture have to rely on process. So you’ll find that companies with the right culture can deliver better results without needing a bunch of process, but companies without the culture end up having to force it with process.
And tbh, despite how bad “Agile” process can be, it’s a lot better than pure waterfall.
Pivotal on the whole does software development as well as I've experienced. When you hit the sweetspot for our core practices it's really impressive. Everything just sort of flows.
Sometimes it goes off the rails. That is in the nature of things. But it goes off the rails as an exception, rather than as a rule.
If there's a key to the magic it's (1) hire for empathy as well as smarts and (2) reflect on what can be done better.
Agile is like Communism. It's never been done successfully and every time it fails the apologists always respond with, "well... they weren't doing it right."
I have tried Scrum. I have also tried various pick-and-chose hybrid approaches which were generally less successful. For example having scrum-like sprints and daily stand-up, but still having a fixed feature set and fixed deadline. This doesn't really work. (You can have deadlines in Scrum, and you can have fixed feature set, but you cant have both!) This happens if management does not really buy into the process. The place where scrum worked was because management was willing and able to try something new and willing to learn.
For what it is worth, I have also seen classic waterfall be successful. But I have also seen several big-scale disasters with waterfall, and in my experience waterfall carry much larger risk and is much harder to salvage when it goes off track.
419 comments
[ 6.3 ms ] story [ 306 ms ] threadOn the other hand, I've also been in orgs that were very successful in developing software in a way that we would recognise as being agile. We just didn't make a big deal out of it. And we didn't do standups (most of the time), or 2 week sprints, or retrospectives. We didn't even pair consistently: we split up on trivial stuff, paired on hard stuff, as needed and available.
In one org we started without TDD, and even without having much in the way of automated tests at all. But when we discovered testing and TDD, boy did we ever embrace it. Not because anyone told us to (mid-to-late 90s, so no Agile Manifesto), but because the difference for us developers was night and day.
So develop software. Concentrate on what helps you develop software. In my experience, you will probably discover something recognisably agile all by yourself. So when you discover that, help yourself to what helps you. Don't make a big deal out of it.
On the other hand, if the company all about how fantastic the culture is...run! Or at least tread very, very carefully.
"What's your company culture like?" - the way that the interviewer responds is a pretty good indicator of what their culture is like.
"If someone does something exceptional, how do you reward them?" works in a very similar way.
That does seem to be the downside of Agile. It's a collection of maybe a dozen different techniques and practices. But if one of those practices falters, whether it's the TDD, or the business side still wants a certain deadline, or you don't bother to demo at the end of a sprint, then the whole house of cards falls like dominoes..... checkmate!
Green teams may need more structure. But the structure should not take over; it should be a tutorial and get rethought as the team matures.
Message to the Fortune 500: Just get rid of anyone with the title "scrum master" they're dead weight. You were fooled, deal with it. I am trying to move my team over to Kanban right now, but we have all this reporting crap up the chain all designed around CA Rally, the worst productivity tool ever made.
You mean the ones following one of those canned "agile processes"?
It always amused me how people could read the Agile Manifesto and then conclude that it's best satisfied by following a canned process.
From how Ive heard it described it seems to be a way for management to impose rules on how developers can approach writing a feature and not necessarily a benefit to the devs.
If I understand it properly, it's essentially write a test so it fails then fill in the feature gap. The whole write first bit is just so you don't have to retest later to verify that your test didn't hit a false positive, right? Something a careful person would check anyways...
TDD was discovered as technique by developers, for their own benefit. Management imposing it can be a bit of an anti-pattern, though it may be well-intentioned. I have certainly pushed teams to TDD, though primarily by example and by the positive experience you quickly get.
Anyway, TDD has nothing to do with management imposition. It is really diametrically opposed, at most orthogonal.
2. "test didn't hit a false positive"
Yes, that's a good thing, though hardly the main point. You write that a careful person does this anyway. I'd rather have a machine do that for me, and have less of a requirement on me to be careful. Computers are much better at being careful, and I want them to do the tedious stuff.
I also am wary of claims of "you just need to be careful". Probably lots of bugs ahead!
3. So why is TDD so great?
First, it's a design technique. Writing the test first forces you to think about how your new production code will be called. It also forces you to be clear about what the code is supposed to do, precisely, before you write the code. Writing the test tends to be significantly less difficult than writing (correct) code to solve the problem. You then write only the minimal code that is required to pass the test, which surprisingly also tends to be fairly trivial. Also, you know when you're done: when the tests are green again.
So you've transformed one difficult step into two fairly trivial steps. Which sounds weirdly magical, but actually works in practice. Also, since you only wrote the minimal code that was required for the tests to pass, you know that you have good test coverage.
After you've done a few, you're in a place to detect duplication and eliminate it by refactoring. This tends to be somewhat more intellectually challenging. However, you are truly refactoring, that is not changing the functionality, so you have a great safety net with the good test coverage you achieved before.
So you get this small steps that are fun and easy, and you tend to not regress.
Psychologically, it's also very pleasant, compared to the usual flow, which goes somewhat as follows:
TDD: So with most normal development flows you get that initial rush and then a lot of downers. With TDD, you don't get quite the same initial excitement, but instead you get these nice regular dopamine hits without really much downside.As inevitable as blowing through the projections.
"Ok, how long is this project going to take?"
"Well, it's still being defined..."
"What do you mean you refuse to tell me how long this project is going to take?"
The analogy is so ridiculous that the person making it clearly doesn't even comprehend the domain he's discussing.
And that's the core of the problem (which, ironically in the context of the article, agile methodologies were supposed to address): wanting or needing something to be true/exist doesn't make it so.
That said, I don't think it's black and white. Some projects, making a simple web site that follows whatever Squarespace template is popular and filling it with some relevant content can be a pretty simple project, even to the point where the buying a pair of jeans comparison makes more sense.
The problem, I suppose, is when you believe strongly that every project can be accurately priced up front. Not every project is a moonshot, but some truly are and it's not always obvious up front which ones are.
If you are inventing something incredible,why are you someone's employee instead of getting capital investment or self-financing from your last achievements?
"We want you to do this."
"Sweet, easy enough, I'll just use X, Y and Z, no problem."
"Oh but sorry you have to use A, B, and C instead. Oh and it'll need to connect to D for reporting, and we'd like it to sync with E which we've never done but they have an API so it can't be too hard, right? Oh and it needs to fit this design we've already approved that's about as far from native to the platform you're developing for as it could be and has a ton of UX issues that require fundamental changes to fix, which you'll uncover as you work. That's about as easy, right?"
"...."
[EDIT] oh and also one "minor" feature we're going to toss in later is actually big enough to build an entire company around so you'll have to waste a bunch of time talking us down from that while we quietly lower our opinion of you, and we've accidentally described a few things which are basically impossible by egregiously violating e.g. CAP theorem or being what's effectively a highly-available distributed filesystem that needs to fit without our timeframe and budget, which is 1/20 what it would take to maybe build that and that alone.
[1] http://www.bleading-edge.com/Publications/C++Journal/Cpjour2...
>because some developer who was bored with his job decided to use a blingy new Javascript framework he had no experience with because he wanted a challenge.
That definitely happens, but I think it's a different problem.
YES! The cost of custom-everything design and bespoke gee-whiz is vastly under-appreciated, IMO. CRUDy Android and iOS apps using built-in widgets and simple color theming? Relatively easy to estimate, and pretty damn fast. Custom-everything, we want both to look pretty similar (so, probably Material-ish given current trends, which you'd think would come for free at least on Android but... doesn't), custom animations everywhere, oh I hate that default date picker on this particular Samsung phone (and sure, it's god-awful) can we customize it, and the designer came up with this layout for this form that requires all kinds of twisty-bendy manipulation to reproduce versus something more straightforward but that's what the "stakeholders" approved so let's do that.
Et c. et c. and pretty soon the app's 10x as expensive (no exaggeration!) because you couldn't live with the easiest-for-the-user-anyway default styles with some custom coloring.
And the Web's at least as bad. Often you're also making your page way less usable and breaking it on certain browser/device combos due to the extra effort you put in to make it "pretty", at great cost.
[EDIT] in fact I've in the past suggested providing design points for "pretty gee-whiz versus not-actually-bad straightforward implementation" to make product owners choose between getting e.g. three normal (and OK-looking but not "pretty") features or one pretty feature in a sprint, to make this cost more visible and give owners on a budget more flexibility in shipping features, but it never caught on—in particular designers hate, hate, hate the idea, I think because it might expose that their work is sometimes (often...) more expensive than it's worth (and vastly more expensive than they're being paid, since it also eats huge amounts of developer time) given you've got some developers with even a hint of aesthetic and UX sense on a project.
These are the same people who get furious at their mechanic or plumber when they double the time estimate.
"I don't know, I've never done X before."
"Yeah, yeah. It doesn't have to be perfect analysis, just a rough idea for scheduling. Not written in stone, ha ha."
"Uh, a week?"
"Okay, great, so if I say 8 days you should have it done by then?"
"Yeah, I hope so."
"Great, thanks."
Day 3: actually doing X requires unforeseen Y and Z which will each take a month.
"We need to add Y and Z to the schedule, which will each take a month."
"There's no room for that in the schedule."
"Then I can't do X"
"We've already costed X and committed to it. We've got to do X. We need to make X happen. When can you get back to me with a path to green for X? And also, let's schedule a post-mortem to figure out why we missed on X."
At the post-mortem, PM doesn't show up: "Uh, turns out we missed on X because I estimated it too quickly and the PM committed to those estimates. I didn't know about Y and Z when estimating, and these are things I could have done to detect them as requirements if I had known to do so."
Email from PM: "Hey, could you give me an estimate for X'? Doesn't have to be perfect, ha ha."
Never say a week when there's the chance something will require a month because you didn't understand it.
You need to learn this lesson. A week is a blink of an eye in software, and only things you understand take a week.
"I don't know why, but I don't feel comfortable guessing less because I don't fully understand X."
"Okay, that's not a good way to estimate. Why don't you spend some time to come up with a plan for X and we can estimate it after that. When can I have the plan?"
I fail to see how your example makes any case on how estimating how many resources a project needs is stupid.
At most your example argues that asking inexperienced and clueless devs to estimate stuff they know nothing about produces highly unreliable information.
The main difference between hacking away at a code base and software engineering is identical to the difference between construction workers and civil engineering. Sure, estimating is hard. Yet of you want reliable estimates you need to check with experienced professionals.
The reason I think my example illustrates a problem is it's moving away from agile. I shouldn't estimate X without understanding it? Okay, I'll take the time to figure out, look around corners, etc. I realize I need Y and Z. I've got to estimate those, to estimate X, so I start working with other teams to figure these things out, and drawing up plans and schedules and who's doing what when to get all this stuff delivered. Now we're basically at the waterfall model where we're planning everything out before doing it.
The other problem is that all the process is genuinely a waste. If we need X then let me work on it rather than do all the planning and scheduling nonsense. The schedule will be wrong anyway. If I have problems, need help, etc with X then I can raise those concerns and management can react in an agile way.
If you look through the managers who are weighing in on this thread, you’ll notice a common perspective: they don’t trust you to actually work on something. That’s why I have so little hope for any software methodology - even if it starts from something positive (like XP, which was the precursor to “agile”, did), it will be turned into “I know all of my programmers are stealing from me, how can I stop them?”
I learned recently that the crane operators that unload cargo ships at some major ports can’t work more than four hours at a time. The movements are too precise and take a great deal of attention. I think you can work two shifts per day but there’s a mandated gap of some number of hours between them.
Wow, so much hubris in such as small post.
Unless you have been doing the exact same thing forever(in which case you are at a risk of being replaced), new work _always_ comes with a degree of uncertainty. Be it new technologies, new business requirements or what have you. I have never worked in two identical projects in my life.
The correct answer would be a 'spike' or proof of concept. Either that, or hire those magical developers you seem to have who know everything there is to know.
I understand this is a contrived example, but it's strange that, when you take a real project that "overran" based on the initial estimate(s), and you apply the formula - just how often the formula is actually really close to the actual amount of time.
Not always - but more often than not.
Managers are there to manage costs, to protect the "business".
I don't think any owner of a business thinks that there is luck involved in creation of their software systems. They absolutely look at the $10 million per year they are spending on "IT" and look at the $26 million per year in value that spending is unlocking in absolute dollar terms.
You mean "realizes there is luck". And hence the problem.
Note that the manager's estimate may be implicit or explicit, conscious or unconscious, but they will make one in the absence of the developer's. An implicit estimate would look like the manager doing a "gut call" when deciding whether or not to give the go-ahead to a feature, with no numbers attached to time and cost.
And that's what Scrum and story points are for: to help you make long term estimates once part of the work has been done. You use the feedback on how abstract story points turn out in practice, to get a rough long term estimate of how long the work is actually going to take. Without being so explicit that anyone can be held accountable to them.
He emphasizes a variation of the cost benefit formula + urgency, called Cost of Delay for prioritizing that basically boils down to "Value / Time Remaining" because your cost is the time remaining. It naturally prioritizes work based on shortest time to value because, unless the value proposition is enormous, dividing by estimated time is almost always going to favor the shorter tasks.
This will of course vary by team, but in general it puts a focus on responsiveness and getting the little stuff out of the way first (including tech debt). When you have a formula that doesn't let the little stuff build up, it gets easier to focus on the big stuff too. It's also make it easy to pivot because it totally disregards prior time investments.
It meshes better with what agile is supposed to be better than any other methodology that I've encountered to this point.
Tech Debt is not little, or we wouldn't carry it around, rather than devote time (and risk of revealing new unknowns) to fixing it. My agile experience has always favored conquering little stuff first, since ppl recognize stories more than points (which vary in value, over time). This, unfortunately, sometimes leads to the hilariously subversive act of splitting a story up into as many tiny pieces as possible so that some people may inflate that "What have you done for me lately" list while more tangible coding is done by others.
He also doesn't place any type of value in points or other, bluntly, useless metrics for measurement. The entire focus is around delivered value. You can break it into however many stories you want, the value is the same. The measurement of how many things you delivered or how much time they took is not important to the company.
What's important is how much value was been delivered to the company. It's the only metric that matters.
Honestly I think that's the intended outcome from the business's perspective. The smaller the piece of work, the easier it is to assign a cost and a benefit to it, and the easier it is to hand off to another developer.
I have never seen this work out in practice. And it's like a 1%-of-the-time thing anyway.
On the flipside, I've seen quite a few hours wasted adding fake subdivisions to tasks, probably close to 30% of the time. Outside of small, known bug fixes or tech debt cleanup, it's rare for a thing to wholly fit within a sprint.
As a producter, the cost of building a jeans factory does not seem to be inherently more predictable than the budget of a software development project.
Even at scale people can pull out of complex agreements if their estimates turn out to be faulty.
Imagine if that cable guy coming your house said between 1 hour and 2 weeks. Good luck making anything approaching a good decision surrounding the cable guy's visit.
I blame the ambiguous manifesto.
I agree with most of the article. Specially the keep-learning part.
All Agile did was put software development teams unfairly under a microscope.
I believe Agile has been tremendously beneficial for the industry globally, especially in some subtle ways. For example, Agile says you have to communicate a lot, if you want to get software done. Here is the subtlety: if, today, you stop telling the average programmer to communicate, they will stop and go back to silo mode, "naturally". At least some of them. If you think about software at the industry scale, you have to think about a wide population which require processes.
I was once working in a nice little company where, one day, they introduced agile. It was so much beneficial. Before, the bosses thought that talking was simply a lost of productivity. You're a programmer right? So code, don't talk. So we would develop software without talking to each other. After we had daily meetings, scrums things and stuff. Our velocity has sky-rocketed.
You have to realise that before Agile, a fair portion of all software development projects that were started would simply bust and never get shipped. The code is a complete monster or the budget is nuked. When I started in the company I just mentioned, I started working on a codebase that was the worse code I've ever seen in my life. You would touch one line and everything would stop working. It was a condensed piece of spaghetti with hacks on top of hacks on top of hacks. Software architecture? That requires some talking and thinking, forget about that, not permitted.
Now I agree with you, agile is not much useful for a hacker who consistently get software done and understand deeply what's happening.
edit: not that I'm an agile guru or anything. Actually I only know the thing superficially. I show up at the meetings, when I'm ask how hard something is, I answer, then I mess around with my office friends. Still I can appreciate it works much better than any process the bosses can come up with.
This still happens, all the time?
Imagine a system that couldn’t help you pick winning stocks, but It could cap the losers at 10% loss, maximum. You’d get rich. Very, very quickly.
Even traditional software projects are now much more incremental due to the ability to update once a week (FF, chrome, etc).
That's had an effect regardless of agile.
Projects still get cancelled, budgets and timelines get blown out, features and testing get compromised. But at least in dedicated software companies, it's pretty rare for something to get through 90% (or 150%) of its allotted development time and then be completely discarded as unsalvageable. (For technical reasons; market changes are a different beast.) Projects in technical crisis are either apparent sooner or have usable elements, rather than failing outright at the end of a waterfall.
Of course, there's a lot of room to debate what changed and why. If Agile led projects to fail sooner or less dramatically, that's still important, but less notable than if it changed them into successes. And a lot of technological advances have helped too; things that might have been outright failures as shrinkwrapped software can become late or overrun projects in the era of digital distribution.
https://en.wikipedia.org/wiki/List_of_failed_and_overbudget_...
It's a weird contrast to normal life, considering a lot of us live in democracies, but once I understood this and started acting accordingly, it helped me handle my work life.
Some are all about the politics, others don't have the luxury of self delusion or actually value the sort of brutal honesty to say "I have looked at the new framework - while trendy it is buggy inefficient crap." The ones who lack the luxury tend to be smaller but small size is no guarantee that they'll just say no to the flavor aid.
Like anything there's a grey zone here. Understanding the political realities of a business is highly beneficial and will help you move forward with the company and let you know when to pick your battles. But I've watched people who do nothing but this lose the trust of those at their level. And that trust is crucial for agile development. I also personally feel like pushing back on the company when appropriate can indeed provide a lot of value to the organization. "Why are we doing this meeting?" "We need another week for testing" "I'd like to see the roadmap you are planning". Just don't push back all the time.
As far as your career - salary and promotions and recognition - you can't just sit in a corner and be quiet. People have to recognize that you're good at what you do and it's hard for others to see when you only communicate in code check-ins.
You have to start attending meetings, speaking at meetings, and being useful beyond the code.
In all areas of life we judge those around us not by the true facts, but on our limited knowledge. Working to improve that knowledge can result in a change in judgment. It seems completely reasonable for there to be a person who does good work but who isn't know for doing such, and as such is viewed worse than they should be. By working to increase the knowledge of the work they do, their evaluation in the eyes of others will improve.
Of course, it isn't a straight forward or simple in practice. There are those who lie and misrepresent, and if you are to obvious about your intent you will be viewed as manipulative.
I don't see why this deserves either the label of BS or a ban from HN?
You need to talk to people to be able to produce the software they need, ultimately.
I think it's doable; say you are the sole developer of a massively used library. The programmers being your only users - they read the API docs, submit an occasional feature request/bug report.
One need only look at the bug tracker on the average "massively used library" to see that you will be communicating with clients a great deal. Your dream of a small group of perfect programmers filing neat and accurate feature requests or bug reports doesn't match any reality I've ever seen. To expect them all to "understand how coding works" is also likely a pipe-dream - users of all ability levels are out there writing code, filing bug reports for things that aren't bugs, asking questions in forums that aren't meant for those questions, mad at you because the direction the project has taken isn't the one they or the company they work for want/need, etc etc...
Feel ya. Surprisingly enough, I'd say that I've not had to attend so many boring meetings in my career but certainly I'm just lucky. Also, once I did get a manager who was an "agile guru" and he was annoying. He talked to me like he's the jedi of software development and I'm his padawan. Thankfully, he got fired.
-- Yoda
If you can't do that, what is the difference between you and a group of outsourced employees making pennies on the dollar?
the world needs both kinds, and a range in between.
But if one kind can do both types of tasks then you only need that kind.
If you are writing software just for yourself, there is nothing wrong with that. If you are writing software for others, be prepared to be steamrolled by people who know how to write good code AND how to communicate with consumers of the software (i.e., paid customers, OSS devs, enterprises, etc., that one will depend on what kind of software you are writing).
I think that a lot of developers with a mindset similar to yours tend to underestimate how important good/valuable feedback (and communication in general) is. And I am saying that as someone who initially went into comp sci for reasons similar to yours. Some of the best engineers I ever worked with had amazing communication skills, and it acted as a x10 multiplier to their technical skills and overall productivity.
You're more than welcome to want to sit in isolation and code away, but I would suspect you would find it difficult to do this and be happy. I've never worked in a team that would work with someone like that.
Most managers crave control over their teams; this is not something that Agile introduced. If anything, Agile let them use the sort of control they were already demanding towards more useful and productive goals, by introducing bazaar-like practices to centralized software development (release early, release often; shorten feedback loops as much as possible; make extensive use of refactoring, software testing and XP principles); while at the same time not being altogether incompatible with self-organizing development teams (these were mentioned in the original Agile manifesto, after all).
I'm extremely grateful for the changes agile wrought on the software industry, and while I think many of the best insights have become commonplace, I don't think formal Agile methods are outdated or exhausted yet. But I do think formal Agile is extremely hard to do right, and has very common failure modes of knowingly-unrealistic planning, unproductive meetings, excess design changes, and tech debt neglected to ship MVPs.
One measure of a methodology is how much value it provides when it's used right, and I agree that Agile shines here. But it's also worth asking how easy a method is to get right, and how gracefully it devolves when things aren't perfect. My experience is that agile advocates commonly neglect those parts, dismissing widespread frustration with 'bad agile' on the basis that if it had been implemented perfectly, those issues wouldn't have come up.
I have plenty of gripes with Agile, but there's definitely a "victim of its own success" aspect to the whole thing.
"Agile in name only" is frustrating compared to a good system, but in many cases it's still far better than what came before it. Basic ideas like "we should expect requirements to change" and "if the programmer doesn't know what their code will be used for, something is wrong" weren't necessarily accepted prior to agile. Projects that are chaotic, mismanaged messes under AINO might well have been orderly deathmarches or cancelled outright in the past. Some of this progress is technological (source control, post-release patching, digital distribution), but some of it really does owe to Agile.
It reminds me a bit of the scientific method. The elaborate eight-step thing taught in school feels like silly boilerplate, but it's shocking to realize that "ideas are tested by experiment" was a genuine breakthrough from a past that made major choices like disease treatments and scurvy cures based on 'reasoning' without testing them at all.
Your goal should never be to become "more agile". You should be looking to improve efficiency, to make sure you are building the right thing, to improve communication so that everyone is on the same page, etc. "Agile" is never going to magically improve anything.
Each participant of "Agile" development should have this hung on their work place.
What to do if you're on an Agile team? There's value in the Lean Methodology. Remember, the whole point was to deliver software your customer needs to fulfill their business objectives. Continually assess your Agile practice and see if what you're doing still makes sense for the project you're working on right now and for the current stage that project is in. This continual assessment with a focus on the end goal of delivering software can go a long way to addressing Agile's modern ills.
I have no idea what emergent architecture is in the agile context, but the idea of emergence is fascinating to me, and has been ever since I read "Emergence: The Connected Lives of Ants, Brains, Cities, and Software" by Steven Johnson.
We see the idea of emergent design everywhere in nature and society. I'm not suggesting everything can be built and designed "emergently", but certainly some things can be. See the book (and concept of) "the cathedral and the bazaar". The idea of something useful coming into existence without central planning is an enticing one, and so I can see the draw.
I also had not heard the term "emergent architecture" in relation to software development. I'm intimately familiar with the concept of emergent order, though, and my grokking it was part of my "political awakening" in my 20s. It led me from being a libertarian-leaning conservative to an anarcho-capitalist. To put it another way, I believe that the system create by independent actors acting in their own self-interest naturally create the most efficient system possible within the constraints of the environment within which they operate.
There's a name for this concept when applied to company management, but I can't recall it at the moment :(. I want to say the word has a Greek root, and it reminded me of "autarchy" ("self organized") but that wasn't what it was called.
There is something to be said for small government, but also can go too far: actors acting in their own best interest can obviously cause harm to society or to the larger group as a whole (See "tragedy of the commons").
Related to emergence is rapid iteration, and it's importance to analysing complexity, which is caputured nicely in "Boyd's law". See https://blog.codinghorror.com/boyds-law-of-iteration/, and then see if you can find the referenced paper by Roger Sessions. (Sessions seemed to caputure the essence of "Lean Startup" before it was cool...)
"Agile" is a lie that before 2001 nobody at all had ever done iterative software development, and all software process ideas that came after 2001 are by definition "agile."
"Agile", as with any other buzzword, is not for people who actually understand the details of the change; it's for someone who needs a handle onto concepts that would fly over their heads without a buzzword. Lets stop litigating the term, and move on.
A very good engineer within my organization said to me, "Our job is to get something in front of them quickly so they can figure out what they don't want." Our successful projects have been ones in which we kept to this maxim.
I think this is what the author was trying to say but I had a hard time following his writing style.
> both Agile and Waterfall are focused on building. Design is about validating.
or
> So…what’s the way out? It’s a smart focus on clear outcomes, not output, with roadmapped outcomes replacing planned milestones, with trusted product teams, not project teams, empowered to vet assumptions and discover the minimal path to value.
Satire? Enough words to fill a hot air balloon or two. "Smart" focus, "clear" outcomes, "outcomes" vs. "milestones"(?), "trusted" product teams (as opposed to... what?), "empowered" etc.?
Far more interesting than the contents of many links that make it to the HN homepage is discovering the fact that they do so.
I mention neither of those.
I'm not complaining about your reply, I think it shows how those texts are (to be?) used: Just like art. You project your own experiences into it, it's more about getting you to think. So naturally different people are going to react differently and read different things into the same text. That's perfectly alright - you get something out of it (only) if you put something in. If there is nothing to begin with, which may just be lack of experience (new programmer), you'll get less out of it. If you have two decades large project experience in various companies and teams you'll get a lot out of it. Let's attempt to make the best out of it then, out of any text, if we are presented with that opportunity and the headline got us to click on it :-)
Let me quote again what I already quoted:
> with roadmapped outcomes replacing planned milestones
"outcomes" vs "milestones"
See?
Original:
> So…what’s the way out? It’s a smart focus on clear outcomes, not output, with roadmapped outcomes replacing planned milestones, with trusted product teams, not project teams, empowered to vet assumptions and discover the minimal path to value.
Rewritten:
> So… what’s the way out? It’s a focus on clear outcomes, not output. There should be dates for when specific outcomes should happen. Teams should be empowered to find the minimal amount of work needed to achieve the outcomes.
It's a really wordy way to say what any good leadership class would tell you. Focus on what you want more than how you get it, and let your subordinates decide how to get it. Both good product teams and project teams are capable of working that way.
This kinda sums up my problem with endless agile vs. waterfall vs. whatever discussions. The interesting parts aren't really new ideas and the extra buzzwords add nothing.
I know such a simple rule is probably causing a lot of you to cringe, but I can very easily fit just about any activity of value you or your team embarks on into that question, and it's a rule the entire company can get behind.
1. How much value is there if it goes right?
2. How easy it is to get right?
3. How gracefully will it break down if it's not done perfectly?
Methodologies don't have to shine on 2 or 3. NASA-style clean-room coding is the gold standard of hard-but-effective; you have to suck it up and commit entirely with a great team, but the results are unbeatable. Waterfall was "good, bad, bad", but used by lots of less-exceptional teams with hideous results.
Agile, I think, improved #3, but not #2 so much. It's commonly sold by appealing to how much it can potentially help, but I think the more honest pitch is "it's tough to do right and it'll sting if we get it wrong, but it hurts a lot less than getting waterfall wrong."
Not everyone. Some are very successful with it. The problem is that everybody wants to copy that success and tries to cargo cult themselves through it without understanding it.
"let’s stop pretending Agile was some sort of cure all."
That's certainly good advice. Agile is more of an attitude than a solution, and attitude is not going to solve all your problems. Even if you are as Agile as you can be, you still need to solve your problems. It's just that being Agile might make you more flexible and effective at solving those problems.
And Lean is certainly more comprehensive than Agile. And I don't think they contradict each other. In fact, they're very similar; they're both about empowering the people who are doing the actual work.
"management thinks the focus should still be scope and deadlines and efficiency, ignoring that the deadlines are arbitrary and the requested time estimates are a form of waste."
Then management is not Agile and hasn't properly learned what it is. That is sadly common. Agile is not magically going to solve bad management, but Agile does emphasize keeping management at a distance and empowering developers.
"Agile actually tends to mask the core problem, which is a systemic, bidirectional lack of vertical trust."
Does it mask that? I think it exposes it. Without that trust, Agile cannot work. Agile insists on that trust. I suspect that every case of broken Agile is caused by a lack of trust. Fix the lack of trust (or use a system that works without trust if you absolutely must).
"Did you know story points were actually invented to obscure time and help alleviate this problem? That backfired too, didn’t it?"
Yes I did and no it didn't. Every single team I've worked in over the past 6 years knew this and kept repeating it every time someone comes along and tries to match story points to actual hours.
This first half of the article suggests the author has mostly worked in very toxic management cultures.
My last company (before going self-employed) had an agile environment and it was mostly a good system to work in.
I don't recognize the fairly strictly Agile (not AINO) environment I worked in in that article. Not perfect, but it worked and better than a lot of places that had no good methodology.
To the extent your work isn't repetitive, the only job process can do is set a cadence on communication & decisions. Yes 2-week scrum does this but in my experience it distracts companies from the real goal of management -- attaching subject matter experts to projects, i.e. concentrating time & resources towards a goal.
Process can get rid of bad people and share around your best people, but unless you have an assembly line, beware of any other claim.
That's just one example of the general value I like to ascribe to "AINO": it gives you a useful mental model of the delta between what you have and a known good process, making it quite easy to name the missing pieces. This usually doesn't make it easier to actually fix their absence, but at least you know where to start with damage mitigation. When waterfall fails, you just throw your hands in the air and say "more of the same".
I often sit between the business and tech orgs, so the way I explain it to the business is this: I can get you detailed status reports and metrics, but they will slow progress and be expensive. So think about why you need that information: if there are legitimate business reasons with dollars attached, go for it. If it’s just to soothe your anxiety about the timeline, a therapist will be far cheaper and more effective.
This is a brilliant summary, thank you.
The best 'agile' experiences I've had are situations where the 'clients' are directly involved, often within the same organization. Instead of a hard scope or deadline, there's just a shared interest in producing a valuable product efficiently, and the users are on-hand throughout the process for feedback and reevaluation.
The worst experiences have been waterfall contracts, developed by an internal simulation of agile. The software team does frequent "releases" to business or management, who provide feedback and feature requests, but the actual recipients are uninvolved outside of occasional demos, or contacted only indirectly by non-programmers. The result is almost always thrashing, with time and effort spent pointlessly satisfying the forms of agile even though the real timeline and customer feedback are unyielding.
We call this wagile
I'd say generally IME they are still preferable, but occasionally can be more painful.
> The Scrum Master serves the organization in several ways, including: > ... > - Helping employees and stakeholders understand and enact Scrum and empirical product development;
So, I am not saying that it is an easy job. Many managers just hear what they want to hear, so it can be quite difficult. But if your team has such issues, be sure to support your Scrum Master with good arguments to help him make managers understand what it means to use Scrum (and become Agile).
[1] https://www.scrumguides.org/scrum-guide.html#team-sm
https://en.wikipedia.org/wiki/No_Silver_Bullet
> So find a good booklist. Follow some good blogs. Here’s a start: If you haven’t read Sense & Respond, Lean Enterprise, A Seat at the Table, and Everyone Is a Change Agent, I suggest you do so pronto. Your leaders too.
I'd add Peopleware to that list.
> Start reading posts by John Cutler, Melissa Perri, Bob Marshall, Allen Holub, Laura Klein, Erika Hall, Neil Killick, and branch out from there.
Anyone care to add their own handful of books and bloggers to these recommendations?
And it turns out that it's not unusual for software developers (and consequently software development organizations) to spend more time in the we don't know what we're doing phase than the we do know what we're doing phase. That sucks from a management point of view, but no amount of process is going to make the problem go away. The best you can do is manage the risk. We don't need good processes or methodologies nearly so badly as we just need management enlightenment about how software development actually works.
For me, I tend to work quicker than other contractors / employees, so I still often beat out the other quotes (time or money). That’s not always the case for others, so they margin of safety may not be there. In which case, multiply by a smaller constant.
It's important to note that padding schedules is not just done in software, it's a common practices in all sorts of engineering. It's not bad that we do this, it's bad that management often doesn't understand that we do this and doesn't understand why. The understanding why part is actually really important, because that means management can be expected to do their part in managing the uncertainty or at the very least they can be expected to be careful about not making the situation worse.
For example, if you ask, "How long will this project take?"
You could just say, "I don't know. At least two weeks, but it might take as long as six months." This may be accurate but it's not helpful.
Or you could say something like, "In a half day I could put together a preliminary design, and identify the most uncertain parts of the project. From there we could figure out what is needed to tighten the schedule or adjust the requirements."
And I think that one of the more common problems with managers in larger orgs is that they can't tell the difference between somewhat arbitrary deadlines (we want this for Q3 to look good in our org, but we can break this project into smaller milestones and push it back if necessary) and deadlines that reflect the reality of business (we want this for Q3 because this will have cascading effects on other parts of our business, and if there are problems we need additional resources or an adjusted scope).
Any time there is a deadline, the manager should have a clear understanding of the consequences of missing that deadline, and be able to communicate that to the engineers. The efforts to come up with a schedule and milestones should be collaborative. Too often there are mandates that have unclear consequences for missing them, and when schedules slip you have no idea what you are supposed to prioritize.
Funny thing. If you do this organizationally, then you eventually end up making larger and larger estimates. IMO it is better for management to understand the inherent risks associated with engineering estimation and build the padding on their end.
Ironic that so many Agile coaches like to bandy about Dave Snowden's Cynefin and then act entirely opposite of what it recommends.
> The Cynefin framework is a conceptual framework used to aid decision-making.
Among other things he speaks about the simplicity of the Manifesto, about the cultish movement and consulting services which rose around it and the cultish, distorted, complex vortex of BS that has formed around a super simple concept.
https://m.youtube.com/watch?v=a-BOSpxYJ9M
I've seen software development done very poorly and done very well, sailing under a variety of flags. But on an eyeballing there are far more bad experiences than good that get labeled "Agile".
Whether this is because of causal differences or because it's easy to affix any label to anything (and most experiences suck regardless of the label) is hard to tell without a control-group universe were the Agile Manifesto was never written.
If anyone needs me, I'll be counting to twenty.
And tbh, despite how bad “Agile” process can be, it’s a lot better than pure waterfall.
Sometimes it goes off the rails. That is in the nature of things. But it goes off the rails as an exception, rather than as a rule.
If there's a key to the magic it's (1) hire for empathy as well as smarts and (2) reflect on what can be done better.
For what it is worth, I have also seen classic waterfall be successful. But I have also seen several big-scale disasters with waterfall, and in my experience waterfall carry much larger risk and is much harder to salvage when it goes off track.