Ask HN: Do You Use Waterfall?
I was just reading an article on Monday.com (https://monday.com/blog/project-management/waterfall-methodology/) that referenced pmi.org and stated:
"According to a study by PMI, 56% of projects used traditional — AKA Waterfall — methods in the past 12 months."
I found this surprising, as all the companies I've worked with have used Agile methodologies.
Perhaps I've been living in a bubble... so my question to HN is:
Do you use the Waterfall process at work? How do you find it? When would you advocate for using waterfall, and when would you avoid it?
17 comments
[ 3.0 ms ] story [ 63.1 ms ] threadHave they really? Or do they do Waterfall with Jira™? I ask because I've consulted on multiple projects where they called their process Agile, because they did some things and called them by their Scrum names, or they had Scrum Masters, but with a bit of critical distance it was easy enough to see they were doing waterfall with extra steps.
No burn down charts for me :)
Agile makes it really difficult to get paid by milestones, and unless you have enough cash to float the entire development team's salaries until the project is finished, agile is not exactly conducive to those types of contracts.
I'm not against Agile, btw... I just get fed up with fanboys who are dogmatic about agile being the only, best way to develop software, and yet they have no real-world experience with large projects that must interact with other systems. Agile is great for when you don't know what you want to build. It's pointless when what you need to build is already well-understood and specified by contract.
In fact, this is where agile (real agile, not Agile-in-name-only) breaks down. Somewhere up the org chart, it reports to someone who is not agile, someone who thinks in waterfall terms. That impedance mismatch either takes a huge amount of translation work by the person below them, or else it eventually kills real agile, turning it into waterfall with some agile lipstick.
When a project hit a certain size and have a strict budget with hard deadlines (where hard means: political party wants it done before their term ends or there is a time restriction on the budget's availability) agile just won't cut it. You cannot say at the end of the 1.5 year project that 40 programmers and managers worked on that "yeah sorry that last dozen features and 100 bugfixes did not make it to the last sprint".
Also when there are several parties and subcontractors involved the "big design up front" stage is not optional anymore.
QA is usually a third party who gets paid to find out if everything is according to the functional spec and documentation and enter at the later stage of a project.
You simply cannot do agile in such an environment - though of course the documentation will contain the fact that you are indeed agile :D
Rarely do I see a company that iterates over a feature until they get it right (See XP). Rarely do I see a company without a roadmap and people discovering and developing a feature together at the same time.
(Shameless Plug: It's so sad that I wrote about it in "Why we always end up with waterfall" [0])
[0] https://www.amazingcto.com/why-we-always-endup-with-waterfal...
The "waterfall" of today actually makes sense for large projects where you have a good idea of what the end product is going to be.
"Agile" as it's used colloquially only refers to the ability to respond quickly to unforseen circumstances, which I think is the better definition.
I'm imagining UML diagrams, control flow charts for the old days vs UI mock-ups and Jira tickets today, but not quite sure how far exactly today's waterfall projects go in defining the work to be done.
Next, the analysts would begin the requirements analysis. They would talk to their customers to ask things like "What would you like to see feature X do?", write up ERDs and DFDs, flowcharts and such, and then come up with a huge document listing all of the requirements of the software. At no point in this phase would an engineer or even the prospective user of the software be consulted.
As an aside, UML wasn't really a thing yet, and mock-ups were rare. Information would just be crammed into an available space on the screen or perforated printout.
Next, the architects come together to decide upon technologies and the overall shape of the software. This was done in a vacuum, with no input other than the requirements document.
Next, the systems guys would decide how the rubber hits the road, injecting their requirements for the system this will run on.
Next, the programmers would implement what the previous guys had output in their documents.
Once all that was done, the testers would come in, use the software and compare to the requirements.
Then the software would be shipped to the customer.
Note that nobody lower would communicate directly with anyone higher in the waterfall.
If at any point in this process something was found to be horribly off, you would decide whether to conceal it to stay on target, or to raise an issue. Once an issue is raised, you go back up the waterfall until it reaches the person responsible for that thing, and then the documentation process adjusts as it trickles down the waterfall (this usually takes a couple of weeks) before you continue where you left off.
The main issue is the lack of communication. Those higher up rarely have any interaction with those lower down. The company has very little contact with the customer, other than to ask questions and then write requirements full of biases and assumptions. There's high pressure to stay the course and not change when the product doesn't satisfy because that would lose money. This is why the customer was asked to sign off on each phase of the waterfall (containing thousands of pages he didn't understand).
And the thing is, this worked (in the sense that software houses made money, not that customers got anything wonderful) because everybody did it.
The changes since then are not only agility (although those are huge wins). There's also MUCH better communication - keeping everyone in the loop (including the customer), observation-based analysis (watching the user actually do things in their day-to-day as an input to requirements, and a discussion springboard for innovation), a/b testing, automated testing, continuous deployment, ticketing systems, UX, etc. All the good stuff that we just take for granted now.
The "agile" manifesto was a good catalyst to modernize software engineering, but it's not an endpoint, and rigid adherence to an ideology is what caused all the mess in the first place.
Except as others stated, we use Jira and have a Kan-Ban board...
We were doing this very long drawn out milestones and now have switched to releases, which eases it up a bit, but my god it is just terrible.
Going off of ideas, designs, and requirements made four years ago just doesn't seem to work out. So weird!
I still have nightmares from some of the horrors I've witnessed.
Burn-down charts, jiras, gantt charts are really just plans of how much work is left, what will it cost, and roughly when will we finish.
Agile is what happens when you do waterfall but fail to actually do a proper review and assign relevant responsibilities. Its a way for software developers to feel like they're engineering, while also giving them an excuse to not do things the way their Dads' used to ..
SPICE means Software Process Improvement and Capability d-E-termination (yeah it's stoopid), but is used e.g. as ASPICE in the automotive industry and is a framework for large enterprise companies to establish policies, roles and responsibilities across their staff management tools. [1]
The SUSE article is actually a nice overview highlighting the processes and workflows and how they iterate on work products. [2]
[1] https://en.m.wikipedia.org/wiki/ISO/IEC_15504
[2] https://www.suse.com/c/an-aspice-overview/
When things deviate from a set of requirements, we can review and adjust the requirements accordingly so our expectations in working together can stay in line. We don't have a luxury to constantly keep changing things after releasing the software because a we work in a more regulated environment where changes will need to be reverified and validated before release, not just by ourselves, but by our customers. In the projects, we are heavily involved in ensuring a good user experience and integration testing of the software to ensure it meets our needs and solves the business problems we have.
I think it should be used when there is more than one company involved in developing an application between the companies at least, and should be avoided when exploring different design ideas and gathering requirements and scoping out the project. In any project, I strongly advocate towards doing whatever it takes to expose problems and areas of uncertainty that should be scoped out as soon as possible. It's a lot more expensive to build a solution around an existing solution than to incorporate it as part of the solution from the start.