Ask HN: Better tools for the software requirements / scoping phase?

DDD, ubiquitous language and bounded contexts in particular, can be enormously helpful with defining better requirements.

I'm not so sure about BDD though in this context. While the notion of the customer / product owner writing specifications in this format in a way developers can use these specifications to test their code sounds great at face value I have yet to see a project where this is done consistently and continuously.

Moreover, some types of requirements can be better explained by using diagrams or UI mockups, which doesn't really fit the BDD paradigm.

[1] https://designsprintkit.withgoogle.com/

I almost have guilt over feeling like my work of design and scoping is effectively useless to a developer, all my mockup layouts have to be built ground-up, my requirements aren't actionable in any way unless they feel like reading them (I try to keep them as succinct as possible, but the nature of working for clients also means I have to be somewhat specific so that people know when they should actually pay us). I've looked into things like Cucumber (https://cucumber.io/) so that my requirements can actually be compiled as tests, but adoption is slow and arduous, and all I'm really doing is adding more work to a dev.

My latest line of thinking is that I need a way to show the user interface, and then the data flow and logic all the way back through the system (usually a back-end DB or a customer legacy system). It's vital that these are presented together, hence my current process is interactive mock-ups built in Sketch (https://sketchapp.com/) and hosted on Invision (https://www.invisionapp.com/) which allows the customer and developer to click around and see it on a mobile screen so they really get a feel for it. Finally I couple that with a BPMN diagram which has swim lanes not just for the traditional system swim lanes, but also for a user (i.e. User taps Submit) and for a user interface (i.e. shows the mockup screen that is displayed), and then the logic flows down through the diagram. (e.g. User, User Interface, Mobile App Logic, Server Logic, Server DB, etc.)

From an logical point of view I think it makes a lot of sense to start from clearly documented requirements, then work forward to design, implementation, and testing with links back to each requirement. I don't know how testing (higher level than unit testing) can really be effective unless they have requirements documents to use as their starting point. Use Cases go some of the way to providing this information but tend to be a bit less formal. Agile methodologies tend to be less formal than older methodologies that emphasised this kind of process more. A lot of Agile is good, but it does also tend to forget lessons from the past.

then as the document is discussed and altered by the owner the use cases alter and flow

It's just trying to keep a prose discussion in line with everything else in dev friendly manner (it can be stored in the docs folder in git and be generated onwards etc)

still playing

It's purpose is to capture design-rationale and bring it to life so that as you refine your reasoning, the changes ripple through your decisions. It can help communicating complicated design decisions that can be awkward to capture in prose.

It's intended to be very low-friction so more like markdown than old requirements management or decision support systems. My dream is that it can help us tackle decisions so complicated we usually give-up e.g. when multiple decisions impact each other.

I am currently in beta and I'd love to talk to anyone interested in the topic. Please drop me a line at duncan at thorny.io.

You make a minimal cost mockup ASAP, for the client to try out, to see if it's what they wanted. Clients can't appreciate requirements, so (like, actual) architects make a scale model first.

The alternative, of doing requirements as a separate phase, was ridiculed as the "waterfall model". In reality, there's interactions between the so-called phases of req, spec, design, code, test, maintain etc.

The truth is that understanding the problem is most of the work - not just for the programming problem, but for the business problem. It's just difficult. And when the world changes, you just have to change with it. If you try to anticipate what's next, you'll invariably get it wrong.

Because the world is changing faster, software develoment has gone from beautifully engineered software that just keeps working, to slapped together solutions, and a fulltime team that runs alongside, slapping patches on patches, continuously.

What's the point of spending the time and money on beautiful engineering, if it's going to be scrapped tomorrow anyway (or even before it's finished)?

The only hope is for tools, libraries, frameworks and languages, that address lower levels of abstraction, that change less frequently. This isn't all good, but the JVM is one example.

Bonus point for the interactivity: I can share the URL for remote work sessions, and the stakeholders on the other end of the line can see me create/adapt wireframes and notes in real time, while I'm explaining everything on the phone (using anonymous access for those not logged into 'The Google').

Google Docs has version management since this year (I think), so I can pinpoint the evolution of specifications over the time. When I'm done I'll just export a PDF and attach to a "Spec done!" mail to everyone.

I would be happy to give you a quick demo!

https://www.delibr.com/demo

If that is the case, surely we can find better ways to uncover the requirements, and better tooling will help solve the problem.

Experience tells me that people don’t know everything beforehand. Thus the key assumption is not valid.

Then the question we should be asking is: how do we most efficiently bring people to where they discover and understand the requirements?

Experience tells me people are much better at giving concrete, specific feedback to a running system than to an abstract requirements document.

Hence iterative development.

In essence requirements are not a document by a process.

With most programs, you'll spend half your time battling its UI and trying to get around limitations.

Use UML, SysML, etc to diagram out your requirements (starting at high level at first, more granular as the system matures). Now build a high-level model of your system design (software and hardware if required). Match parts of your system to the requirements. This lets you see gaps in two directions: requirements you haven't addressed (because they aren't connected as "satisfied by" anywhere on your model), and questions to ask for more detail on a given requirement that could drive the design.

As others have said, there will always be late requirements that change things, so it won't resolve those. It can, however, show stakeholders how much changing a requirement can cost in terms of rework/time by showing how the model has to change to accommodate.

The caveat with this is: the model needs to reflect your actual design, and you need to keep it up to date. The times I've used it so far it has been a useful exercise.

In short: if you have good people it shouldn't matter! Experienced people will extend the assumed scope beyond the stated requirements without being asked, and either do so efficiently within the resources available to the project or bring an appropriate level of attention to the limitation before it becomes an issue.

The somewhat less PC and far snarkier explanation, in the immortal words of twitter: Don't pick the right tool for the job, pick the right framework which secures you extra work for being a tool. - @iamdevloper (via http://github.com/globalcitizen/taoup)

http://hitchdev.com/hitchstory

The time when it was most useful as a "BDD tool" was when I was working with an extremely technical stakeholder who was proposing behavior in a command line tool he wanted.

I simply wrote 'tests' that described the command line tool's behavior and showed the YAML to him. He corrected the mistakes I'd made by misinterpreting his original requirements and then I built the tool to that spec and when it passed I was confident I'd nailed his requirements.

QA picked up bugs afterwards but they were all either (quickly rectified) mistakes he'd made in the spec or environment issues. I had zero spec<->programmer communication issues even though (and here's the crazy part) the domain was very complex and I didn't understand it. It had to do with a highly customized software system I didn't understand which enacted some sort of financial process which I also didn't understand.

Cucumber can do this in theory, but in practice the spec is not sufficiently expressive and the stories end up being ridiculously, unusably vague. Unit tests could also do this in theory I guess, but good fucking luck getting a stakeholder to read them even if you do manage to write them "readably".

I'm taking this process a step further. Although these YAML specifications were useful for me in the past to collaborate with stakeholders they're still not amazingly readable. For instance, the "YAML inheritance" makes it easy for programmers to maintain but harder for non-technical stakeholders to understand.

Instead of sacrificing maintainability for readability I created a process to easily generate readable stakeholder documentation from the 'YAML tests'. I used this in the libraries on the website above to generate always-in-sync-and-still-highly-readable documentation.

I think this could be used to essentially write low level "unit" tests which generate documentation which stakeholders can interpret (e.g. defining the behavior of a complex pricing algorithm) and use that to get a quicker turnaround on understanding, defining and modifying desired behavior in complex domains.

I believe if you can get the small team scenario working over-and-over again scaling will work itself out. So far I have no reason to believe this isn't the case -- and I've applied the principles in the book both to functional coding and program management.

mjul's comment is the key one: you can't know everything before you start. That doesn't mean you can't know anything. It means that there is a "progressive elaboration" that has to happen on an as-needed basis. Otherwise you're stuck either not knowing enough to get going -- or having created a monster or a tools/information system that ends up running the project instead of the team.

There are some sanity checks for whatever backlog/requirements system you are using. Instead of my continuing to pitch, I'll just list the things that your system should do no matter what kind of system you have.

- Handle whatever detail is needed before actual development happens.

- Be able to "flip-around" and start from scratch within an hour or so. (And "starting from scratch" means beginning with nothing and ending with the team starting coding) While keeping all that detail in test 1.

- Be reusable with other teams doing similar work. A backlog/requirements system can't make work fungible, but it can enable better conversations in other teams without being a burden.

- Limit meetings around organization to under an hour or so. Yep, you gotta have those "meta" meetings from time-to-time and talk about things like release plans. But they shouldn't take over your afternoon.

- Help the larger organization (if there is one) learn and grow over time. Orgs learn from the bottom-up. Good tools should facilitate this learning.

- Drive directly to acceptance tests. Good backlogs are testable. Things shouldn't be in there that don't drive tests.

- Have controls in place to prevent abuse. As soon as you create some tool for the requirements/scoping phase, somebody is going to go all architecture-astronaut and overuse it. There has to be controls to prevent this from happening. The tool should facilitate the work of understanding and scoping, not replace it. (I think even a lot of tools vendors get confused on this one).

I even wrote an analysis compiler that demonstrates all of this as part of writing the book. So if you think all of this is impossible -- happy to do a demo.

There are a few other testable criteria for whatever your requirements/backlog system is, but that should be enough to get you started deciding whether some system is better or worse than another system.

Transferring valid understanding into the final artefact is a key constraint in many projects (reading Goldratt and thinking of this transfer of understanding as a constraint was helpful for me)

There are many ways to fail. Some of the traditional ways to succeed are

i) rely on an individual who really understands the desired tool, and who has the authority and skill to communicate and be the final arbiter (including sometimes to write it all themselves). Sometimes this can be also done by a small group.

ii) write a really clear, well-written, very hard to misinterpret document and get professionals to develop and test this system (before the document goes significantly out of date)

iii) Run an agile process where the business can describe what they want in small pieces that can be delivered so quickly that little understanding is lost

Obviously what works is massive contextual, depending on the domain, funding, resource reqts etc. (Glen Alleman is good on this)

So I would argue 'good software tools for requirements' are critically dependent on your approach for how you are going to turn 'understanding' into a 'system', and you don't want to worry too much about them until you are happy with your approach. At that point you can start building your 'meta-tools'.

I would argue that the reason things get called out as poorly defined or change is because risks are not addressed early, and hypotheses are assumed to be theorems.

Make sure your teams test the main assumptions early, with actual code if at all possible. That will call out why your stories aren't clear enough.

Tools are useful, but they won't solve your problem.