Checklists are criminally underused in most industries. It's a testament to how great they work when looking at highly-antifragile contexts: aviation, military, etc., where a mistake can (and often does) cost lives.
The problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
Which is why programming, reality, and even recipes are so hard.
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
That's one of the reasons I loved Ansible from the moment I saw it. As the OP points out, traditionally machines accumulated ad-hoc changes over a long period of time. Describing the "known good" state and running this "checklist" to make sure it is in that state both documents the checklist and evaluates it.
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
I made a checklist to migrate storage from one host to another recently. Like TFA mentions, it ended up being incomplete with some key details missing. But I filled in the gaps as I went and saved the checklist for next time.
Tim Harford mentions in his podcast "Cautionary Tales – A Fascination with Failure / Death on the Dance Floor" that checklists might have prevented the 1981 skywalk collapse in Kansas City Hyatt Regency Hotel (killing 114 people).
It's really a good podcast (pretty much all of his podcasts are)
A checklist can be gradually automated, step-by-step.
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
I've been fascinated for checklists for a long time, and I've been thinking about creating this "checklists on steroids" app, where checklists can be shared, created from templates, executed (and information about each executing collected), etc.
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Checklists are extremely hard. Writing them requires the skill of the programmer and direct response copywriter, and few people have both. Ensuring that people actually use the checklists is brutally difficult. And creating systems to improve the checklists as they are used is even harder. I suspect checklists being so hard is the reason they're so rarely used.
I have be wondering about do-not-check lists to go with checklists.
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
I learned about checklists (versus to do lists) from Dr. Atul Gawande. He has done a great deal of work on checklists in medicine because of the complexity in the field and as a way to force/facilitate communication.
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
When business approach me asking for automating a business process, I usually ask them to do the process in manual steps and create check lists for it. The check lists force them to think it through and are the agreed upon steps to perform the process. After they do it by hand for a while and find out where the pain points are, we can then automate. Since the check lists are already in place, automation is really easy.
One thing that might not be obvious about checklists is how they're used.
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
Many people in this discussion seem to confuse checklists with to do lists.
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
I use checklist as a pilot, but trying to introduce the same habit to software engineers are hard.
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
Checklists are one of the most powerful universal tool I know for avoiding/minimizing disasters/failures. In the context of software, I try to create user interfaces as masked checklists, because it forces making critical steps/decisions visible and clear (common domain specific understanding for both developer and end user), and guides user to goal. It's almost like old-school "wizard" UIs, but professionals in mind instead of beginners.
Also, critical business processes are great expressed as checklists.
21 comments
[ 5.8 ms ] story [ 38.8 ms ] threadThe problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
But any checklist needs to be a living document that has is easily updated and under version control.
Personally, I used MyLifeOrganized documents stored in Subversion.
It's really a good podcast (pretty much all of his podcasts are)
https://timharford.com/2023/07/cautionary-tales-a-fascinatio...
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Would be grateful for any feedback!
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
[1] https://www.npr.org/2010/01/05/122226184/atul-gawandes-check... [2] https://www.youtube.com/@CaptainSteeeve
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
"plans are useless, planning is essential."
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
Of course, these checklists manifested as Excel spreadsheets.
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
Also, critical business processes are great expressed as checklists.