HN Explorer

How to Help a Group Have Useful Postmortems?

2 points by meristem ↗ HN
I consult for a startup that wants to begin having postmortems. In this particular case product and engineering people will be at the table. The desire to create a retrospective reflection and learn from it originates from a place of surprise and frustration. Everyone is walking in with some negative feelings. LOTS of feelings.

This is the first time I will be the only person who has been in such a meeting before, and there is groundwork I want to do before the meeting.

If you were involved in starting postmortems or were around when they started happening, 1. What were the main roadblocks? and 2. What was specifically helpful in creating an environment where people believed they were not there to be blamed?

4 comments

[ 5.0 ms ] story [ 19.9 ms ] thread
tossaway9000
Start off by saying that the goal of the post-mortem is not too blame anyone or point fingers but to find out where improvements can be made and where downtime and incidents can be reduced. Every time a team is paged at our company we have a post-mortem meeting to review the incident. If we have a lot in a specific week we do them in batches when possible. We pull in anyone involved with identifying, triaging, or resolving the incident and create follow-up tasks based on the meeting. You might want to try the Five Whys approach as well: https://en.wikipedia.org/wiki/Five_whys
Jugurtha
Templates are useful. We have templates for our issues (features, bugs, and incidents).

This enables us to open an incident report when we notice something, and add to it as we troubleshoot.

One thing that is paramount is setting the example. The most senior people should set the tone and lay it all out when they've caused an outage or something.

There's a reason we reject job applicants during interviews whenever there's a hint of fuckery indicating they will try to avoid accountability or lying. We estimate that if they're doing this in a low stakes situation on questions we already have answers to, they won't hesitate to lie and hide traces during real incidents where things are blurry which will slow down fixing something, recovery, mitigation, and then prevention of future similar incidents.

Senior people should overshoot in honesty whenever they cause something to break, and establish that the important thing is to recover and find a way for this not to happen again. This will be replicated by others.

The issues and incidents reports help clustering issues and solve underlying problems. For example, when you do incident reports, patterns emerge and will lead the team to ask better questions to solve a more general problem.

Establish that the point of these reports is so we won't have to deal with this in the future. This is for our sanity.

I think we have established a good dynamic where there's no finger pointing, because very early on, team leads pointed the sharpest fingers against themselves and laid it all out there for everyone to see.

This does not prevent people from having feelings when they break something but they have eyes on the prize (mitigate, recover, identify the problem, fix, prevent by making it impossible to have that problem again or document it). We get there faster that way.

runawaybottle
Post mortems are shame-theater. Real life has a version of TLDR, and it goes roughly like this: ‘sorry, we are going to try this again with a new set of people’.

It’s rough. No need to be dramatic and have school plays every Friday. Move on.

Pick the new direction and people, and give a few of the older group a chance to stay on in a fitting role. Let those who can’t handle it move on. Don’t waste time, as we can all be elsewhere.