Would you mind elaborating? In particular, do you refer to the technical people ("no need to take care, even if I mess up no one will blame me") or the managers ("those cowards just want to avoid being accontable for their mistakes by blaming the process")?
My last company had a culture of blameless post mortems, and also a strong culture of accountability, so definitely no one would fit the first definition. And the company was too small to have non technical managers.
My current company (big corporation) officially supports blameless reaction to incidents, but internally has a strong culture of cover-up and blame ping-pong, but I still cannot think of anyone who would believe "blameless means no accountability". It just comes from deeply ingrained habits (lots of people do their whole career inside the company, lots of them even getting their masters at the company's own "university". Behavior is hard to change when everyone comes out of the same mould). I also think it comes from inadequate processes: where in the previous company the postmortem was mostly a meeting done by and for the technical team, my new company has a strong culture of "escalation meetings", where the highest managers quickly get drafted in, which obviously has a strong influence on the capacity to conduct an open and blameless process.
I find most “philosophies” regarding work to be BS. Honestly most teams just get too big. People don’t know each other and this allows things to go unseen for too long. When people know one another and work together in real time, and have healthy relationships, they can then evaluate each other’s work, rely on each other, and generally get the job done with a minimum of fuss.
I fully understand that some people are not looking for that kind of thing at all. Many would rather be nameless cogs, and some would simply rather not do the work (technical or soft skills). That’s fine too. For that latter group, whatever the management buzz words de jour are are likely to be employed where you work sooner rather than later.
Ye when org. and team size grows "accountability" is not practical anymore since the decision makers have no clue who to blame in all but obvious cases. And the workers start to do what they get paid for, not what the nominal culture dictates.
My take is that it is probably better to accept bad workers in big orgs rather than trying to reach some kind of competence utopia, to get the incentives right.
> too many people think “blameless” means no accountability for doing bad work
No, that's still somewhat blameful.
If you had stuck with "blameless does not mean no accountability", I think you're right.
But the emphasis of the article specifically does not assume people do bad work on purpose:
> assume the individuals involved had the best of intentions, and either they did not have the correct information to make a better decision, or the tools allowed them to make a mistake.
I'd like to hear other takes on how accountability plays into blameless culture.
"The org’s leadership was clear that the individual was not fired for making the mistake, but for the attempted coverup."
Somehow I have a hard time believing that. The author using "individual" in prose is a give away for militaristic leadership style, where you quickly learn to cover up any mess up so the direct superiors don't lose face.
True, it still fails the game theory test. You should still assume the need to cover up, because it might work, and in a (common) poor intitutional culture, save you from reprimand/job loss. Worst case, you are found out, and fired, but you had the chance to hide or obfuscate the fault. In a common blameful culture, you may have been blamed simply for (being seen as) the one causing the fault, whether volunteered or hidden.
If you cover up in a genuinely blameless root cause analysis but are found out, the blameless culture would see that as a systemic fault (them not communicating the importance of blameless root cause, like the fire chief of the example), and if not found out, it is merely a less effective analysis. It is an unfortunate upfront cost imposed by the proponderance of BF cultures on BL ones.
Like a market for lemons, it means you should assume CYA until a blameless culture is clearly and overtly demonstrated, not merely stated.
Ye. Being fired for a "cover up" kinda validates the employees assumption that the author's boss at the time was the wrong kinda guy to be honest to.
Unless the sysadmin got 1000 mails because the fired guy did "sudo rm *log" trying to guess the admin password I don't even know how you find out if something is a cover up, a mistake or a set up.
"shortly thereafter termination of their employment"
Most of all the author's gratitude toward the patient fire chief
whose leadership example he praises.
We could do with more of this in cybersecurity.
We're all surrounded by insecure software and defective devices
churned out by companies chasing a quick profit. We're mired in
complexity, non-determinism and broken systems. The experts who
designed them throw up their hands in confusion. And we're hamstrung
with laws and policies, which despite their laudable intentions only
beget twisted systems filled with perverse incentives and loopholes.
How do we solve this? Blame employees! And then humiliate them! Make
our cyber team into a punitive, even vindictive internal police force?
Karen from HR is sobbing into her cardigan sleeve because ICT caught
her out in a "Phishing drill". Now she'll have to go on a humiliating
"this is how a computer works" course - despite the fact she was
programming before her IT manager was born. No doubt she'll get her
own back in time.
As ICT and cyber teams we could learn a lot from more traditional
organisational culture.
I'd love to chat w/you about this re: cyber. I am at a software vendor in the incident response space. It's always good to know other folks in the industry and collaborate on ways to help improve what I'll rephrase as "the human element" in the broader cyber space.
On my part I avoid any feelgooderies, and one should have a case in court on why a mistake took place.
Why it should not potentially affect compensations on progressions is beyond me. It might not as a singular event but but patterns of risky or disorganized behavior are commo rather than individual mistakes. Errors come in bulks but they might not express at the same time. In my experience there are people that do zero,every blue moon or infinite mistakes,
and there are people that are good in zero,one or very many domains.
What an organisation needs for growth and coworkers need for sustainable stress reduction is the minimization of mistakes/output ratio.
If people get blamed a lot they switch to stereotypical behaviors and cant work in an innovative, creative fashion.
A mistake took place? Is it institutional, personal? What other mistakes would present themselves if that mistake was a pattern? What is better for morale than finding another mistake due to analysis before it expresses itself?
But the blameless culture as highlighted in the article does not prevent to identify when someone messed up personally or is not doing effort to follow the process/bring the results. It means to start by assuming the best of intentions, and see how one can avoid repeating the same mistake in the future. See the example with the fire department in the article.
Note that you bring in the example of a court, and this is how a court works: a suspect is presumed innocent until proven otherwise.
The main point is that a blameless culture fosters honesty and team work. A blameful culture fosters covering up and lying to look good to the manager. But if someone is careless, they will get into trouble in both cases. See the example with the person who deleted the production DB: they would likely never have said "guys, drop everything you are doing, I deleted the prod database and need help" in a blameful environment - even though this is the best reaction to have for the business. But even in a blameless culture, it takes a lot of courage to admit such a mistake and ask for help. Assuming that people are not afraid to do such mistakes just because they will not be immediately fired for it is quite absurd.
One overlooked aspect in punnishing companies is that the workers getting the most done also cause the most bugs and trouble.
They are essentially filtering for the very same "C players" they despise and mock.
Like, the output disparity in bigger teams is extreme in programming if you let people loose. And you better not stick your head out in bad workplaces and the advice should be to do as little as possible to avoid being fired.
Quality is obviously a thing your team should care about, but quality is also a process problem. If an individual continually produces poor quality than the "process" should fix it via not merging their code or pip or something else.
What we are avoiding here is a CEO asking, "What caused the outage" and your manager responds "Bill wrote some code that caused a deadlock". It's possible Bill is working on fragile parts of the system, or writes most of the code. Blame at an individual level for an incident is unhelpful and, as OP points out, possibly creates a team with worse delivery incentives.
quality is not a process problem, process is generally put into place to protect against bad decisions which often also constrains good decisions (because it typically does so by constraining decisions).
Let's say "done" as in God descends from the heavens and estimates every Scrum® team player's work output on a fair scale, to avoid going into meta discussions about almost unmeasurable value output.
I like the expression "Bad News Fast": it gets the message across very efficiently, and if it's correctly incentivized (even just verbal praise) it can be a powerful tool to have an accurate view of the situation.
Practising the workflow itself can be helpful too.
Some years ago, we had an "incident" where one of our developers discovered he had admin access even though he was logged into a different account of his. It was a multi-tenant SaaS.
Since he was supposed to be admin anyway, I wondered where the harm was.
Now, though, I think he was just practising the blameless incident response in case something serious did eventually come up. Once that happens, you don't have time to practise in peace anymore.
> He very likely has no idea how much impact he had on my career, but I’m very thankful for his patience with me that day.
There have been many folks like this, in my career. I am humbled and grateful.
I worked for a Japanese corporation, where they didn't "blame," per se, but did figure out if someone was Responsible. If so, that someone was expected to cop to it, and the team would then close ranks, and fix the issue. Lesson learned.
I do sometimes wonder if the (American) culture of blamelessness has led to a general lack of a feeling of responsibility among software engineers, leading to general carelessness. The idea of responsibility that becomes opaque as you move up the hierarchy sounds good, but I also expect that also has a lot to do with Japanese culture.
The US has a very strange cultural approach to apologizing and accepting Responsibility.
I know that part of it is lawyers and police. If you say "I'm sorry," in many contexts, that puts you on the hook, legally. Most lawyers will advise you to keep your mouth shut, and never, ever admit a thing.
I was taught to promptly admit when I'm wrong; regardless of the consequences. Also, to not cop to stuff that I'm not responsible for.
I have often had arguments with (American) people, where they are 100%, without-a-shred-of-doubt, wrong. Often, embarrassingly so, yet, they Will. Not. Admit. It.
I could bring the spirits of Carl Sagan and Stephen Hawking in, to show them they are wrong, and they will still never admit it.
It's actually kind of sad.
My ability to apologize worked extremely well, in the Japanese corporation. They respected it, and I was given a great deal of deference.
On the other hand, the Americans tended to be pretty much dicks about it. If I apologized, they usually tried to use it to gain some leverage, and bully me.
Inability to admit wrongdoing is just one of many diagnostic criteria for antisocial personality (sociopaththy). In my experience its only mid level managers and up that are explicitly trained not to admit wrongdoing in the US. I've always felt free to do so and all the good engineers I've worked with have done the same.
> I do sometimes wonder if the (American) culture of blamelessness has led to a general lack of a feeling of responsibility among software engineers, leading to general carelessness.
I don't think there's some general "(American) culture of blamelessness," and I think your comment might misunderstand the point of it.
My understanding about "blameless culture" is it's all about avoiding counterproductive things like scapegoating, blame-shifting, defensiveness, etc. Removing those things encourages the full cooperation of the team in resolving the issue, by removing the anxiety of punishment for being the guy who's being blamed. Everyone's supposed to take responsibility.
I understand the point of blameless postmortems, thank you. I am suggesting that Americans in particular take it too far, to the point of excusing negligent behavior (which you should be trying to prevent) rather than just avoiding punishing people for mistakes.
At the same time, we have made the words "you screwed up" almost a swear in the workplace. If you can normalize "you screwed up" so that everyone hears it (and let's face it, everyone screws up once in a while), those words don't hold nearly as much power. In a sense, the ideals behind giving every child a participation trophy have been extended into the workplace under the guise of avoiding counterproductive things like scapegoating, and I'm pretty convinced that's a bad thing.
Using the literal words: "You screwed up." -- can you make an example of a way that would be helpful in an incident, either during the process, or after?
I can't. There's no value in it. What did we, as a team, do that allowed the incident to happen? Yes, John Smith shouldn't have dropped the tables on production, obviously, but does he really not know that as part of the incident response that he's (presumably) also dealing with?
If he's truly not aware that was a mistake, there's an underlying transparency issue that goes way beyond telling an individual they screwed up.
I can, in that given situation. Your assumption is that John Smith should be told "you screwed up" for dropping the prod tables. He shouldn't - he made a normal mistake. The negligent party here, who should hear the words "you screwed up," is Johnny's boss or tech lead, who decided that those tables should be droppable in the first place, despite the obvious risk.
"Negligent" doesn't just mean "made a mistake." It means something more like "their carelessness led to a mistake."
That person hearing "you screwed up" will cause significant behavior change. I daresay it will encourage them to make the prod tables very hard to drop, and since they are presumably a smart person, when combined with the postmortem of the incident, it will encourage them to look for and proactively fix similar problems, and generally align the team with good DevOps practices.
It is important in all of this that the right person gets the message. I assume you expect that to not happen, since that is one of the theses of "blameless postmortems."
I suppose I agree more with you than before, but I still think that aside from the fact that a manager or tech lead is ostensibly used to hearing people be angry (or frustrated, or whatever) -- why is John's boss more deserving of "You screwed up." than the person who dropped them? Yeah, obviously, the prod tables shouldn't have been droppable. John still shouldn't have dropped them. In fact, John, the original developer who implemented them, anyone who altered them since, the tech lead, manager, and frankly anyone who knew this data was critical could have all raised the alarm.
Ideally a blameless post mortem allows the freedom to identify any of the potential fixes that could've stopped this, and empowers anyone who could've dealt with it to deal with future issues. If you blame the manager then that can implicitly absolve everyone else in the chain.
With that said, I would agree that having a primary owner of things does matter. For that reason, sure, making the manager more aware might help in future. I still think it's a bad idea for org culture though because many managers will respond to "You screwed up." with trying to ensure future blameables find their way to another target. Instead, I'd prefer approaching the manager with "We could've caught this in [any of the ways we could've caught it].", and if the manager doesn't care at that point they're just fully incompetent.
I will point out that in the ideal scenario you're talking about, there is probably a next sentence after "at that point, they're just fully incompetent." That next sentence involves some real-life consequences of that incompetence. At that point, the postmortem is not really blameless, the blame is just one step removed.
I agree that what you are describing is the ideal, but I don't think it happens often. What I have seen more is that the postmortem turns into 10 different open bugs, 5 of which (usually bandaids that stop that particular failure mode) get closed quickly and the rest of which (often including the real solution) get put off until the next crisis.
Ownership is really important, but hand in hand with ownership is responsibility. That step is missing in many orgs today.
I don't know if there is an American culture of blamelessnes. Lots of variation, such a blanket statement needs more than confirmation bias.
With that said, I think your comment describes kiss-up and corporate fiefdom culture. There are plenty of examples of that. I wonder if non-American cultures are different in that respect.
One very key distinction is freedom to admit error vs "the baby is not ugly." I don't think blameless cultures are the norm, where people volunteer to write post portems and voluntarily submit after action plans..
I like the article and I have been promoting a blameless culture for most of my career. To respond rapidly to failings, we need transparency about what has happened. The same is required to ensure it doesn't happen again.
I have seen a lot of incidents and I cannot think of one where a single person was to blame. Sure, one person ran a command or made a bad commit. However, someone else granted them access, someone trained them. A manager either reviewed the process performed or never considered a process was required. A lot of company cultures do not promote proper risk management in technical processes.
I get a lot of satisfaction from technical post-mortems. There is always something that could have been done, a process that could have been in place or a software/infrastructure change to mitigate the problem.
A couple of companies have worked for have had individuals that did not subscribe to this culture. They would want a name. I never knew why exactly. Maybe it was to block a pay bump or defer a promotion. As the tech lead or dev manager any team failings are my responsibility and in companies where bullets are fired I take them. I find this protects the team and helps a blameless culture thrive amongst engineers.
Generally, I find this culture leads to fewer incidents and problems as the openness when things are going wrong allows for faster response times and software/process changes in review.
I have worked years in a culture that attributes all mistakes to defects in the process. And the only possible individual mistake is not to follow the process. Process culture is just as bad as blame culture in some ways. It leads to ever more complex processes. The teams I've seen do best, the manager of the project will allow process to be informal and treated as best practices only. Then people are held accountable for bad results by putting them on tasks that can cause less damage. Good results mean more critical tasks. No single mistake kills a performance review or promotion, or makes one. We all make mistakes and get lucky. But over time reapeated mistakes or successes do have an impact.
Process culture can be nice in the ideal scenario where the processes can have huge parts automated or programmatically guard railed. However, when that isn’t the case, I agree the paperwork barriers can get dreadfully tedious and introduce their own opportunities for mistakes
I've not heard of the idea of blameless culture, so I enjoyed reading this from that angle.
One of my worst and most consequential work experiences was with the opposite type of culture.
There's lots to be said about it, but in reflecting on things I've often felt like one of the worst parts of that type of culture (
"blameful"?) was a profound lack of trust, a feeling that the blame belied some lack of real interest in improving the situation, as opposed to being punitive against certain individuals for unrelated reasons. That is, the individuals who were targets of the most blame as far as I could tell were generally being targeted for cliquey reasons that had nothing to do with performance or anything of that sort. Conversely, people in the "in group" were given a free pass for all sorts of serious problems. Getting a free pass wasn't really the problem, it was a sense that blame wasn't really about the ostensible blameworthy act, it was that it was being meted out as a kind of superficial leverage for some other thing.
Also, because of these types of issues, serious, legitimate administrative, communication, and other systemic problems never got addressed, because the blame was being used as a kind of social ostracizing mechanism, and the actual underlying problems weren't fixed in their entirety. So in many ways a lot of the problems got worse, not better.
In contrasts, in other environments (even the same place at a different point in time) where there's an emphasis on problem solving and figuring out what could be done differently by everyone, and assuming the best intentions and basic competence, trust underlies everything. You're motivated more because you believe that actions have fair consequences, and that everyone has each other's best interests in mind.
How does blameless culture work with at will employment? It seems like there could be a real risk that a company's claims that they are blameless ends up being a trap.
IMO, it works just fine. (You can object to at-will nature of employment in general, but I don’t think this makes it particularly worse.)
Suppose employee E makes an error. Management decides to fire them or not. Firing someone for one error (of an accidental, non-malicious variety) makes no sense whatsoever.
The set of companies who do fire for a single accidental error is vanishingly small and very likely the least desirable of employers (as they’re likely making other colossally bad decisions as well).
If you’re working for a sane employer, at-will’s overlay with blameless transparency is fine. If you’re working for a terrible employer, you want to change that anyway.
A sane employer could decide to act on a repeated, long-established pattern of errors, which is frankly good for the organization and the remaining employees. In that case, the transparency around “most everyone’s error rate is a couple per year, while Pat’s error rate is 4 per week for the last 3 months; it’s time to part ways with Pat” is helpful.
This "blameful, not blameless" wording is a bit confusing to me. Are we blaming or not? "Blameful" isn't even a word, according to my web browser. The browser is showing red squiggles and so is my brain.
I think it'd be a little more potent to just say hey, there's a difference between saying someone is partially responsible for an issue, and pestering them about it (e.g. telling them they did a bad thing when they likely already know - I suspect that is what people want to avoid when they go "blameless"). Blame is blame, and disrespect is disrespect. I like to see them as orthogonal.
The article's fire suppression story doesn't even seem like a blame problem to me. It was a jumping to conclusions problem. But again, maybe I only think this because the definition of "blame" is being played with.
pestering them? What people are actually worried about is firing them, which still happens even at companies that pretend they know better. I heard from friends at Salesforce that this guy was fired, for instance. https://www.theregister.com/2021/05/19/salesforce_root_cause...
I did not know that arctic wolf had a blameless culture. In fact, I had the opposite expectation. The wolf usually represents a solo or selfish kind of thing. Which is technically not accurate to real wolf packs, but pop culture misleads about this.
I did know arctic wolf was big about being friendly to Trans and Autistic people, which I did like.
For me, long ago I was busy enough at work, I embraced blamelessness because I don't have time to waste on that crap. I immediately ended up with results where people who were expecting blame and didnt receive it were suddenly willing to get better. I learnt almost counter-intuitively or unexpectedly that you must have a blameless culture. Been blameless for at least a decade.
Props to arctic wolf, and well my current employer who is awesome.
I think the key for me when it comes to blameless culture is humility.
For me and the teams I'm on I try to instill an attitude of humility that at its core says "Any of us on any given day have the possibility of making mistakes."
Mistakes happen. Anyone can make one. Finger pointing is usually a sign of either overconfidence ("I would never be that dumb!") or insecurity ("I need this person to take the heat so I look better.").
To me that's what is important in blameless culture. It encourages people to approach things in a humble manner, which in my experience leads to better outcomes.
You need to have trustworthy people for this to work. I feel like its hard to steer an established culture for this reason, the problem is lack of faith and trust in the first place
I feel like people misunderstand what blameless culture is about?
Blameless does not mean it is ok to break stuff. Or that you are not supposed to feel shame. Or that doing shit job comes with no consequences.
Blameless culture was supposed to mean that it is more important to figure out why something broke and find solutions to fix it than pursue penalties for people who did it. That pursuing penalties and assigning blame tends to turn into this nasty political circus that frequently makes it impossible to figure out what really happened and if somebody is found to be responsible, it frequently is the person who is least adept at politics.
So we say: "The only way we know to be able to maximise improvements based on factual analysis is to stop clouding the process with politics of blame finding."
So this is more like getting away with murder on a technicality.
Not blaming is not the goal. It is the means to uncovering the truth and figuring the way to prevent it in the future.
51 comments
[ 3.4 ms ] story [ 109 ms ] threadMy last company had a culture of blameless post mortems, and also a strong culture of accountability, so definitely no one would fit the first definition. And the company was too small to have non technical managers.
My current company (big corporation) officially supports blameless reaction to incidents, but internally has a strong culture of cover-up and blame ping-pong, but I still cannot think of anyone who would believe "blameless means no accountability". It just comes from deeply ingrained habits (lots of people do their whole career inside the company, lots of them even getting their masters at the company's own "university". Behavior is hard to change when everyone comes out of the same mould). I also think it comes from inadequate processes: where in the previous company the postmortem was mostly a meeting done by and for the technical team, my new company has a strong culture of "escalation meetings", where the highest managers quickly get drafted in, which obviously has a strong influence on the capacity to conduct an open and blameless process.
I fully understand that some people are not looking for that kind of thing at all. Many would rather be nameless cogs, and some would simply rather not do the work (technical or soft skills). That’s fine too. For that latter group, whatever the management buzz words de jour are are likely to be employed where you work sooner rather than later.
My take is that it is probably better to accept bad workers in big orgs rather than trying to reach some kind of competence utopia, to get the incentives right.
No, that's still somewhat blameful.
If you had stuck with "blameless does not mean no accountability", I think you're right.
But the emphasis of the article specifically does not assume people do bad work on purpose:
> assume the individuals involved had the best of intentions, and either they did not have the correct information to make a better decision, or the tools allowed them to make a mistake.
I'd like to hear other takes on how accountability plays into blameless culture.
Blameless does not mean "you scan screw up as much as you want". If you start being careless, you will be shown the door.
Would love to know how this was explained. I wonder if the author remembers how it was put.
Somehow I have a hard time believing that. The author using "individual" in prose is a give away for militaristic leadership style, where you quickly learn to cover up any mess up so the direct superiors don't lose face.
Blameless ...
Unless the sysadmin got 1000 mails because the fired guy did "sudo rm *log" trying to guess the admin password I don't even know how you find out if something is a cover up, a mistake or a set up.
"shortly thereafter termination of their employment"
There was no due process there that is for sure.
Most of all the author's gratitude toward the patient fire chief whose leadership example he praises.
We could do with more of this in cybersecurity.
We're all surrounded by insecure software and defective devices churned out by companies chasing a quick profit. We're mired in complexity, non-determinism and broken systems. The experts who designed them throw up their hands in confusion. And we're hamstrung with laws and policies, which despite their laudable intentions only beget twisted systems filled with perverse incentives and loopholes.
How do we solve this? Blame employees! And then humiliate them! Make our cyber team into a punitive, even vindictive internal police force?
Karen from HR is sobbing into her cardigan sleeve because ICT caught her out in a "Phishing drill". Now she'll have to go on a humiliating "this is how a computer works" course - despite the fact she was programming before her IT manager was born. No doubt she'll get her own back in time.
As ICT and cyber teams we could learn a lot from more traditional organisational culture.
Should I DM you from info on your site?
PS: your website is so great. Love it.
Why it should not potentially affect compensations on progressions is beyond me. It might not as a singular event but but patterns of risky or disorganized behavior are commo rather than individual mistakes. Errors come in bulks but they might not express at the same time. In my experience there are people that do zero,every blue moon or infinite mistakes, and there are people that are good in zero,one or very many domains.
What an organisation needs for growth and coworkers need for sustainable stress reduction is the minimization of mistakes/output ratio.
If people get blamed a lot they switch to stereotypical behaviors and cant work in an innovative, creative fashion. A mistake took place? Is it institutional, personal? What other mistakes would present themselves if that mistake was a pattern? What is better for morale than finding another mistake due to analysis before it expresses itself?
Note that you bring in the example of a court, and this is how a court works: a suspect is presumed innocent until proven otherwise.
The main point is that a blameless culture fosters honesty and team work. A blameful culture fosters covering up and lying to look good to the manager. But if someone is careless, they will get into trouble in both cases. See the example with the person who deleted the production DB: they would likely never have said "guys, drop everything you are doing, I deleted the prod database and need help" in a blameful environment - even though this is the best reaction to have for the business. But even in a blameless culture, it takes a lot of courage to admit such a mistake and ask for help. Assuming that people are not afraid to do such mistakes just because they will not be immediately fired for it is quite absurd.
They are essentially filtering for the very same "C players" they despise and mock.
Like, the output disparity in bigger teams is extreme in programming if you let people loose. And you better not stick your head out in bad workplaces and the advice should be to do as little as possible to avoid being fired.
quality is a thing and forward movement with quality is possible.
What we are avoiding here is a CEO asking, "What caused the outage" and your manager responds "Bill wrote some code that caused a deadlock". It's possible Bill is working on fragile parts of the system, or writes most of the code. Blame at an individual level for an incident is unhelpful and, as OP points out, possibly creates a team with worse delivery incentives.
avoidance of negative quality is not quality.
Some years ago, we had an "incident" where one of our developers discovered he had admin access even though he was logged into a different account of his. It was a multi-tenant SaaS. Since he was supposed to be admin anyway, I wondered where the harm was.
Now, though, I think he was just practising the blameless incident response in case something serious did eventually come up. Once that happens, you don't have time to practise in peace anymore.
There have been many folks like this, in my career. I am humbled and grateful.
I worked for a Japanese corporation, where they didn't "blame," per se, but did figure out if someone was Responsible. If so, that someone was expected to cop to it, and the team would then close ranks, and fix the issue. Lesson learned.
I know that part of it is lawyers and police. If you say "I'm sorry," in many contexts, that puts you on the hook, legally. Most lawyers will advise you to keep your mouth shut, and never, ever admit a thing.
I was taught to promptly admit when I'm wrong; regardless of the consequences. Also, to not cop to stuff that I'm not responsible for.
I have often had arguments with (American) people, where they are 100%, without-a-shred-of-doubt, wrong. Often, embarrassingly so, yet, they Will. Not. Admit. It.
I could bring the spirits of Carl Sagan and Stephen Hawking in, to show them they are wrong, and they will still never admit it.
It's actually kind of sad.
My ability to apologize worked extremely well, in the Japanese corporation. They respected it, and I was given a great deal of deference.
On the other hand, the Americans tended to be pretty much dicks about it. If I apologized, they usually tried to use it to gain some leverage, and bully me.
I don't think there's some general "(American) culture of blamelessness," and I think your comment might misunderstand the point of it.
My understanding about "blameless culture" is it's all about avoiding counterproductive things like scapegoating, blame-shifting, defensiveness, etc. Removing those things encourages the full cooperation of the team in resolving the issue, by removing the anxiety of punishment for being the guy who's being blamed. Everyone's supposed to take responsibility.
At the same time, we have made the words "you screwed up" almost a swear in the workplace. If you can normalize "you screwed up" so that everyone hears it (and let's face it, everyone screws up once in a while), those words don't hold nearly as much power. In a sense, the ideals behind giving every child a participation trophy have been extended into the workplace under the guise of avoiding counterproductive things like scapegoating, and I'm pretty convinced that's a bad thing.
I can't. There's no value in it. What did we, as a team, do that allowed the incident to happen? Yes, John Smith shouldn't have dropped the tables on production, obviously, but does he really not know that as part of the incident response that he's (presumably) also dealing with?
If he's truly not aware that was a mistake, there's an underlying transparency issue that goes way beyond telling an individual they screwed up.
"Negligent" doesn't just mean "made a mistake." It means something more like "their carelessness led to a mistake."
That person hearing "you screwed up" will cause significant behavior change. I daresay it will encourage them to make the prod tables very hard to drop, and since they are presumably a smart person, when combined with the postmortem of the incident, it will encourage them to look for and proactively fix similar problems, and generally align the team with good DevOps practices.
It is important in all of this that the right person gets the message. I assume you expect that to not happen, since that is one of the theses of "blameless postmortems."
Ideally a blameless post mortem allows the freedom to identify any of the potential fixes that could've stopped this, and empowers anyone who could've dealt with it to deal with future issues. If you blame the manager then that can implicitly absolve everyone else in the chain.
With that said, I would agree that having a primary owner of things does matter. For that reason, sure, making the manager more aware might help in future. I still think it's a bad idea for org culture though because many managers will respond to "You screwed up." with trying to ensure future blameables find their way to another target. Instead, I'd prefer approaching the manager with "We could've caught this in [any of the ways we could've caught it].", and if the manager doesn't care at that point they're just fully incompetent.
I agree that what you are describing is the ideal, but I don't think it happens often. What I have seen more is that the postmortem turns into 10 different open bugs, 5 of which (usually bandaids that stop that particular failure mode) get closed quickly and the rest of which (often including the real solution) get put off until the next crisis.
Ownership is really important, but hand in hand with ownership is responsibility. That step is missing in many orgs today.
With that said, I think your comment describes kiss-up and corporate fiefdom culture. There are plenty of examples of that. I wonder if non-American cultures are different in that respect.
One very key distinction is freedom to admit error vs "the baby is not ugly." I don't think blameless cultures are the norm, where people volunteer to write post portems and voluntarily submit after action plans..
I have seen a lot of incidents and I cannot think of one where a single person was to blame. Sure, one person ran a command or made a bad commit. However, someone else granted them access, someone trained them. A manager either reviewed the process performed or never considered a process was required. A lot of company cultures do not promote proper risk management in technical processes.
I get a lot of satisfaction from technical post-mortems. There is always something that could have been done, a process that could have been in place or a software/infrastructure change to mitigate the problem.
A couple of companies have worked for have had individuals that did not subscribe to this culture. They would want a name. I never knew why exactly. Maybe it was to block a pay bump or defer a promotion. As the tech lead or dev manager any team failings are my responsibility and in companies where bullets are fired I take them. I find this protects the team and helps a blameless culture thrive amongst engineers.
Generally, I find this culture leads to fewer incidents and problems as the openness when things are going wrong allows for faster response times and software/process changes in review.
One of my worst and most consequential work experiences was with the opposite type of culture.
There's lots to be said about it, but in reflecting on things I've often felt like one of the worst parts of that type of culture ( "blameful"?) was a profound lack of trust, a feeling that the blame belied some lack of real interest in improving the situation, as opposed to being punitive against certain individuals for unrelated reasons. That is, the individuals who were targets of the most blame as far as I could tell were generally being targeted for cliquey reasons that had nothing to do with performance or anything of that sort. Conversely, people in the "in group" were given a free pass for all sorts of serious problems. Getting a free pass wasn't really the problem, it was a sense that blame wasn't really about the ostensible blameworthy act, it was that it was being meted out as a kind of superficial leverage for some other thing.
Also, because of these types of issues, serious, legitimate administrative, communication, and other systemic problems never got addressed, because the blame was being used as a kind of social ostracizing mechanism, and the actual underlying problems weren't fixed in their entirety. So in many ways a lot of the problems got worse, not better.
In contrasts, in other environments (even the same place at a different point in time) where there's an emphasis on problem solving and figuring out what could be done differently by everyone, and assuming the best intentions and basic competence, trust underlies everything. You're motivated more because you believe that actions have fair consequences, and that everyone has each other's best interests in mind.
Suppose employee E makes an error. Management decides to fire them or not. Firing someone for one error (of an accidental, non-malicious variety) makes no sense whatsoever.
The set of companies who do fire for a single accidental error is vanishingly small and very likely the least desirable of employers (as they’re likely making other colossally bad decisions as well).
If you’re working for a sane employer, at-will’s overlay with blameless transparency is fine. If you’re working for a terrible employer, you want to change that anyway.
A sane employer could decide to act on a repeated, long-established pattern of errors, which is frankly good for the organization and the remaining employees. In that case, the transparency around “most everyone’s error rate is a couple per year, while Pat’s error rate is 4 per week for the last 3 months; it’s time to part ways with Pat” is helpful.
I think it'd be a little more potent to just say hey, there's a difference between saying someone is partially responsible for an issue, and pestering them about it (e.g. telling them they did a bad thing when they likely already know - I suspect that is what people want to avoid when they go "blameless"). Blame is blame, and disrespect is disrespect. I like to see them as orthogonal.
The article's fire suppression story doesn't even seem like a blame problem to me. It was a jumping to conclusions problem. But again, maybe I only think this because the definition of "blame" is being played with.
I did know arctic wolf was big about being friendly to Trans and Autistic people, which I did like.
For me, long ago I was busy enough at work, I embraced blamelessness because I don't have time to waste on that crap. I immediately ended up with results where people who were expecting blame and didnt receive it were suddenly willing to get better. I learnt almost counter-intuitively or unexpectedly that you must have a blameless culture. Been blameless for at least a decade.
Props to arctic wolf, and well my current employer who is awesome.
For me and the teams I'm on I try to instill an attitude of humility that at its core says "Any of us on any given day have the possibility of making mistakes."
Mistakes happen. Anyone can make one. Finger pointing is usually a sign of either overconfidence ("I would never be that dumb!") or insecurity ("I need this person to take the heat so I look better.").
To me that's what is important in blameless culture. It encourages people to approach things in a humble manner, which in my experience leads to better outcomes.
Blameless does not mean it is ok to break stuff. Or that you are not supposed to feel shame. Or that doing shit job comes with no consequences.
Blameless culture was supposed to mean that it is more important to figure out why something broke and find solutions to fix it than pursue penalties for people who did it. That pursuing penalties and assigning blame tends to turn into this nasty political circus that frequently makes it impossible to figure out what really happened and if somebody is found to be responsible, it frequently is the person who is least adept at politics.
So we say: "The only way we know to be able to maximise improvements based on factual analysis is to stop clouding the process with politics of blame finding."
So this is more like getting away with murder on a technicality.
Not blaming is not the goal. It is the means to uncovering the truth and figuring the way to prevent it in the future.