54 comments

[ 3.3 ms ] story [ 118 ms ] thread
Instead of a "no shaming" culture, I'd like to find a culture of "don't fuck up". "No shaming" and "no assholes" just leads to people cheerily and politely fucking up.
Don't the two go together? Shame drives people to cover up their mistakes. That produces more fuckups.

There's an old improv game where someone suddenly points to you and you have to continue their story without missing a beat and without doing a bunch of other arbitrary things. It's designed to be impossible. But if—or rather when—you fuck up, the rule is that everybody applauds, praises you, and gives you hugs. It's disorienting at first, but then you can feel yourself acquiring entirely new patterns in response to mistakes.

The thing is, trying to demand absolute perfection results in the inevitable imperfections getting hidden.

If you want them to get fixed, there can't be any incentive to hide them. Which is where "no shaming" rules come from.

In my experience, a no shaming culture is one where mistakes get fixed instead of hidden, and everyone can learn from them.

Fwiw, I work for trading firm, which trades 10+ billion dollars daily and mistakes can be incredibly costly. I would argue the lack of shame associated with admitting mistakes is vital to our level of code quality.

I have yet to meet the developer who does not make mistakes. I have met a few arrogant enough that they'd never admit to them.

Ah, "don't fuck up," I wonder why nobody's thought of that before. It sounds like maybe you don't like your coworkers these days?
I would rather know about something early so that I can fix it. If your entire business culture is one of berating people for making mistakes and introducing bugs, I can guarantee you that there are some really good latent ones that will bite you in the ass some day. God help you if peoples' lives actually depend on the code you write.
I'm won't be the first HN'er to note that the business of aviation has evolved a culture much like the ideal being discussed.

You can see it when you read NTSB AARs, TSBs, even the regs (FARs) themselves. Even the language is chosen so as to avoid shame.

When blame is ascribed it's explicit and for the purpose of explication of root causes, but, more generally, the tone of the narrative is always, "we've critiqued _the process_ and here's how we see to improve it."

People are only seen as participants in the process and aren't targeted for blame or shame.

The amazing safety statistics of commercial flight speak to the efficacy of this philosophy; software project management would do well to adopt much of these strategies, I'd argue.

Upvoted, because I don't see a culture of working hard to not fuck up and shaming being mutually exclusive.

Shaming is a poisonous behaviour but so is the belief we're capable of perfection. So also is the idea that we should be tolerant of those who constantly fuck up. Fuck those who can't/won't improve.

If we put the infrastructure and attitudes in place that assumes that we will definitely fuck up, that sub-optimal solutions and people will be introduced, then fucking up becomes less common, communication isn't inhibited by shaming/bulling attitudes and, god forbid, we can raise our individual and collective standards and become more attuned to stink and how best to amortize it.

Everything can be automated, even quality checks.

There is no such thing as culture of "DO fuck up", only cultures where avoiding mistakes is secondary to other goals like hitting the schedule date.
There's a huge difference between a "no shaming" culture and a "don't be ashamed" culture.

It's critical for people to be able to be open and transparent about their mistakes and the process leading to them without fear of being put through the ringer for it, but that's not mutually exclusive from an environment where the type of appropriate embarrassment/remorse necessary (usually in the form of a thorough postmortem) to truly learn from one's errors is expected.

If your name is Alice and you live in Wonderland. Or Harry in Hogwarts.

In the real where requirements change constantly, libraries that you rely on rely on left-pad, budgets are short - or hell people just learn to write better code - so the old need to be refactored - the debt accumulates.

Code debt is like entropy - always goes up. The normal state of a software project is under staffer, under budgeted, in constant flux requirement wise and for yesterday.

Doing it right with reviews, refactoring, consistently is a luxury. You can try to have a process that minimizes chaos to minimum, but every project will devolve into unmaintainable mess. You cannot afford to be pure while you have the huge state monad called rest of the world that gives its best to crush you.

Like every "we are doomed" answer there's a core of truth in here: If something is used and/or changed it will probably break down.

Given that and trying to use it as starting point for something more helpful: Sofware is nothing special here. Your things will - depending on their quality - break down with time, especially when you tinker with them and/or use them. You can push this further into the future using maintenance, repairing damaged parts and so on, but at some point you will have repaired so much that you will in essence have a new thing, not the old thing anymore.

The task is to balance your efforts with your use case. An engine which has to work for one formula one race is constructed very different from one which has to work 200 or 300k kilometers. Again, software is nothing special here. If your software has to work for one run "doing it right" may be a luxury. If you need your software for years to come your luxury becomes necessity.

"Right" is never fully defined, and keeps changing. Therefore, you can never do the right thing.

My solution is radical, but necessary. 99.999% of code should never be written. Let's start saying no to businesses with arbitrary demands, and let's all agree on a set of standards. Only then will we have affordable and quality software.

If we did that then we might actually start resembling a field of engineering. Unfortunately software is much more nebulous than structural, mechanical and other fields of engineering. There are less clear cut right and wrong things to do.

Part of what gives software it's unique advantage in today's world is it's ability to change to meet evolving demands quicker than any other way of formulating solutions. As such unclear specifications are somewhat of a pillar of software engineering.

However with unclear and perhaps rapidly changing specifications comes a lot of judgement calls on what to build, how to build and what are reasonable shortcuts to take. Software engineers are almost alone in this, most other professions have the "correct" way and all the ways you can do it wrong. In software there is no "correct" way and just many different ways of kinda achieving something "right".

I guess, if it was possible to build towers and bridges at the same speed we build software projects, we would see the same level of nebulousness appear.
"99.999% of code should never be written"

Interesting. So if you are a freelancer, you will reject 99.999% of would-be customers. Not exactly a winning formula.

Or, if you work for some kind of organization, you tell them, 99.999% of the time, that the business demands are "unreasonable" and that code should not be written. That will get you fired.

Either way, I am not sure where you are going with that narrative.

> My solution is radical, but necessary. 99.999% of code should never be written. Let's start saying no to businesses with arbitrary demands, and let's all agree on a set of standards.

Problem is identifying those 0.001%

All all the tools mentioned and similar tools are great and it would be a good idea to integrate them in the development process, but I don't think they're that useful to diminish technical debt.

Technical debt to me is uncessary dependencies, anti patterns etc. The only way to prevent this is a good code review were people pay attention to such things. Ideally the code author and the reviewers have the right thinking: after checking for correctness, they ask themselves "is this code modular/reusable/easily testable/easy to debug/..."

Hopefully the testing is already here, but that's in a perfect situation

> Odds are far better than good that your high performers are achieving what appears to be high levels of productivity by building technical debt into the application by taking shortcuts whether intentionally or unintentionally.

Perhaps this is a true statement; however, I have no doubt that 10x programmers do exist who do not build technical debt into the application. Some people really are that good. I don't think the average person realizes exactly how good the best in the world are at what they do. For instance, I ran track for 8 years and competed in college. You don't realize exactly how insane a 12:37 5K is until you've trained as hard as you can for years and you're not even close to that time. Nor do you realize how difficult a perfect rendition of Liszt's transcendental etudes are until you've spent years trying to learn them on your own. I imagine the best programmers in the world are similar, but it would take the average programmer decades of programming before the magnitude of the skill gap becomes evident.

10X programmers are unicorns.

If a 5K of 12:57 was really "10X", an average runner would take just over two hours. That performance is nothing like 10X.

I don't think that's a good analogy - a run has a very defined course, a very defined technique (you must run, not use a motorcycle), and very defined end point. Most programming problems have infinite solutions, and you're typical 1x developer can only see what they know and have experienced.

To carry the analogy forward, a 10x runner might take a shortcut, use a better implement to complete the course, etc.

Yeah a (slightly) better sporting analogy might be chess. Like chess, programming is a game of knowledge and forethought, where intelligence is an asset, but far from sufficient to elevate you without a ton of practice. However unlike chess, there is no scope or rigid definition of programming, it is as if there were as many pieces with different rules as there are possible positions on a chessboard, and furthermore that the rules of the pieces are made up themselves of subpieces all of which are programmable, and that you can essentially change everything from the behavior of the squares up to the overarching rules.

Now in chess, as in programming, there are clearly some individuals who excel by orders of magnitude over beginners and even average competent players. But because of the vast scope of possibility, programming "grandmasters" will do things that no one else could have even thought of in one domain, while in unfamiliar domains they might require a good deal of study before they surpass the average journeyman practitioner.

Meh, analogies suck.

It's more about taking an analogy where you compare a normal distribution to something that isn't a normal distribution.

Physical capabilities of organisms (e.g., strength, speed, jumping height, etc.) tend to fall into normal distributions.

Knowledge work where you can take advantage of powerful (mental | software) abstractions don't tend to fall into normal distributions, as previous work can make a bottleneck disappear, leaving you with more mental bandwidth for other concerns, which quickly expands your capabilities.

intelligence is a physical capability.
I wasn't talking about intelligence.

I'm talking about productivity in knowledge work which is different.

You're assuming that the difficulty is linear in time, while it could easily be exponential.
I'm assuming we're talking about output measured in the real world. At the forth standard deviation you might be 1 in a million, but you're still not outperforming an average person by 10X.
Then your definition of 10x doesn't apply to a software engineer either. What measurable metric does a "10x developer" outperform the average developer in?
The interpretation that I like is that "10x productivity" is more about what someone doesn't do than what they do do. It's in the sense that they seem more productive, partly because they're really good at what they do, but mostly because none of what they do gets thrown away. They're good at the intangible stuff around the tangible job. Someone can be extremely productive, but they won't be "10x" if they produce a lot of stuff that ends up being useless for any one of dozens of reasons unconnected to the actual job itself.
I've never understood this argument. To me, it seems obvious that 10X programmers exist.

How much faster would John Carmack produce a new graphics engine than a recent bootcamp grad? How much better would Carmack's be?

I know I've gone to unfair extremes with the question, but it still illustrates the point. Really excellent developers are waaay more productive. The upside is huge. All the time saved avoiding blind alleys based on experience, all the cases where experienced intuition identifies the cause of a bug that would have taken much longer, etc.

the popular claim is 10X better than average. Not 10X better then suck. And definitely not 10X better than a total newbie at the one thing you've worked on your entire career.
Also, it's not actually "10x". That's just easier to write than "an order of magnitude", which is a subjective thing in productivity. The idea is vaguer than the "10x" abbreviation implies.
(comment deleted)
(comment deleted)
It's interesting how different this advice is from what some of the greatest engineering cultures have done.

At Microsoft during its heyday in the 90s, it was common for people to say things like, "That's the dumbest fucking idea I've ever heard" and the behavior came straight from the top. Similarly, Steve Jobs and Jeff Bezos are well known for humiliating engineers in front of an entire room after a presentation.

http://www.businessinsider.com/things-amazons-jeff-bezos-tel...

There's no way to prove the issue definitively either way, but I suspect this blog post suffers from the cognitive bias of confusing how the world should be with how it is. At the very least, a "no shaming" rule isn't a requirement for building a world-class engineering organization and attracting top talent.

I've worked for real jerks before. I don't think the economics of it work out. This being said, you learn a certain stoicism in the face of it. If you catch a real rant, a stone cold "You finished?" at the end might be in order.

Don't get me wrong, I've raised my voice at people but only after they've persisted in a particular error for months, after receiving many many external reference to why it's an error.

I've no interest in working for a Bezos-type because I see that as a "cult of personality". It's something people have to work around.

If people are risk averse to sharing ideas, then you'll get fewer of them. Perhaps that's a good thing. But it costs very little to learn to provide feedback in a non-confrontational manner.

I imagine how well people tolerate what they shouldn't has something to do with being Authoritarian Followers. And something maybe connected with those who glory in hazing culture. And we must grow past that kind of idiocy.

If I worked for someone who treated me like that and then, decades later, I heard that they'd finally died, I'd say IT'S ABOUT FUCKING TIME! and have a little party.

I think I know what you're talking about when you mention Authoritarian Followers---something like abused kids who only know how to deal with situations in which they are abused, even when they grow up.

The worse part is they want to have "followers" themselves as well, subconsciously, repeating the cycle.

I still have many properties of Authoritarian Followerism in myself, but I'm making progress.

One of the difficult parts is many people around me show blatant signs of that followerism and it's easy to get sucked back down with them.

E.g., a coworker protested my protestations against our company stack ranking employees based on "quantitative" metrics, and then many months later backpedaled on his words when someone who had a more important title said the same concerns.

Not sure Microsoft in it's heyday was a great engineering culture. The were a monopoly, which allows bad engineering to occur without ill effect for a long time. I think many of their problems today are due to being tied to code written in those days.
> Similarly, Steve Jobs and Jeff Bezos are well known for humiliating engineers in front of an entire room after a presentation.

Michael Jordan had a reputation for being a jerk too, but it's cargo culting to think that's how you get world class performance.

Jordan could do it and be idolized for it because he was the best on the court usually. Microsoft in the 90s was the powerhouse. Working for Apple has (had?) carried a certain panache, and Amazon is a similar powerhouse.

Looking at Microsoft (or Ggole, Apple, Facebook, etc) is a distraction. They have the resources to keep great engineers from leaving toxic environments - not just by paying a lot, but by having the most challenging and huge scale projects, and by having other great engineers. Those things don't apply to Acme Software Inc. Most businesses aren't building world-class teams or making cutting edge products. In that environment cultivating a team where someone speaking up might be shot down and made to feel ashamed will lead to developers refusing to share ideas, which is potentially bad for the business, and worse, refusing to share experience, which is definitely bad for the business.

If an engineer feels they can admit a mistake early then the rest of the team can help to fix the issue before it escalates to the point where you lose a client. If an engineer hides their mistakes because they're afraid of how their boss might react, things are left lingering, and they don't get fixed. That's how really big problems happen.

This is definitely true today. It wasn't true when MS was a startup and Bill Gates was particularly harsh and it certainly wasn't true when Steve Jobs returned to Apple. At that point they were only a few months away from bankruptcy.

Another data point is the vastly greater acceptance of giving workers a tongue-lashing in China vs the US. In that case as well, the harsher culture has certainly been growing faster in terms of engineering-driven companies. In fact, I've never seen a US company of comparable size that can move as fast as a Chinese tech company. Part of this is due to differences in IP and compliance laws but I believe culture also plays a role.

To be clear, I don't like that employers treat their staff harshly. That doesn't necessarily mean the practice is less effective, though.

Humiliating your subordinates in public is akin to beating your wife. I'm sure there are plenty of excuses: she's been sloppy doing house chores, she got fat and is not attractive anymore, you're under a lot of stress at work. Whatever. The problem is the abuser's untreated psychological issues and has nothing to do with the victim.
> “Do I need to go down and get the certificate that says I’m CEO of the company to get you to stop challenging me on this?”

In my experience at Amazon, there's a lot of free discussion and invitations for criticism of management ideas, but once a decision has been made, everyone is expected to get behind it for at least the next 6-12 months.

I don't think most teams at Amazon engage in "shaming" behavior, as much as they set a standard. Ie, if we agreed that all new modules are going to have 100% or near 100% test coverage and someone turns in a commit that doesn't have any tests at all, they're going to get some shit from the rest of the team.

There's a difference between an implementation that doesn't work as well as hoped that needs to be refactored, and intentionally adding technical debt and going against best practices to save yourself time. Quirky implementations are treated pretty indulgently at Amazon as long as you fix them once it's clear they're a problem, while laziness and causing harm to other developers who have to maintain the codebase is clamped down on pretty hard.

Most people would be in favor of "shaming" an airplane mechanic who doesn't follow the safety checklist after they've been taught how to, but not so if it was their first day.

I think that might be focusing on the smoke instead of the fire. The sort of people who make amazing things happen are driven by rage.

They take the success of some area of the world so personally, and hold themselves to such a high standard, that everything they themselves do is insufficient, everything anybody else does is woefully insufficient, and everything the rest of the world does is evil. When the closest anyone, including the person assigning grades, can get to "good" is merely "doesn't obviously suck right now" it makes sense there would be a lot of shouting.

In my experience, the most important skill in an engineer is figuring out what to build. Someone's skill at the actual task of programming is trivial in comparison.
I'm pretty sure the marketplace has decided that doing things "right" isn't particularly valuable.

There are places currently where you'd expect an extremely high degree of rigor to be applied to a variety of engineering and technology problems because the potential financial impact of a cluster of interrelated faults would be irreparably financially and reputationally disastrous. But instead you will often find the overwhelming culture is to instead hire legions of employees/contractors ranging from demonstrably incompetent to tacitly complicit in knowingly producing garbage just so long as that garbage can be produced on reliably unrealistic (were any amount of rigor to be applied) timelines, and where the optics of "progress" vastly outweigh the quality of releases. And while none of this would be a spoken or agreed upon culture, the set of incentives in place guarantee that results do not align with the institutional rhetoric.

And as much as it pains me personally to see cultures like that (as a 1x engineer who labors naively over correctness in design and implementation)... for the most part I think the marketplace got this right. If you consider realistically that the percentage of software programmers/developers/engineers/architects who work on something truly "mission critical"/"carrier grade"/"matter of life and death" is a rounding error compared to the number which work on the next Dashboards as a Service, or Fitbits for Dogs, or next-gen tween sexting mobile platform, or just toiling away fruitlessly on shelf-ware in large directionless companies surviving on the carcasses of well-heeled dinosaurs in their "enterprise sales channels", then quite obviously doing things "right" comes along with almost no actual value other than just the ability to claim you did it that way.

If building most software came with anything remotely resembling the risk and liability of building a high-rise apartment building, or a suspension bridge, or the hydraulic controls of a jetliner, then one would likely see much more common practice, conventions, and thorough auditing of software that gets created. Rather than "best practices" being outliers, they'd be the norm, and they'd be much more codified and immutable than simply the most recent blog post about scar tissue earned from a metaphorical software skyscraper collapsing into The Bay one sunny Tuesday afternoon.

As an informal support of the above claim. How many among us have worked on projects which included, as a central pillar of the engineering process and culture, all the following forms of testing... unit, integration, property-based, concolic, end-to-end, chaos, regression, soak, and mutation, and have indefinitely pushed back releases until all those forms of testing could be exercised and validated? Whatever the number I'd wager a year's salary that there are more people who don't even know what most of those things are than the number which have been required to diligently do all of them. Probably by an order of magnitude. If we're talking about mythological 10x unicorns, perhaps we can start with that one?

The reason the debt metaphor is used for technical debt is that debt isn't necessarily a bad thing - it just something to understand and manage.

Most people, when they buy a house, go into debt (e.g.: a mortgage) for the initial purchase - without that debt, they would never be able to save up the money needed to buy that house.

A startup seems similar - the first goal of a startup should be to show traction and market fit before running out of runway - and if building up technical debt lets them achieve that more quickly, then that is the right thing to do. This first phase seems to be where the 10x programming would shine.

Great point. Debt is a necessary part of bootstrapping a new project. That being said, technical debt is not a blanket excuse for violating elementary engineering precepts, shipping shoddy components full of bugs or a necessary evil part of the final product. The team must be able to be specific about the technical debt and execute a reasonable plan to repay it.
Technical debt actually isn't a great metaphor. It breaks down too quickly, as in this case. You don't have to "repay" technical debt. It's not really debt.

A company only needs to build something that works. If it works in spite of violating elementary design precepts, shoddy components, bugs, and necessary evil, then it still works. The only thing that matters is the result. Investing resources into optimizing it so that it works better than necessary is merely an option, whereas getting it to work wasn't.

The iron triangle includes cost and schedule in addition to performance. If it would take more money or time than you have to make it perform better, then it just won't, because you don't have that much money or time. A product that frightens the maintainers, but keeps the lights on, is a good result.

Of course, you might optimize yourself into a local maximum if you're weak on strategy. That's why nearly every organization in the world has some kind of separation between "officer/executive" and "enlisted/staff" in the hierarchy. Someone needs to balance the triangle and someone needs to write code and there isn't enough time for the same person to do both.