Ask HN: Good ways to capture institutional knowledge?

Wouldnt the answer be: commit these dedicated, talented, key individuals to the knowledge tranfer effort, exclusively? You're trying to prepare for losing them. So, try it out!

Pull them off, and assign them to a cushy knowledge transfer process. You'll more safely learn about your true dependencies & get some of the knowledge you want at the same time.

Having a single wiki instance for your company is an awesome way to collect knowledge. You can write up operation run books, design documents, references all in an informal way.

Software. Software. Software.

Encode your knowledge into software, into data structures, into tests and active documentation.

Human organisations have faced these problems for thousands of years. And never solved them well. Maybe we need the new kid on the block to help

Cross training. It's easy to lose institutional knowledge when someone leaves if they are one of the only people that has worked on their projects. Much harder to lose that if you do even minimal cross training. Even if it's only once a quarter, have each person sit with someone else, have that someone else explain their job, their major projects, their details, pitfalls, strengths, etc. Lots of notes should be taken. A knowledge base should be a given, but that will only take you so far. A side benefit is such a wider picture of what's going on will better inform the work of everyone, keeping people not only on the same page, but understanding all the details of how those pages interact.

Edit: Yes, they'll hate it, but this should include a tour of duty Sales. Poor relationships & acrimony between sales & dev are based on a failure of understanding each other's jobs. And sales are the closest to the needs of customers, short of sitting with customers themselves.

Have the key personnel fully participate in knowledge capture. As far as the tool to be used for capturing said knowledge, don't waste time...Just use the fastest-to-set-up and the easiest-to-set-up wiki (or something similarly fast and easy). After the key knowledge has been captured, and likely after the key people are gone, you can look to see if you even need to migrate to a different platform than the original wiki. Knowledge transfer can get tricky in general...but since you're on a time constraint (key people leaving), don't waste time with UI or prettiness, etc. Just capture all the things!

Get your key knowledge holders to dedicate 20 percent of their time to documenting what they know. However, most of them are so used to knowing stuff that they aren't sure where to start or what the need is, specifically. Therefore, make them take requests from people who want to know what they know. Then what you end up with is really knowledgeable people spending 20 percent of their time asking everyone around them what they can document and documenting it. Check in every now and then and ask to see what they've come up with.

The easiest way is just to make sure product/business discussions happen over email rather than Slack. This way those discussions can be searchable and discoverable by anyone within the company at any point in the future. We make software for this (FWD:Everyone), but there are dozens of other similar solutions as well. That's the great thing about email, because the standards are open you'll be able to extract more value from that same data with each passing year using products that haven't even been created yet.

Conceptually this is similar to having a wiki, except for that unless your company has people whose full time job it is to maintain the wiki then it will always be out of date and inaccurate; just deploying some wiki software tends to be pretty useless, and even in the rare cases where they are maintained the medium inherently doesn't preserve the tacit knowledge contained within the decision-making process itself.

Mandatory vacation. This is something that the finance industry has used for a long time to guard against fraud -- it's hard to cover up something if someone else has to do your job for two weeks straight at some point -- but it also serves as a mechanism for requiring you to cross-train people.

Two weeks of paid vacation where the company isn't allowed to email them or call them for help: I guarantee that documentation practices will increase substantially.

You’re not going to preserve their knowledge via a wiki. At best a wiki would be a snapshot in time. They’re irreplaceable knowledge is likely an understanding of how change happens within the company.

The answer is by requiring close team effort. Pair programming for example.

I’d one person leaves the organization, there should be “adjacent” employees who understand what they contributed.

Simple rule: Require that new hires should be able to become productive without having to talk to anyone (physically or electronically). The rest will fall in place.

The value of institutional knowledge is that it's already in someone's head. Reading documentation, searching a wiki, email archives, may be better than reinventing the wheel but the real way preserve it is retention.

I would invest in that if your are in a complicated or specialized domain where it takes months or years for someone to really get their mind around it. One thing I have noticed is that private work environments tend to be present at the low turnover environments I've seen. At least a cubicle.

Another option is extending low hour contracts to departing employees as sort of an off-ramp to their role. They are on a few hours a week or as-needed to handle the dwindling number of cases where they are most needed.

Strong change control processes: if you want to know why something was implemented the way it was, the ticket authorising the implementation should have all the details including test results and names for who built, who tested, and who signed off.

It's a hard problem, but I've had some success with a few approaches. One is pulling the team into a conference room and giving a deep-dive presentation on some aspect of our system. I made a point of doing it via a web conference with recording enabled so someone could always go back and watch/listen to parts in a pinch.

Another is calling an impromptu huddle when I'm about to work on something that nobody else feels prepared to handle on their own. When some crazy bug crops up that I'm about to troubleshoot, I'll often ask a few other team members to shoulder surf while I talk through the approach I'm using to run the bug down. It's a great time to ask the group what they think we should do to test their knowledge.

I'm not sure about the "successful companies institutionalize the knowledge of their employees" part.

In such context a process becomes a (written) 'procedure/specification', and some folks stop innovating, they just do it "by the book".

HR quickly grasps this and hires people with less and less skill, cheap personnel 'just able to apply to procedures'. Other ones feel like cogs in the machine (especially the best ones, hunted by competitors) and quit.

Someone departing with the 'procedures' may let a competitor obtain a rather complete grasp of it and adopt the best bits.

Letting each team decide about this and establish cross training seems preferable to me, and has many other benefits.

If a system is not documented in terms of what it does, how it does it, and common remediations for when things go wrong, then it's a fragile system depending on one person to keep it going.

You need to get (particularly, senior) engineers to buy into the mindset of documenting everything and making the documentation the first place to look (not after all else fails). Leads should hold be held accountable for their docs meeting some standard of "this is a useful doc".

If I were in your shoes I'd get senior engineers in a room and be honest with them about the situation. You're going to need them to do work that they might not be naturally inclined to do, or work that might not seem like a "productive" use of time. You might want to get them to agree on what a good doc looks like (the standard), and what things need to be documented. Maybe take a whole day to do this, with snacks and coffee. Good luck!

One of the things I practice personally is more-or-less documentation driven development: https://gist.github.com/zsup/9434452 Although I'm using writing developer-level code, so I'm not literally writing end-user documentation. But I follow the principle; I generally start any significant function by first writing the documentation for it at the appropriate level of detail.

I usually also try to budget a day or two at the end to look at the documentation and generally clean it up. I can't always get "fresh eyes" on it per se but at least I can make sure it seems to basically flow.

The side effect is your major project also comes with basic documentation for very cheap. Honestly, it may even be "cheaper than free"; I do this because I think it helps make better code, faster. The act of writing the documentation doubles as a self-directed design review, and I couldn't even tell you the number of times I've documented some particular thing only to realize how stupid it is before I even wrote a single line of code [1], and started rearranging things at the cheapest development stage there is. But I concede in advance that proving it's "cheaper than free" is pretty hard.

I don't present this as a full organizational solution, of course, but unlike those full organizational solutions, this is something anyone reading this can pick up and start trying out tomorrow, whereas "redo how our organization conceives of how we store information" is a bit less immediately actionable, shall we say.

[1]: "Wait, I'm requiring what precondition of the callers? Wait, I'm going to return six values? (Better make a new struct/class/object.) I seem to have an awful lot of functions asking for the same 4 parameters (again, new struct/class/object). I'm asking for how many incoming parameters? These are awfully complicated instructions I'm giving about what they can and can't do to the return values. These instructions on the transactionality of this call are stupid complicated." etc.

1. Wiki w/ comments

2. Corporate university

3. Culture of anti-knowledge hoarding, pro automation, well-documented procedures and no "we need Joe, only he knows how to do vital task X."

4. Succession planning

A per-project or per-process README would be a good first step. Ideally keep the notes/instructions as close to the place where the work will get done. It's important for someone to test the instructions in the README on a new machine --- 9 out of 10 times you'll find a step about authentication or some system dependency was not documented. Credentials are extra tricky, so you'll have to think extra hard how to make that work (e.g. some sort of central key store, shared password manager, or ENV vars that need to be defined so you avoid putting any sensitive info in the README).

For something even better than a README, you could document the steps of a technical procedure in a Makefile (or Fabfile) that your colleagues can run. It's important to keep the scripts readable and stupid (as opposed to abstract and powerful like ansible), so that people can read the steps. Some people refer to this as "runnable documentation."

The pushback will come from two places: people who can't write (the overwhelming majority of people) and people who usually act impulsively and without a sound basis (also the great majority of people). The presence of these classes of people in a company will lead to the punishment of people who can and do write, for two reasons. The non-writers won't be able to tell that the writers have done something useful and therefore won't be able to highlight it as an accomplishment in peer or manager evaluations. The irrational actors will get all the accolades for "having impact" even when their random, unjustifiable activities have clearly harmed the company. Meanwhile the people who were writing the design docs will be judged for having less impact.

The way I like to measure this is there should be more writing than programming going on within the company. Some investigations, research efforts, or designs will lead to nothing, however every implementation should come with research, design, and retrospective documentation. In that case there will always be at least as many written artifacts as programs.

The way to prevent the nightmare of an illiterate workforce with only oral history is to hire people who can write and practice documentation-driven development, instead of hiring people who can memorize leetcode trivia.

Force people to stop asking the most experienced person to answer questions or solve problems.

Have people document how they diagnosed problems after fixing it, including queries for searching logs.

Thank people for documenting things.

I'll focus here on the knowledge of the key people you're about to lose. Ideas:

* If they're good at documenting, and willing to, task them to do as much of that as possible. The documenting might be in adding code comments and API docs, writing separate text files, etc. The person might not be able to document off-the-cuff, but have to work through a topic slowly, such as going through and re-understanding some old code themselves, going through a manual process that they do automatically and reflecting on the whys, etc.

* If some of the information is amenable to giving a talk to other employees, with a Q&A section, that might work, too.

* Another option is to have another employee interview the person leaving about one or more topics, and either type notes as they go, or record it and get a transcription. The interviewing person should be able to understand the topics.

* For tasks the person leaving does, you could have other people do the tasks while the person leaving is available for questions, and one of them documents as it goes. Depending on the task, it might make sense to have the knowledge-holder right there, both answering and observing, rather than only available for questions on-demand.

Side suggestion about accessibility/discoverability/maintainability of all this new documentation: consider keeping the medium simple, and avoiding a proliferation of locations, formats, a dozen bullpoop communication SaaSes, etc. For most software work, for example, inline embedded source code comments and API docs can be an easy way to try to keep a lot of information accessible in context and maintained. Some other information that doesn't fit well in source code, such as ops architecture and procedures, might be Markdown files in that same code repo, or another one. The occasional video file you just can't check into git might be a rare indispensible one, but can still be linked from a Markdown file that's in your repo, but even then, maybe you also have a text transcript in the repo, or someone turns a talk into edited docs in the repo.

Incidentally, much earlier in organizational knowledge sharing, I vaguely recall a study by a consulting firm (sorry, no cite handy, and I'm not 100% sure I remember which big-name firm), in which they found that people were resistant to having their knowledge captured in a system, because that knowledge was an asset of the individual. Your key people leaving might be more altruistic than that, want to help out their colleagues, want to have a good word-of-mouth reputation, have a sense of professionalism about it, have equity in the company, etc. You might like them to do knowledge transfer to a degree that's really above&beyond the call, so consider how you might acknowledge and thank them for that. It might also be a good example for others, and promote more proactive good practices for organizational knowledge.

Write write write and keep writing. Then expect to do 10x as much reading. It's exhausting and definitely relies heavily on employees' writing and reading comprehension skills.

HashiCorp produces a mind boggling amount of prose (non-code text). Every employee can read every RFC going back to the first sketch of terraform which was completely rewritten in a second revision. Mailing lists are alive and well. PR descriptions and discussions are often longer than the code being changed.

No other company I've worked at has had this dedication to recording decisions, and all of them have struggled heavily with losing institutional knowledge. HashiCorp isn't perfect, but I want all future employers to at least pretend they're remote-first as that seems to be the forcing function for writing everything down.

Update: while writing skills are helpful they're definitely secondary to just ensuring knowledge is splatted down somewhere in some form. Perfect is absolutely the enemy of good enough, and I'd rather struggle to gleen knowledge from an unformatted readme in a deep dark corner than have nothing at all.

People who have been in the job for a while aren’t always the best people to explain something.

What I’ve often done is asked new hires to document what they discover. New people are easier to mould to a new behaviour and often have the questions you need to know. When documenting becomes the habit, more people do it. Current 500-person company is very good at documenting many aspects, because we’ve done it since year 1.

We work on https://usecodeflow.com, which is a way to capture the way code logically flows (especially used for new-hires to learn a new codebase). Feel free to email me in my profile if you want to chat!

We record screencast videos demonstrating how to do a process. New people can watch those videos to learn how to do it.

If we find a better way to do something then we make a new video.

There's a team member who transcribes videos into google docs for people who like to read and search in google drive.

It's pretty simple and it works wonders for an international team.