Ask HN: How do you manage your companies knowledge base?
My problem is, I know that we are missing information, what I don't know is how to tease out that information from myself and other coworkers. Do you have any advice on how to get the knowledge out of my coworkers and into the knowledge base? Or, do you have a book you recommend? I have found a few books on knowledge bases but the reviews seem to be pretty hit and miss.
Edit: After talking with some friends about it I think I was able to articulate my main issue I have. In making this documentation it feels like I am winging it. However, all my training has been in coding and coding always has standards, guidelines and frameworks. It's hard for me to just work on the docs as I feel like I should be following some sort of standard as I do it.
Edit: Currently I am stuck with Confluence as the actual technology to use.
119 comments
[ 4.0 ms ] story [ 247 ms ] threadWhat we did was to put knowledge at the center stage, make it as easy as possible, pleasant to maintain, and avoid at all costs new/extra tools. We eventually settled for Notion. I'm sure you can find about notion benefits on your own, but essentially it does all that. It's super intuitive, easy and beautiful, easy to get around (albeit sometimes slow). Get everthing in the same place as much as possible, with every new tool you'll loose out (knowledge bits here and there, extra connections that stop working, or that people don't bother to check)
But getting a good information platform was only half of the story. We needed to start thinking and working differently. We decided that if we ever make a decision, or arrive at a conclusion, that's worth sharing, then it should be worth writing. And it's by doing this, and expecting all other team members to do it as well, that we arrive at this stage.
I'm sure this doesn't work for all teams, and other teams probably arrived at other, even better solutions, but this was what worked (wonders) for us.
I like the idea of any sharable decision or conclusion needing to be documented. We are a data warehouse for our organization and so we rely on business rules ALOT. So many of those business rules live only in the code with no explanation and no one remaining who can explain it.
I think we're at the turning point of beginning to think and work differently. I'm going to start with my small team of three, and once we get going then try expand it to the rest of the department. It is kinda hard as I don't really have the authority to enforce it, the best I can do is strongly suggest.
To have any chance of being up to date, knowledge of the code and functionality should be as close from the code as possible.
The biggest challenge is outdated information: since the responsibility is shared, sometimes I come across articles that haven't been updated since 2011 - for example, because somebody left, and I see some bits aren't up to date anymore. But I feel uneasy about deleting the article or just doing anything about it unless it's one of "my own" or directly related to my department. But in spite of that, I think it's working quite well overall. I used Confluence in another company and it felt too invasive and in the end nobody used it.
To help with finding things we are using a slightly modified implementation of the Johnny Decimal system (https://johnnydecimal.com). That's one of the news things I have implemented.
I like the idea about no orphaned articles, I think we have a few of those that need to be placed somewhere. Pointing out that responsibility thing is good, I'll have to make sure that we change our attitude a bit towards that.
Unfortunately, I am bound in the software I can use for the task as we already have Confluence and I am not in a position to change that.
No version control, no edit history. Sheesh.
Everyone can search for it online, or un MS Teams. It’s versioned and everything.
We can easily export to markdown or even reveal.js (for ppt) using pandoc.
Once you get over your personal dislike of Word, honestly it works quite well!
The tech might not like it at first, but it’s waaaay easier to onboard non tech people into writing docs.
Ultimately, most people are digital maximalists when they'd be better off being digital minimalists. I (in my personal life and in the small teams I work with) purposefully delete things that I do not wish to accept the burden of owning. Either something is important enough to justify the burden of ensuring it's accurate, or it isn't and it is deleted.
I have found that thinking about documentation in these terms, it becomes clear what to document and what not to document, and it forces better practices elsewhere in the business -- like writing code that clearly communicates the why, or designing business processes that leverage a core set of business information.
"Only Bob Jones knows the rest of this story. Call him at x1234 to ask him about it."
- Often, teams "own" stuff but the real knowledge is stuck inside individuals. This type of thing would need to survive the relentless reorg cycle that seems ubiquitous in tech companies.
- The health sector seems to have longer overall tenures than the tech sector. At the very least this kind of system should somehow link into the HR administration, so that if a document owner leaves there can be an alert "This document is now unowned!". Of course, that alert also needs someone to monitor that.
- What would you use as a metric for the asset-ness of knowledge? Some types of knowledge seem more important than others, but it is not clear to me how to quantify that.
Your point about digital minimalism is very good though. A lot of things that should be deleted are instead kept around in a form of digital hoarding behavior.
Knowledge is something you consume, digest and then use to inform thinking. New knowledge is built on old, and business outputs can be traced back through the knowledge that shaped them. Documentation, on the other hand, is a statement of fact(s) that are applicable only in a specific context (often temporal) that do not contribute to the evolution of a business.
Liabilities aren't inherently negative, they're a valuable tool in the right circumstance: a piece of documentation written for a customer support agent describing "how to issue a refund" has value but it can't be leveraged to grow the business, it can't be built upon, and it must be maintained. The business then must make a tradeoff between investing in knowledge or taking on liabilities.
If we were to represent this visually, knowledge would be the branches of a tree that other branches grow from, and documentation is a dead end.
Most of the documentation I've encountered in my professional life has been written to meet an arbitrary requirement for there to be "documentation" about an output, and so it's a rushed recital of the bare-minimum facts about a thing that exists. Nothing can grow from it. Whereas, when I've encountered it, the knowledge used along the way has been very valuable. A business can create an asset by capturing everything learned, and leverage that asset to inform the next decision, and so on and so forth. After all, that's what individual employees are doing already, it's just happening siloed in their heads.
Returning to the customer support example: years of knowledge collected about our business might teach us that our product's sizing is unique in the marketplace and many customers end up refunding their first purchase to order a different size. If issuing a refund takes 5 minutes per customer, and if all we have is documentation describing the 10 step process, a request from stakeholders to improve customer support efficiency might manifest itself as software development work to reduce issuing a refund to a 5 step process... but knowledge about the business would inform us that what customers really want is to ability to find the right size that fits. Rather than make a documented process easier, we might implement a purchase option where we charge for 1 item, send out 2 sizes, and the customer returns the one they don't want to keep. If all our business knowledge is captured, and not siloed, then a new employee should be able to theorise that, not only an employee who has been around from the start.
I'm not sure how confident I am in the way I've framed this, and using the word "documentation" seems like a risk because "documentation" means "everything written down" to most people, but I appreciate your question because it was interesting to think about: maybe someday I'll have more clarity in my thinking on this.
Every document is either "live" or a "snapshot".
Live documents have an expiry.
Documents are snapshots by default.
Respective team meetings, it's a agenda to talk about what needs to be updated, after updating a draft will be sent and later uploaded in the portal.
https://diataxis.fr/ https://ubuntu.com//blog/the-future-of-documentation-at-cano...
For new dev setup/infrastructure stuff we use a github Wiki and the new starters point out issues in that for us - their first tasks are to update it if they find issues or missing info because it forces them to go and dig around to find the knowledge islands - they're the best people for it because it's who we want to solve the problem for.
Admittedly we've got a small company and a nice tight tech stack but there's still tribal knowledge in 2-3 people's heads and distilling that out to the team is a slow process of pair programming and explanation.
I think we have been structuring documentation ineffectively for a while. GainKnowHow structures knowledge by prerequisites in a Knowledge Web, which I think is the best way to figure out how something works.
My goal is to let you "program" your organization through skills in a Knowledge Web that represents how your organization runs. You can write tests to see if employees actually understand their skills and there are change management features that tell the relevant employees when a skill has changed through skill diffs.
Here's an example of how to train your software engineers https://gainknowhow.com/software-companies.html
Personally I'd rather see a knowledge base that is in pure Markdown, perhaps a static site that auto-publishes on push to a specific repo. I just like the idea of being in control of the text, plus it means you can write these docs in the comfort of your editor instead of using a slow web UI. Right now I end up writing it offline and copy / pasting it along with formatting it in Confluence afterwards, it feels like a little wasted effort.
We try to reserve these types of docs to be guides, it's more like blog posts not so much a detailed instruction list on how to use some library or app. This way they don't really get outdated because a library or step changed.
I'm a big fan of never deleting guides and notes though, for the not needed ones I'd still keep them around but maybe move them into an archive section. Really, a static site generator has all of the tools to make a really nice system for this. You could also spend a day to make a little back-end to offer full text search too.
We've been using a mix of tools until now, Guru, Google Docs... but have lacked discoverability, powerful search, and in Google the ability to create sites for teams, etc.
Personally I use Obsidian and the files are sync'd via Syncthing.
What I wish would exist: A markdown driven wiki with Git (or something like it) in the backend so that you can clone the info locally, or refer to the master version, and even stage big changes to an area, etc. - basically something halfway between Obsidian and Confluence I guess.
[1] - https://www.athensresearch.org/
Your users can use any Markdown editor that they like. But one person should be responsible for creating new documents, so that there will be consistency in naming and placing in the hierarchy. Naming things is hard, leave that part up to the Knowledge Base owner.
Well, that's against the philosophy of a wiki. Also, in a small team, nobody is the "knowledgebase owner" - he's just another developer. Putting a human in the workflow creates friction. But I'm totally with you on the "naming things" problem.
A full-blown corporate knowledge management system is a repository you can throw anything into - invoices, emails, memos, reports, etc.; the system will automatically generate a knowledge taxonomy and article summaries. But such systems require much more maintenance effort than something simple like a wiki, and are overkill for a small team of developers.
And there are a ton of WYSIWYG Markdown editors. I've used Typora in the past.
As for you ask, the best solution that I've found, is to let people document things on Github and index those documents in another tool. That way there is no duplication, you let people use the tool they know, can use Markdown, but still benefit from searching, assigning owner, up to date date, rights ... That's how we do it at Dokkument
Obviously it's not user-friendly at all. Only people who know how to use git can use it, which isn't great for collaboration with non-techie folks. Ideally, I'd build a little editor widget that could be embedded in the page...
To serve your md files as a traditional wiki in browsers, there's a git backed wiki named Gollum that also uses md files in a basic folder structure. https://github.com/gollum/gollum You can see where I'm going with this.
Gollum doesn't have user authentication or anything fancy, it just renders and edits md files. I tried it. There didn't seem to be a difference between Obsidian's and Gollum's markdown. When I committed my entire Obsidian vault to a git repo, I could still choose to have Gollum serve the entire vault, or just a subdirectory in the repo. I could also disable all editing in Gollum.
While Obsidian is working directly with the md files, Gollum doesn't update until I actually commit the changes. Obsidian is basically an IDE for my wiki now.
I was mostly satisfied with Joplin syncing to OneDrive prior to today's experiment. But now I think Obsidian + Git + Gollum deserves a closer look. It might be a bit overkill for my personal wiki, but it could work in a team setting if everyone works on the wiki like they would a normal git project.
What is especially nice (less for us, more for some of clients) is that it has clear, understandable and working out of the box ACL system in place - and this is not obvious or easy to setup/configure for many wiki-like projects out there.
For maintenance and filling knowledge-base - I think that what's works for us is establishing and following rules, for example: one cannot mark feature as complete without creating/updating documentation page related to it, documentation maintenance is checklist element of task completion flow.
Each page has an owner, a contact, and an expiry date. Owner is responsible for keeping it up to date, and if you see a page that is expired you can ping the owner (unfortunately we don't have "alert on expired page" yet). If the owner is unavailable, the backup should be able to answer any queries/update the page, and if they can't then the failure of that lies on the page owner. There are a few pages that have empty backups, by virtue of our company size, but all the critical stuff has an owner, backup and is checked once every month or two.
But it's definitely better than no system at all!
This is true of any system of knowledge sharing, even if it's in-person-watercooler chat. The hard part is getting people to do it, no matter what the system is.
> This becomes a problem if multiple people leave at once or someone is very busy when expiration comes up and rubber stamps the renewal with outdated information...
The only solve for multiple people leaving at once is to ensure that you have all of their knowledge documented ahead of time, and if you have that, you don't need anything like this. A tool or a process can't solve people problems. For outdated information, this is no worse than the confluence abyss that I've experienced in previous jobs.
Documentation management and information architecture planning is work, and properly planned your maintenance will be minimal (NOT non-existent, but minimal) because the system will be designed to be as frictionless as possible.
But nobody wants to either pay for that or accept the time it takes.
We tend to keep more ephemeral project management stuff in our tracker or google docs and more long-lived content in our Notion tool. I'd be curious how your team balances that.
Mandating the doc stays up to date when noone is actively using it is an exercise in time wasting
The problem you are facing is the weakness of having individual specialists. The problem is not the doc maintenance but organisational
This seems like something chaos engineering and/or a proper ops team with regular disaster exercises would be able to pick up, but (like you mention) in most startups those seem to have very low organisational priority.
I would start by reflecting on the way in which getting that knowledge out of the coworkers into the knowledge base is incentivized.
I keep my notes in markdown files. A lot of my coworkers use Obsidian, CherryNotes, OneNote, and I'm sure that there's a dozen other solutions out there. Keeping notes locally is actively encouraged by the fact you can't rely on the Confluence server to be up after business hours and the fact that the centralized knowledge base upgrades destroyed our documentation efforts twice over the past 10 years.
I also keep my notes short, omitting everything that's obvious to me. This makes them less valuable when it comes to knowledge sharing, but makes them better for me - short notes mean less visual noise to filter out. Writing a version of those notes that's more comprehensive is extra effort. That extra effort needs to be visible, treated as 'real work', encouraged and valued. Otherwise, I will prioritize other tasks that are visible, treated as 'real work', encouraged and valued.
The first thing I would worry about is making sure the central knowledge repository is as convenient as the local notes. Then, I would examine if creating / updating documentation in the central repository is encouraged, and in what way.
1. Using fileshares grouped by customer (but this needed updating and not all employees uses those shares to file their work. Let alone the fact that sometimes it is not permitted to store confidential stuff on our own fileshares. We also do defense contracting work so that is a big no-no) 2. wiki's (already mentioned here), but this needed constant updating and for some the actual "coding" and using the markup language of a wiki was too much of a boundary 3. Using Yammer/Slack etc. This works OK as it is pretty low level, but it needs a certain critical mass to use and when you have 300+ employees with all kinds of customers who might also use their own Slack/Yammer what-have-you it's hard to get the employees conditioned to also check the internal channels each day/week.
We finale settled at the following mechanism which is a nice starting point to solve the "content/context" problem namely using the resumes each employee has to keep updated (and fiercely enforced by HR) and using a smart search algorithm to find the best knowledge-match of each employee. It doesn't work flawlessly and it had to be customized for us, but it works better than other methods. It's still a two step approach:
1. Search the CV for the best content-context match 2. Call/mail/IM the colleague and try to figure out if his/her knowledge helps you in tackling the issue you have.
Another method I saw someone use (and apply within a large online retailer) is to "tag" each person with his own knowledge set and visualize the links between them. So, again using my field of work as an example: Bob has a lot of knowledge (explicit in the form of certs etc.) of Palo Alto firewalls and is considered the internal "guru" in this specific field. The more Bob gets consulted for his knowledge the larger his "circle" of knowledge is (you can actually visualize this). What you eventually build is a knowledge graph were each point in the graph represents an employee with a certain knowledge set and every time he/she gets consulted for that specific knowledge by another employee a line is formed. This is a very powerful system but requires maintenance to update the knowledge sets. Setting this up requires interviewing each employee with questions like:
- What is your skillset and how do you score on a scale of 1-5 (or whatever scale one sees fit) in this specific set? - Who consults you the most for this knowledge set? - Who do you consult the most for which knowledge?
The end result is the aforementioned "chart" with "dots" representing ones knowledge. The larger the dot/circle the more knowledge one has.
It takes Notion content and generates a docs website
I am the creator of this tool so don’t hesitate to give me any feedback
https://workpilot.io The platform is part wiki/ part proċess management tool. After lot's of user discussions it became apparent that lots of business know how falls into those two categories. With the latter being easier to reuse and track.
As an expert I often think things are obvious that are not. So it takes a few rounds to make useful documentation.
How do you deal with outdated info that's still useful to keep around?
I have a few internal wiki-pages which are 80% 'stuff I tried that doesn't work but I'd like to keep around because the tries are valuable in case I re-visit these angles', 20% 'stuff that worked in the end'. I find the 80% confuse newcomers, and I still have to explain which parts are important.
On the other hand if you have good version control over the wiki, and a way to search the version history as well, kill the zombies.
Just like killing zombies in your code, killing zombies in documentation sounds good. https://www.bitnative.com/2012/10/22/kill-the-zombies-in-you...
You separate out them out into 2 related parts of documentation. My guess is that you want them in the same wiki page because that's how you wrote it and that's how the information lives in your head since it's all for the same project, but from an information management point of view, they're two different pieces of information with mutually exclusive metadata properties.
More broadly, ideally you talk to other people and find out where THEIR 80% 'stuff I tried that doesn't work but I'd like to keep around because the tries are valuable in case I re-visit these angles' information lives and collect it all together and presented in a way where people know going in that they're looking at historical information for the sake of current decision making.
It takes discipline, as otherwise everyone skips the "write documentation" step, and it will annoy some people with high priority issues.
But IMHO, it's the only thing that actually scales if you truly want comprehensive, continuously-updated documentation.
Side note: An excellent way to generate drafts for missing content is have your new hires write them as they get up to speed. By definition they're looking everything up, so they'll notice gaps. And they have time, before they're fully up to speed. And as long as someone with knowledge proofs their draft, their lack of experience shouldn't matter.
I rely on git repos using markdown because it's fast and I can search, however things like tables etc can get annoyingly complex to upkeep.
We are currently in the process of creating a data dictionary for our company.
I put down a rule that we are not ever going to create a SEPARATE WordDoc/Wiki/Evernote/GenZ-tool.
If it is code, document its meaning (english explanation for laymen) in the doc string. If it is data, document its meaning in the column's metadata (all database systems provide a property/comment/description capability at table & column level). If it requires complex diagrams, put these in whatever files (doc/image/pdf/mp3), store it as a blob in a database and create a link it in original code comments/column metadata.
All this data is then constantly pulled into some data-mart on top of which a query-able UI is created.
Every documentation must be literally "Tied" to actual systems making money for the company.
If a new code/data is created or updated, and it's missing documentation, PR is not approved. Once the basic technical setup of tying code/data with comments/metadata is done, enforcing this rule of updating documentation is the job of management/CTO culture.