Ask HN: What is key to good technical documentation?
And of course, management is eager for you to get stuff up and running, as the project is already severely delayed, needs to be shipped as soon as possible, and the customers and investors are getting impatient.
There is only one problem - all the three senior engineers which were working on the codebase have left suddenly, only their manager is still there, and you have to start with reading and understanding the code, and what needs to be done. It is somewhat commented, but, of course, it is not easy to understand. And as it looks, the previous developers did not had any time to leave you proper technical documentation! What are you going to tell the manager?
But then, suddenly, a fairy godmother appears, which has a magic wand in her hand. It is a kind and witty documentation fairy, and she says: "I will fulfil you a single wish. By my magical powers, I will give you exactly the documentation which you think is most important. It can be anything you want. Anything! The only condition is, it must not be more than a good software developer can produce in a month."
What do you wish for?
89 comments
[ 2.5 ms ] story [ 149 ms ] threadRemove the unneeded layers
Use of standardized frameworks
Proper datamodels
Proper naming
Proper error handling or no handling at all
A few diagrams
What to tell the manager.. maybe that he should’ve been more involved in the overall quality of the product and motivation of his team
- up to date
- contextual
- progressive
- easily accessible and updatable
- most be in CI/CD pipeline with builds and tests
All those might seem simple, but in practice are hard to achieve.
Achievable in single month is unrealistic requirement. Documentation is ongoing process, evolves with the code - its as unrealistic as getting good software in a month.
I know. I ask what can be done realistically in the given time.
"this service sits between x and y, doing z. Z is handled by libraries a and b. We couldn't use c because reason d. This is how you call it, and this is an example result. You'll notice that we're passing both the user id and email address in the request, because we didn't want this service to talk to the database. Oh also this whole thing depends on a local sqlite cache because shrug"
High level breakdown of the code structure. What is each module / package / namespace intended for? Why is it laid out in the certain hierarchy it’s in? If I have a question about some functionality, how do I map it to a subset of the code?
Who are the stakeholders? Who will get mad if this breaks and why? How many systems does this depend on / integrate with? How do I contact those people?
What are the data flow or job flows? Does a build get triggered in CI? Is the code published as a library? Does a web service get deployed? Does a human have to compile something and then rsync it to a production server every Tuesday or else everything crashes? What are all the regular tasks? Why is each one needed, how can a test version be run?
Aside from that, then just the basics:
- all functions (or all exportable functions) have consistently formatted docstrings
- docs are concise most of the time, but overly verbose in any “danger” sections of code that rely on unusual, specific or brittle logic.
- well documented description of project version control. What are all the usual code management tasks someone needs to do, and why? What is your strategy for hotfixes, releases, rollbacks, sharing branches, etc.
* High level architecture (what pieces talk to each other, how they communicate, what they do)
* A list of 'here be dragons' areas, places where the code is gnarly
* A bug database, which will give you history of decisions
Then I'd ask the manager/product owner for a requirements document (or documents), as that should be something a non technical person can/has put together.
Frankly, though outside the scope of your question, I would also look at hiring one of the departed senior devs on a contract basis, just to ask questions of. I have offered this for the past few jobs I have left and there are times when 15 minutes of my time could save a few hours of investigation.
For more contextual stuff, like, for example, build procedures, I am wondering whether using a Wiki would be better. It is not only easier to keep it up to date, but it also allows to cross-link topics and this in turn makes it easier to split up information into smaller, easier to find and more digestible units.
Of course, normally a Wiki must be maintained and organized, otherwise it becomes a steaming mess, but if it is a focused effort to write a dozen pages which document a specific area, this is not hard do organize reasonable.
The rest is in the code. And 80% of all »projects« lack at least one of the points above.
I know that philosophy well, but I do not really agree with it - not at all. For example in concurrent code written in C, the way things are synchronized is normally implicit. You can understand it if you read all of the code, but not by looking at single functions. If you add accesses without proper synchronization, you will get quickly undefined behaviour which can cause extremely nasty concurrency bugs.
I also agree with the several comments made that it is often more helpful "why" something is done, than "how". This is, of course, valid for in-code comments as well, but IMO equally important in overview documentation.
> And 80% of all »projects« lack at least one of the points above.
Yeah, I know. In fact, I have rarely seen good technical documentation.
Knowing the business and thus rules is the most important thing.
We support a global business and Asia regulations are different from Europe, products are managed differently by different traders, etc.
Knowing the business gives context to what the software is doing and why because this kind of software is written by people who didn’t understand the rules in the first place. It isn’t abstracted correctly and there is a ton of if-else branching for edge cases, variable/function/class naming that is inconsistent and often incorrect, etc.
I would also +1 a known bug list. You can spend a lot of time tracking down a “bug” that seems new because it’s an edge case that rarely appears. Each time it comes up the users think it’s new and a complain.
I am consistently amazed by how quickly these diagrams can convey complex business requirements to non-technical people. I could write 30 pages of email, or draw 1 state machine on my whiteboard and snap a photo of it. Someone should try to run the math on information density of a state diagram vs a verbal or written communication of the same requirements. I would bet there are a few orders of magnitude of delta in there based on my experiences.
It's way faster and easier than doing it in software. This allows you to iterate really quickly using rough sketch diagrams (also helps to open your mind up since it's just you and the board), and then once everyone agrees on the design someone can take your whiteboard sketch and turn it into a computer-generated diagram for the final documentation.
We have actually found very few cases where we need to have 'formalized' diagrams generated using software. Most of them just live in our GitHub issues and pull requests. With enough practice, your hand-drawn diagrams can look good enough to ship directly to a client for approval. If we do need to make a computer-generated drawing, we just use some web tool like draw.io or use a local copy of Visio.
It's open source and also cross platform, since it's written in Java.
It has command-line interface too and integrates with most popular IDEs.
You can host your own rendering server too and just reference your diagrams as URLs (https://github.com/plantuml/plantuml-server).
There is a 3rd party, independent rendering server integrated with a web editor. I recommend that if you want to quickly test PlantUML:
https://www.planttext.com/
I used to work at Microsoft on a team with poor documentation. By my reckoning developers writing code would have preferred better documentation while developer managers couldn’t justify the time to write it. Both preferences were sensible.
From the implementer perspective you could get caught up on a problem for several hours before having a conversation with another developer who could unblock you instantly. If the content of that conversation had existed in some document it could have been searched and several hours could have been saved.
From the managers perspective, product requirements and features changed so rapidly that a few days spent writing documentation could easily create a document that no one would use. Indeed, this was the case in a few experiments.
I lead a team for awhile and we hit the first problem a lot. However, I noticed that most of the conversations which unblocked my team had happened over email (I was CC’d). The tone of emails did not make for perfect documentation but they were helping my team so they were better than nothing. I decided to start taking some of the emails and pasting them into a OneNote document. After a few months the result was a large collection of imperfect documentation created very economically.
The OneNote document was wildly successful:
* Because the notes were from emails which had at least unblocked one member of my team, they had proven value.
* Because developers were already writing emails there was low cost to pasting them in OneNote.
* Developers could also correct the OneNotes and everyone would get the changes unlike an email thread which was only accessible to people on the to and cc field.
I also believe that the imperfections and limited editing features of email made it easier for people to write these documents: they didn’t feel like the documents needed to be well-formatted or perfectly written, if someone had something to say they had unlimited license to paste it in the document.
So thats been my experience and I will bet on it again in the future. The only caveat is that I worked on games which is the cause of the rapidly changing requirements. “Unfun” features could entirely be cut and we essentially rebuilt the product every few years to release a new game obsoleting much of the documentation. This sort of thing may not work for a public facing API for example.
I know that good documentation takes time.
What you write about OneNote is interesting. The only caveat is that using Microsoft Windows would be extremely unproductive in this environment (it is embedded software which needs to run on a Linux platform, so using Windows for documentation creates a lot of additional friction).
I think this basically has the features of a Wiki in which you could insert emails and other help information, and whose organization can be ordered and evolved over time.
Do you have any experience with [Zim Wiki](https://zim-wiki.org/) (a Desktop Wiki software) or Docuwiki (which is tailored especially for documentation) ? I've used the former quite successful for personal information management (it is free software, with free as in "free speech", and it even runs under Windows). What I like especially about it is that it can be integrated really easy into version control with git, or use the existing git version control of the corresponding code.
At some points in my past, I've also used [Dokuwiki](https://www.dokuwiki.org/dokuwiki) very, very successfully as member of a group. For other environments, wikis do not seem to work so well - I am still struggling to understand what makes the difference.
Personally, I've zoomed in on Thinkpads, old business models. They are sometimes a bit bulky, but rugged. They don't break and can be upgraded and repaired well. The old ones have really good keyboards which is important for me as a console user. And they have usually out-of-the box good driver support. And that's something I am expecting today. 20 Years ago, I tolerated a lot of driver problems, as I was a poor student and Linux driver support was often patchy. Today, I expect that it just works. And last, given Microsoft's history with Nokia, Maemo, and the N9, I would not expect any reliable Linux hardware support from Microsoft (Fool me twice, shame on me).
Effective Writing for Engineers, Managers, Scientists, by H.J. Tichy.
https://www.amazon.com/Effective-Writing-Engineers-Managers-...
It was fantastic for me because it describes writing as an iterative process with subsequent rounds of structuring and refinement, and that helped me to overcome blocks. I learned also a lot on how to make my writing more concise and to the point.
For a scientist and programmer, it is really essential that one does not simply assumes that one already knows how to do it! For me, it is way more difficult than it looks - but also way more satisfying than I thought, when it is finished.
I agree that this is not for everyone, because it needs to be learned, is hard to learn, and is a very different set of skills to coding. And there is clearly a field of work for specialists.
One lesson learned that I took away from this is that writing is a separate task, and it is difficult to do it at the same time as producing code.
But back to the main topic. As a documentation writer under time constraints, what could I do to find out what the not-yet-present reader of the documentation might need to know?
2- Understand the workflows within the user interface and then map each one of the workflows / business processes to the logical code and the logical code to the database calls / tables.
3- Once you have built that mapping (tree structure) then you can tackle each one of those functions without feeling lost in the entire code baseline
4- As you work on each of the workflows/ business processes you can start comment the code in your terms
5- IntelliJ provides functions to conduct dependency mapping of the code, and database tables to make sure you don’t have circular dependencies.
6- Once you have completed the walk through and identified the workflows / functions / business processes you can than put a weight on each of those. Large, Medium, Small size.
7- After completing that sizing, sit down with your manager and help him understand what each one of those means in terms of cost, schedule and performance.
8- if none of the above is done, you will have a really hard time coming up with a way to explain to the manager what will it take or when will it be done and how many people will it take if they give you a date to finish. Just because someone wants something done in their time, doesn’t mean it can be done in that time.
Last thought: please don’t get discouraged, if you feel stressed and pressured already, you have lost the battle. Do the best you can with the tools you have, even if it’s not someone else’s best.
I am wondering if it would help to have a document which takes the whole list of requirements from the requirements specification, and explains how each requirement is realized in the code. (Or of course, if it is still missing, in which way and where it is missing, and what would need to be done to add it).
(Edit) Forgot to mention one important factor. Make sure to build a good relationship with your manager so that he understands that you are doing everything you can to make this work. The conversation is better understood when the relationship is there. Keep him aware of the work you are doing so he understands it as well, and he will have the data upfront to share with the customers.
1. Key data structures
2. Code layout/architecture (what modules/layers calls which modules/layers, etc)
I like Blender's example for how to do this: https://www.blender.org/bf/codelayout.jpg
These also tend to be relatively static so they don't need to be updated every month.
I remember reading "Practice of Programming" by Brian Kernighan and Rob Pike (which is a really, really fantastic book!), and it talks about the importance of data structures.
https://en.wikipedia.org/wiki/The_Practice_of_Programming
This comes up again and again, and it is so true! The core data structures are one of the most important aspects of a program. In fact, when I start to write a program, I think about the central data structures first. It is clear, that to a programmer who takes over a project, they need to be explained as one of the first and most important things.
Yes, data structures usually define programs. The code tends to follow more or less naturally from the data structure and the interactions with it.
If you misdesign the key data structures of your program it will balloon in size and complexity. Edge case bugs will swallow you alive.
If you make it just right typically (almost) everything will fall in place. This is still true in 2019, even with all of the distributed aspects that have been added to most programs/systems of programs.
I agree about that book, it's good stuff.
* Glossary for any special terminology that can't just be googled - especially project specific terminology that may be confusing, and double especially terms or phrases which are used slightly differently to the way everybody else uses them (e.g. 'user' means something slightly different almost everywhere).
* Lots of "why" describing why the software behaves in ways that seem surprising at first glance.
On my most recent project I split the docs in two; a 'guide' with example code and output - it has more of an 'overview' feel. The other half is the technical (API) docs built from comments in the source code.
It's not the most exciting project in the world, but it's a Rails form builder that meets GOV.UK requirements. Here's a draft of the guide if anyone's interested.
https://govuk-form-builder.netlify.com/
It's generated via Nanoc which loads the form builder and uses it to generate the example input, rendered output and HTML - in theory it can't be out of date or broken.
Even less arguable is when examples that worked are broken by an update and remain unchanged--they'll work for people on outdated versions but break when they upgrade, and also break for new users, and nothing will sufficiently explain why.
Please add your documentation examples to your testing pipeline (or write tests clearly enough that they can be good examples, and just use those).
This seems like a pretty good practice to me. Are there any good testing libraries for keeping your documentation examples tested and up to date?
https://hitchdev.com/hitchstory/using/alpha/generate-documen...
The same YAML files that are used to regression test the software are used to generate markdown docs for the website.
> Working software over comprehensive documentation
If your application is so complicated it needs documentation to understanding it, there is a problem with the way the application is structured. In the magic month I'd get the application broken down into independent parts which can be developed and tested separately.
Or a problem with [edit: with how some people interpret] Agile. In this case, it is the latter.
The agile manifesto doesn't say that you shouldn't write documentation. That's an incredibly damaging idea.
2. What specific technologies are in use (languages, frameworks, platforms, third-party services, etc.)?
3. How does configuration/security work? In other words, where are keys/tokens floating around that I need to make sure don't get into the wrong hands or misplaced?
4. What are the best resources/references for the technologies in use?
In essence, what's the landscape and where are the "breakpoints" that could wreak havoc for customers and the business?
Documentation should also provide mental shortcuts. Rather than forcing you to figure out what algorithm the code implements, it could just tell you.
It should also tell you things that are surprising, and might accidentally miss or misinterpret.
Documentation should tell you information that you can't just as easily glean from the code. A `doTransmorgification()` method doesn't need the comment `// does transmorgification`.
Documentation shouldn't attempt too tell you everything with one form of documentation. Documentation on a line or method or class is good for one sort of information, but it doesn't cover the overall architecture of the program. That is often best left as a separate piece of documentation.
It should consider the reader. Is the reader looking for an introduction to how to use this library, or are they looking for what property the method has in some edge case?
Documentation should use the easiest to follow, least technical language it can without being inaccurate. It's not an academic paper; it's an explanation to your fellow developer.
Documentation should tell you how the responsibilities are divided up in the code. If you implement an interface, will you need to record metrics or will that be handled for you?
Documentation should cover any processes you need to follow when making certain kinds of changes. If you need to do something in two separate deploys, those steps should be written down.
First separate it out in what the different processes are that are running in the working system. How are they communicating? What is their function? Why was it separated in this particular way?
For the more difficult of these programs/processes: what is their rough structure and what function does each part have? Why are they difficult? What strategies were used to conquer this difficulty? What are the most common ways in which they get extended?
How does the build work?
The worst documentation I had to parse recently was react-beautiful-dnd, and I’m saying this somewhat conservatively as it’s evident a lot of effort has went into producing something comprehensive. But I was paralysed by the volume of it, reams and reams of methodology and design decisions, how to contribute, and the history of the project. In the end I had to use one of the community codepen examples to get where I needed. Without meaning to stereotype too heavily I see this mostly in junior friendly React projects where you have more people contributing emojis to markdown than you have architecting your APIs. In case anyone from react-beautiful-dnd/Atlassian reads this, you have all the documentation you need, but make it less like a textbook and more like a two page CV.
The best documentation I’ve seen is spider-gazelle (crystal). It’s the prefect balance of plain-English explanation and example code. It probably helps that is was architected and written by the same person, and they were able to really hone down on exactly the right information to transfer.
Edit; I’ve just realised that your specific example is for an internal enterprise project, rather than a public facing library so the variables you optimise change slightly. I’d be maintaining and developing this software as a contributor, and the documentation is specifically for on-boarding me. I’d want to see more of the architecture rationale, point-in-time thought process, and explanations of business constraints that led to certain decisions being made (e.g. what prevented optimal architecture first time around)
Sorry for an interjection. The documentation I am having in mind is technical documentation for future developers of a code base (its implementation). Not users of the API a code base.
Also, it would be possible to document the wrong stuff or stuff that people really already know, but it is basically not possible to document too much, because of the time constraint.
If internal servers are involved, the way to setup these servers from scratch should be included too.
Now that I have all the parts, I can deal with the rest.