107 comments

[ 2.8 ms ] story [ 192 ms ] thread
This is such a valuable recommendation. I've been doing the same since 20+ years back. On and off I've been bad at it, but I always end up coming back and continuing.

I think of it as more of a daily diary though, I mix random notes of things I've done, thought of doing, or interesting links and/or command-lines I've found useful so I don't have to reinvent them as often in the future.

It has proven to be immensely useful for myself. Can highly recommend!

My way is mainly org-mode files through Emacs over the years. Now having switched primarily to VScode I use a simplified org-mode version to get similar feeling, and it works pretty well. Everything is mostly text anyways!

The author mentions utilizing Slack for this; do so with caution. The free Slack tier only keeps 10,000 messages in history with the added drawback of now your knowledge stash is stored on servers with a 3rd party company rather than files you own / control.
I think Slack actually keeps all your history, it just hides older messages from you. If you subscribe to the paid tier, you should be able to see the older messages.
It’s not writing that’s hard, it’s finding it later, and more importantly remembering that I wrote something down.
Agreed. I wonder, what might be some mechanisms that would aide in finding things later? I feel like i often forget some keyword to find history items like this, but i _can_ find something near by.

So maybe a way to narrow your history scope?

I have recently made the switch from using Mac's built in Notes app to using obsidian.md which focuses heavily on linking. It seems to be quite good for searching and recalling information. Although I have been using it for about 3 weeks.
I have a few GitLab projects I use for this. Some are actual code things, and a couple are just for notes. I make issues all the time along with using the wiki.
1989 - 2007: notes.txt, 2007 - the future: notes.md
I'm still wondering like where should it be. We all carry a phone with us. Seems like adding an extra thing (like earbud holder) sucks... servers "what if it went down"... At the moment I use different combos.

I have made a few things, like a url-rest thing that's encrytped at rest where the entire website page body is a textarea element. Authenticated and unauth (local) versions. Desktop app. Phone note widget. One I'm working on is a chrome extension one since I use Chrome everyday.

The main thing is it has to be made by me, ego. I used to use One Note but now I'm scared/cringe to look back at my younger self's thoughts ha.

I use Evernote for this.
(comment deleted)
I use Obsidian[1] pointing at a local directory of markdown files which is also synced across my devices via Syncthing[2]. Works flawlessly across my various Android/Fedora/Windows/etc devices. Cost: $0.0/year.

I don't really use any Obsidian plugins. Obsidian has good builtin navigation, search, and markdown processing (e.g. LaTeX, code syntax highlighting). It's just as easy as opening a new file in $EDITOR, except its a markdown renderer & synced across all my devices.

There are also a lot of bells and whistles like graph viz, export to website, etc. that haven't gotten in my way.

[1] - https://obsidian.md/

[2] - https://syncthing.net/

Obsidian has LaTeX support without using a plugin?
It supports MathJax without a plugin, so at least a subset of LaTeX.
(comment deleted)
Inline (markdown-embedded) LaTeX, yeah. You can also export to PDF. But I would definitely not use it as a full-blown (La)TeX environment.

Although, it is a perfectly good environment for drafting such manuscripts.

I do this with Obsidian + Git - I'll check out syncthing. Thanks!
Checked out syncthing myself after seeing this comment... What a great utility! Thanks for suggesting.
(comment deleted)
I use the same setup + logseq. Gamechanger
What's the benefit of using logseq?
Logseq has a build in query language. You can query every block of information.

It's organised around blocks rather than pages. So you tend to organise your thoughts in packages.

And you can reference blocks in your current page so you have a direct link. Though I think Obsidian does this to.

The differences are quite subtle but profound.

Understood. Do you use anything to sync data between both tools?
> I use Obsidian ... Cost: $0.0/year.

For the use-case of a professional developer employed at a company with more than two poeple, it is $50.00 / year [1]

[1] https://obsidian.md/eula "Get a commercial license for each user" -> https://obsidian.md/pricing

The easiest way to incorporate this into a typical developer's workflow is using the vscode-journal plugin: https://marketplace.visualstudio.com/items?itemName=pajoma.v...

How it works: From any file, in any project, in VsCode just press CMD + SHIFT + J, select 'Today' and it will plop you into the journal page for today with today's date at the top.

I've been keeping a 'work diary' in VS Code - tried a couple of simple notes plugins but I ended up just pinning my diary.md file in my main window... always easy to get back to it.

I think the only other QoL I have for it is the markdown snippet I created that creates the current day's date and other boilerplate data when you type 'td'. It works really well for me, and I back up to a private git repo.

Emacs org-mode is better than Markdown.
+1 for Emacs Org-Mode.

But of course you'd have to have bought into the Emacs ecosystem. Then again, Org-Mode may be a good first step!

Evil mode makes org-mode very approachable for anyone familiar with vim. Grab doom emacs or spacemacs if you want it to just work.

Also, Elegant[1] is an attractive and simple config. It would probably be worth packaging it or something very similar up with CUA bindings and marketing it as just a standalone notetaking app with Emacs just being an implementation detail.

[1] https://github.com/rougier/elegant-emacs

I find I always fail out of the append-only approach (even though I realize it means I'm spending more ~orgnizational time/energy).

I can't claim to do it "regularly" in the sense that I do it every day, but I guess I do something like it in that almost every one of my projects has one or more PlainTask documents (i.e., documents that use org-mode-alike PlainTasks extension for Sublime Text).

These documents usually start one day as a scratch space to remind me what I was doing before I take a break, or to plan out the next few steps. In any non-trivial project, it'll tend to grow to at least a few hundred lines, and almost all of them accrue a top-level section, at the top, titled KNOWLEDGE, where I (my better self, at least) try to record slippery or hard-won technical details I picked up along the way (even if I'm not smart enough to note them the first time or two I have to bang my head against them).

I may not consult most of them again, but they can be a huge help when I get a chance to return to something that I had to back-burner 2 or 3 years earlier, and all of the critical detail has been displaced by newer ephemera...

Since about 2001 I used YYYY-MM.txt plain text files and have a shell script to help create notes in the most friendly way I could think of from the command line at https://github.com/nickjj/notes.

Totally works fine for a knowledge log when you're streaming high level details from your brain into text. I still use it today. The repo hasn't been updated in 2 years because it's a ~20 line script with a simple API that hasn't failed yet.

But when you want to really go all-in with in-depth notes on 1 semi-broad subject it's tricky because in 1 month's time if you're hardcore deep in the woods of learning, applying and using something you're going to end up with hundreds of concepts from an assorted set of tools and it kind of stinks to have all of that info sitting in 1 file. Think about using something like Kubernetes. That's really Kubernetes, Kustomize / Helm, EKS, various cloud hosting details (networking, etc.), Terraform and ton of super useful commands / context. Details you for sure want recorded for later.

For this type of info I've been building up a personal knowledge base with https://obsidian.md/. It's really nice and I highly recommend it. It's been working well for keeping things reasonably categorized without wasting a lot of time on the details around keeping links and tags up to date. It also has Vim mode that's good enough where day to day writing feels natural.

Hey Nick! Thanks for linking to this! Had seen this once and forgot to bookmark it! While reorganizing all my knowledge notes lately just kept racking the brain on who created it.
I use a tool called stup now and was going to recommend it but now I’m intrigued by this.
I love this! And have been doing something like this for years.

The zero-effort part is key in my IME: if it's not zero effort, it won't stick. And if it doesn't stick, you won't reap the nonlinear long term benefits. There's the obvious benefit of magically reaching into the past and remembering things. But the hidden power that surprised me was the day-to-day experience of clearing cognitive clutter by typing things out in a place that I trust won't get lost.

Shameless plug: two years ago I hacked together a little node script to give my pile of markdown files a more friendly UI than grep: it would pass them through pandoc and massage the DOM and slap a a bit of JS on it. I've been using it everyday since and it's become my favorite productivity hack. I'm currently in the process of rewriting that ole hack in Go to make it more stable and easy to distribute (and to learn Go!): https://github.com/amirkdv/codex

Hate to nitpick but #!/bin/bash is not actually zero-dependency, as bash is not installed on BSD by default, and if installed won't be located at /bin/bash.

You could probably just change to #!/bin/sh without issue.

Thanks for bringing this up. It does mostly work the same with shell but the -p flag of read isn't available with POSIX compliant shell. I ended up changing that slightly and pushed a patch to use sh now. Every other feature of the script worked the same with sh without modification.

As for using #!/bin/bash instead of #!/usr/bin/env bash, that was an oversight from 2 years ago. I addressed that too in the above patch by using #!/usr/bin/env sh.

I failed to do this throughout my first career. It’s one of my biggest professional regrets - perhaps the biggest. Reader, don’t be like me.
My append-only system for *nix, sits in my .bashrc:

  # Open Personal Logfile at end, optionally appending log entry for today if it doesn't exist
  function ed ()
  {
    local _cur_date _most_recent_date;
    _cur_date=$(date +%m/%d/%Y);
    _most_recent_date=$(grep --color -o -E "^# [0-9]{2}/[0-9]{2}/[0-9]{4}" ~/${LOG_FILE_NAME} | tail -n 1 | cut -d' ' -f2-);
    if [[ "${_cur_date}" != "${_most_recent_date}" ]]; then
        echo -e "\n#\n# ${_cur_date}\n#" >> ~/${LOG_FILE_NAME};
        echo "Command                                 Comments" >> ~/${LOG_FILE_NAME};
        printf "%0.s-" {1..81} >> ~/${LOG_FILE_NAME};
        echo >> ~/${LOG_FILE_NAME};
    fi;
    $EDITOR +$(($(wc -l ~/${LOG_FILE_NAME} | awk '{print $1}')+1)) ~/${LOG_FILE_NAME}
  }
  
  >>>
  
  #
  # 12/03/2021
  #
  Command                                 Comments
  ---------------------------------------------------------------------------------
  <some cmd>                              # My description of what cmd does, in my own words
Churned through a lot of ideas/apps for this over time, what works (not 'best' but 'well enough') for me right now is LogSeq.

- Daily notes as roots

- Local storage

- Plain text storage

- "wiki-links"

- Backlinks and easily traversing notes

- Hierarchical format for notes (with folding, so I can "tuck away" a bunch of text and screenshots when I don't want to look at them)

Me too! It's really nice, and integrated version control by git is great
Ah.. time for the monthly note taking generic PKM HN post. Has nobody mentioned Zettelkasten yet? I see org-mode already made an appearance... Either way you go, don't forget to self-host!
If it's not plaintext and client-side encrypted, is it even worth writing down?
No one mentions indexing the content as a really important part of logging your notes?
There is a documented phenomenon called the Photo-Taking-Impairment Effect, where taking photos of something can reduce your ability to recall details about it at a later date. It was originally theorized that it was a sort of 'cognitive offloading', but further tests have shown that reviewing and deleting the photos does not make up for the deficit.

https://www.sciencedirect.com/science/article/abs/pii/S22113...

I remember finishing a complex math test in high school and checking my calculator's memory to find that I had done some rather simple single digit multiplication that I had no recollection of. Ever since then I've tried to be conscious of what I offload and what the long term consequences might be. For example, I'm confident that my ability to spell has decreased as spellcheck technology had advanced.

I'm not recommending against keeping a log. In fact I keep several myself. I use google keep for simple things that I eventually want to forget. I use anki flash cards for spaced repetition study of short concepts that I want to remember. I use oneNote for sprawling interconnected datasets. And lastly, I used waldenpond.press to print and bind stories and articles that I want to remember and revisit.

I just think that there is a risk to haphazardly externalizing more and more of your cognition. I started a new carrier 5 years ago, and found that many of my skills had atrophied over the years. I think I spent more time in my first 6 months relearning abilities I had taken for granted 10 years earlier than my new job duties.

This is why Socrates refused to write things down. As he said, writing is a crutch for memory. Just as a man who leans on crutches will have his legs atrophy, so will one who relies on writing have his recall atrophy.

I on the other hand am a prolific notetaker because my recall isn’t eidetic as it was when I was younger and I need the crutch to operate at the level I became accustomed to.

I hate when you think you've made a good point only to find that Socrates beat you to it! I think his sentiment is valuable, but I don't think that kind of extremism is the answer for most people. There is a fine line between a crutch and a ladder, and if you don't take the time to figure out which you are using you will find yourself spending more time leaning than climbing.
Crutches normally connote injury, but I bet a non-injured person who actually did athletic sports on crutches would become really dexterous with the things. Four points of contact with the ground. Or they could be used to assist when balancing.

Anyway, I don't think the crutch model is correct, even at the risk of disagreeing with Socrates. I think we just learn some way to learn, and part of that is a memorization ritual. Socrates learned to load things into his long term memory by discussing them. The vast majority of over-30 (at least) American who learned to load things into long term memory learned to do so by the ritual of writing them down with pen and paper. I'm sure there'll be a contingent who learn a ritual of typing notes.

Writing often improves memory and retention. I understand the sentiment but I believe writing something down (even in a digital way) improves recollection because of the physical movement involved which is (distinct from each written item).

A picture, and typing, is the same action regardless of the substance being captures. There is not enough uniqueness to the process to help memory. But writing I believe is.

Memorizing facts is a pointless exercise and waste of cognitive energy because there is too much information and it's easily replaced by inorganic storage media. What's more valuable than winning Jeopardy! or being a London black cab driver is 1. developing critical reasoning skills, learning skills, and mastering first principles and 2. offloading important factual details onto searchable, error-free documentation that others can use. It's far more useful and reliable than fallible, car-crash-vulnerable, knowledge-hoarding.

Crossword puzzles and memorizing lists won't stave off dementia either.

Knowing facts personally is very important. Obviously the type of facts you're describing here are usually not useful. However if I had to look up the syntax for a while loop that would be horrible! Imagine if I had to look up what looping structures were available! Your personally known set of facts is a huge factor in how effective you can be in any given endeavor
Very true. Being able to do routine things quickly and accurately without much mental effort reduces cognitive load that makes capacity available for tackling larger more complex problems. The other thing to note is that we are not only limited in size of what we can hold in our head, but also time. Not having to look up an easily found answer means that we can get to a solution before running out of time.
This is a really great point!! It's not only about knowing facts, it's also a bit more meta and it's knowing what to know and what's worth jolting down somewhere
If you use the loops in this particular language often, then you memorize it just by practice along otherwise there is nothing wrong to look up a syntax for a language you use once in a blue moon (f.s.)
> there is nothing wrong to look up a syntax for a language you use

Of course not. My only statement was that the person who doesn't have to look it up will be (all else equal) more effective.

The effort to memorize the syntax may exceed the number of times you actually use it--wasted effort is not efficient.
If you are coding in 5 different languages at work, then I am afraid you will have 5 cheatsheets open in front on you, no matter how good your memory is. Cheatsheet access can be faster than human memory access, especially for the infrequently used stuff.
Disruption and creativity are often best when pulling arbitrary knowledge from between different domains/industries.

I keep a key list of links, but more importantly a strong basic reference knowledge of everything I've read, to be able to quickly find the more in-depth knowledge when needed.

What I've read is that e.g. doing Sudoku puzzles won't save you from dementia, but you'll be starting from a higher point so that you'll be older when you reach a certain diminished level than if you didn't exercise your abilities beforehand--use it or lose it (sooner).
> Crossword puzzles and memorizing lists won't stave off dementia either.

No, it actually will, since your brain will have more connections so it will take longer for it to degenerate to the point of dementia. So as an example, someone would develop dementia at 85, but the brain activities will push it to 20 years later, probably beyond the death point.

I think maybe it is underestimated how much cognitive load there is in taking a photo. All your attention is going into the act of taking the photo (is it level, is the light right, is it focused, etc.)
That and more importantly that you unconsciously aware that it's saved somewhere that you can recall / review later. So your brain doesn't store the information to long term storage.

In terms of computing, it was only stored on ram, and not going to hdd / ssd, and later will be flushed to be replaced with other info.

Which is not a bad thing if that information isn't crucial for long term.

A lot of what you're writing rings true. That is, if one is rote copying or "offloading", you're less likely to retain or truly understand material. I can certainly relate.

But on the other hand, concepts you're building up in interconnected data sets (one-note etc), or anki cards you author _yourself_ are likely forging new neural pathways via the 'generation effect'. They are more a case of 'note making' than 'note taking'. In these cases, when you're reviewing, you're likely not just recollecting random unrelated facts, but re-experiencing the understanding you built up whilst authoring.

Perhaps thats a nuance to the risk that you're highlighting: whatever the medium (anki, org-mode, etc), there are risks to our cognitive abilities atrophying over time if we skip that generation phase.

The fact that certain skills atrophy if not used is the brain prioritizing efficient allocation of resources. If you're not making daily and active use of relevant concepts then the only way to reliably remember them is to use spaced repetition on carefully crafted question answer pairs.

Task relevant context residing in an [intermediate-term memory](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6596466/) is built up over time as you gain expertise, it's more durable than working memory but impermanent, unlike long term memory. According to this model, fine details are unfolded in working memory and details not relevant to current attentional focus shift into latent intermediate storage.

Those structures locate the expert at a point where they are quicker and more effective in solving domain relevant problems but also bias which paths are taken in problem solving (such as not noticing you're using a calculator for trivial arithmetic). It's loss is why the same person can write complex algorithms or mathematical proofs and come back months or a year later and be unable to make heads or tail of it.

It's quite likely that long term memory does not store at the same level of detail as intermediate context, relying on inference in recovering into WM. Keeping a notebook of detailed derivations of unintuitive or complex workings reduces the complexity of inference during recall and helps for more successful and reliable recovery of desiccated memories. Notes are also important for things you expect to fall out of intermediate context due to rare use but caused a great deal of headaches to solve.

Tools do have a cost but good tools are net positive. Good tools will be pivotal in an era of exponentially expanding knowledge, fixed human capacities and slowed population growth. From oral stories to books, from the abacus to computers, humans are unique among all animals in how much we extend our cognition beyond our brains.

It's important to note that "at a later date" means 10 minutes... not exactly a longitudinal study.
I dont know if this is what you are talking about but I have noticed so many times that when trying to recall an event I took a photo of, first image I get in my head is that photo. I try build the rest around that photo and it only works for very old photos/memories. For recent ones (e.g. photos of my newborn) I only see the photo in my head instead of the actual visual event.

This makes me sad when I try to remember seeing him for the first time, I can only remember a few things and lots of photos. Now to mitigate that, I still take photos but avoid reviewing them immediately. I would rather go through them later when trying to recall. When I review immediately, it feels like the photo overrides the visual memory of the real event.

This study is interesting, but very narrow and might not be relevant to a lot of practical situations. In particular the reverse effect is observed on other subjects, manual note taking improving memory for instance.

I think the important part here is that the “log” isn’t the goal nor something we are explicitely asked to do, but a step that comes after we experienced the work. I would compare it to taking photos of your own painting: arguably you wouldn’t be remembering a piece less because of the photo, you’ve been staring at it for hours/days in the first place.

I very often fail in identifying the things that need to be added to the log. List running docker containers? So easy it's not necessary to log. That one obscure bug that took days to fix? So many things were tried that didn't lead to success, that the path to the solution is not clear anymore.
Five years ago I decided to buy rocketbook, which I use as a sort of journal. Obvious things go in there, like meeting notes & action items, but also notes from watching conference presentations, weekly Todo list tracking (ala bulletjournal), and outlines for phone calls I need to make (I find the preparation helps with the anxiety of making first phone calls, by collecting important data i might need and key points i need to accomplish ahead of time). I think I prefer the analog nature vs digital notes apps, and the scanner app helps with posterity concerns. Most notes are not that important a year from now! I used to take notes while reading books, but I've since moved that to part two of the system.

For things where I absolutely want to remember something, it goes into Anki. While reading books, I make time to go over the chapters and extract key ideas into Cloze deletions. Helps with recall, and the Cloze format helps with recall since you're given far more context. And perhaps useful for search.

For shell / computer stuff, keeping history around helps a lot. In theory important commands could become aliases or functions and kept around via git, but this is more rare.

I do have a gnote / tomboy setup that I use mostly for long form writing, but that's less about journaling experiments & discoveries for later reference and more about getting plans and ideas out of my head so I can stop thinking about them, and drafting the occasional blog post.

Glad to see this has ressonated with so many of you!! Thanks for all the amazing insights in the comments where, as always, a lot of the value lies.
I use simplenote for this. Mandatory for me is: - synchronisation across multiple machines - synchronisation on android - markdown - reasonable search - sorting by date

Over the last few years I have tried many applications for this and many have failed.

I really liked boost, but the whole paid/free/app/webapp mess with whole backups disappearing freaked me out.

Right now I'm trying to decide exactly how much information I should store in my knowledge base. There's a balance to keeping the base small enough to actually find the information needed and avoid it becoming a huge bloated mess.

For example, I keep separate Notion pages for each programming language I use. I also recently started learning Rust and have built up a page that includes even basics like how to declare a variable. The idea was it can exist as a beginner guide and someone can run through it relatively quickly and get a jist for the language.

But if I keep using the language, that information may eventually become obsolete (since I'll have it memorized) and will probably be a waste of space. Then as I advance through the language, I'll want to take note of more advanced concepts and maybe even some specific use cases of problems/solutions to code I'll have written.

I guess that's the beauty of editing a knowledge base over time. You can edit it to conform what's in currently in your head right now. So maybe in a few years from now, having those simple variable declarations will be a waste of space and I'll just want advanced topics covered.

Or maybe I'll walk away for a few years and need to look up basic syntax again.

In my experience there's not much risk of having a "huge bloated mess" so write more rather than less.

I keep all my work-related knowledge in just a single flat text file, which after four years at my current company is just under 20,000 lines long and ~750 KB. It's easy to grep and so I can find virtually anything quickly.

I think trying to get too fancy with organization could lead to troubles. A badly-organized knowledge base is hard to fix.

Most of the time I'm just kicking myself that I didn't write down something that I was sure was in there.

I've been using GitHub Issues for this for the past few years, and I absolutely love it.

Each of my projects has issues. If I'm fixing a bug or working on a feature I open an issue for it, then I post a stream of comments as I figure out the details, make decisions, find examples of code that I need to use and so on.

I have my own private notes repository which I just use for this for things that don't relate to a current project and that I don't want to publish yet.

I add a "research" label to my repos, and open research issues for ideas or for things that I want to explore that are relevant to the project in some way. Recent example: https://github.com/simonw/sqlite-utils/issues/344 - researching STRICT mode that was just added to SQLite, and seeing if it would make sense to support it in my sqlite-utils library.

GitHub's issues API is comprehensive, so I have a cron that exports all of my issues and comments into a SQLite database that I can access with Datasette. I have a private one with all of my projects, and I also run a public one at https://github-to-sqlite.dogsheep.net/github/issues with just the issues from my public Datasette-related projects.

I use search against these often. I occasionally even re-discover old notes from Google searches, which include the ones from my public repositories.

Many of my issues have dozens of comment replies, all from me. It's a great way of keeping a log.