Ask HN: How do you organise/integrate all the information in your life?

323 points by tonteldoos ↗ HN
Hello fellow HNers,

How do you organise your life/work/side projects/todo lists/etc in an integrated way?

We have:

  * To do lists/Reminders
  * Bookmark lists
  * Kanban boards
  * Wikis
  * Financial tools
  * Calenders/Reminders
  * Files on disk
  * General notes
  * ...
However, there must be a better way to get an 'integrated' view on your life? ToDo list managers suck at attaching relevant information; wikis can't do reminders; bookmarks can't keep track of notes and thoughts; etc, and all the above are typically not crosslinked easily, and exporting data for backup/later consumption is hit and miss from various services.

So far, I've found a wiki to be almost the most flexible in keeping all manner of raw information kind of organised, but lacks useful features like reminders, and minimal tagging support, no easy way to keep track of finances, etc.

I understand 'best tool for the job', but there's just so...many...

271 comments

[ 3.3 ms ] story [ 227 ms ] thread
Org mode for Emacs does most of that.

Even finances and accounting: http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-l...

I have a wiki currently, which is easily accessible and updateable from anywhere by web. Is that something that org-mode can provide, or do I need to ssh into a machine (ie, harder from a mobile device)?
There are some apps but I found that sshing to another machine was the best solution. An alternative is to run Emacs locally.
Absolutely agree - I've got most of my stuff hosted such that I just need SSH (or a VPN) to access it. However, usability on a mobile device in this case is limited (albeit not impossible).
Usability is not a great problem if the phone has a big display (5.5>) and you use evil(vim)/evil-leader for your keybindings. JuiceSSH and mosh also help with latency and disconnections.

The main problems for me are:

    1) Keeping everything synced. Especially Emacs's state. 
    2) I don't trust a VPS with all my personal information.  I have *everything* in org-mode files.
The first problem can easily be solved by using a tmux session that is shared between smartphone and laptop. The drawbacks would be that I can't access said session without Internet and also that I lose emacs's gui features like the pdf-reader. x2go is fast enough but they don't have an android client. I have yet to try nomachine.
Org mode can easily generate a static site from all your content. I've never tried remote updating, but Org mode data is all just simple text files, so you could just use Dropbox or whatever.
After years of failed attempts I gave up trying to organize everything and started to brutally eliminate most of it. It has been pretty liberating.
That is hopefully a few steps further down the line :)
I found that my bookmark lists, to do's, notes, personal project plans, everything went stale after a time anyway, if not acted upon. It something wasn't important enough to read, study, or develop, then it really wasn't that important. Thus letting it all go was much more palatable.
Exactly. I see bookmarks as a kind of todo now, which completely changed the way I organize them. (I do have a single folder called "reference" but I almost never go in there).
It's funny - I had the same insight and decided on the opposite solution. Many of bookmarks are an 'aging' zone, where content sits around until I'm bored and interested in reading it, or decide it no longer looks appealing and delete it. Things like speculative HN articles go there, and 90% of them become irrelevant while the other 10% become more appealing.

I also have a reference folder for bookmarks, which I treat a bit like old tax returns - things that are rarely useful, but vital if I need them.

Yes, interesting how the same insight can lead to different approaches.

I used to work with a guy whose desktop was a complete mess and he would never try to organize the chaos. He had high screen res and it would take him a minute or two to find something on his desktop but he could do it. Then when it got full he would take the entire thing and paste it into a new top-level folder, called "old desktops". Of course this included his old old desktops folders, recursively. So the old desktops hierarchy became like archeological strata for everything he had ever done.

That sounds like hell to me. But on consideration, I can't judge; with good search features I lose all drive to organize my files cleanly, and only bother locating things I won't remember the name of.
I used to hate getting rid of anything. Then I realized that for most of this stuff, odds were I was never going to look at it again between now and when I die. It's kind of morbid, but makes it easier to get rid of that folder of things I meant to read from 3 years ago, or that box of artifacts from my past that has remained unopened through 3 apartment moves.
I've found that moving is a great way to face the reality that a lot my stuff isn't needed. Especially, as you mentioned, the stuff that's remained in boxes from previous moves.
I've taken to using bookmarks as an 'aging' chamber. I used to plan on "going through" my bookmarks, which was an imposition. A this point, I bookmark interesting-looking things I don't want to read presently.

If they're still interesting later, they get read. If not - and this is maybe 80% - they get deleted. Storage is easy, so it's pretty satisfying to keep things and just not worry about deleting them later instead of organizing.

I'm going through the phase now of simplifying my life and getting rid of stuff.

Very difficult. I'm by nature a very nostalgic person. Hard to part with items that brought me so much joy.

However, I came to the realization: Those items, while they gave me joy in the past, are literally preventing new experiences, and new sources of joy.

Need to make room for the future, and the only way is to throw stuff away.

This is literally what I'm doing today with the guidance of Marie Kondo's "The Life-Changing Magic of Tidying."
I like the book. However, I wonder how effective it is in the long run. Is it legitimate or is it just selling a dream?

A while ago I posted a question on all the top reviews of the book on Amazon: After 6 months/1 year, is their house still tidy, or was it not really sustainable? Very few responses, but I think one person said it was sustainable, and no one said it wasn't.

Read it a year ago, my sock drawer is still tidy...
OK, how about the rest of your house?

My mess is usually not with clothes, but with tables full of stuff, cabinets full of stuff, garage full of stuff, etc.

Did you go through a phase of getting rid of stuff, and then in the subsequent year, managed not to add more stuff to fill in the space you through away?

(And how bad was your house/apartment before KonMari?)

Honest question.

TBH the sock drawer is the most successful outcome :) The rest of the house - I threw out a lot of stuff, but didn't bottom it. It was definitely worthwhile and I need to resume. The problem is, you're not allowed to throw out other people's stuff; 2 kids and a husband quickly fill spaces and what the book says about them learning by your example doesn't seem to have worked...
(comment deleted)
> However, I came to the realization: Those items, while they gave me joy in the past, are literally preventing new experiences, and new sources of joy.

Oh wow, thank you! I've had the same problem you mentioned, extremely nostalgic, with a side of "emotionally connected/attached to object", and this mindset and perspective is, I believe, exactly where I need to be, and remember, when weighing the keep/let go decisions.

OneNote + Outlook works okay for what you're asking, but I have a hard time sticking with it because the interface isn't well-refined.

I like OneNote a lot more than I like Outlook, but the integration is reasonably good. You can create tasks from OneNote that give you reminders in Outlook, which link back to the rich-text "source material."

They are good for a fair few things (I looked hard at things like OneNote and Evernote in the wiki role, but chose a wiki because of crosslinking), but the data is hard to extract from their formats.
You can cross-link in Evernote and OneNote. Also, Evernote has a great feature called Context which is activated if you pay for a subscription (worth it) that automatically shows you relevant notes based on your current note. Also, Evernote is a lot more portable than a Wiki so you can carry it around and search through all your notes (including inside PDFs and other documents, OCR and images).
I Google Spreadsheet almost everything. All my files I keep in Dropbox. Code on Github and Bitbucket.

I can be up-and-running on a new system in under an hour. Just install my IDE's, get the Adobe CC from the cloud and get started.

How do you deal with calender and finance related issues? I wish I was just referring to code here, but...it's everything :-/
Automate everything. Pay all bills with autopay. Recurring transfers to your savings accounts. Auto invest in index funds.
Sure, but how do you keep track of what you're doing? Or some random invoice that came in? Or the reminder to pick up the drycleaning tomorrow?
https://www.google.com/keep/ is fantastic for reminders. It's integrated into Google Now (So "Ok Google, remind me to do x at y") and can do time or location based reminders. You can also share tasks.
Emacs.

   - Todo lists and reminders. org-agenda.
   - Bookmark lists. org-capture and org-protocol.
   - Kanban boards. I don't use this, but kanban.el.
   - Wiki. Org-mode files and grep/ag with helm.
   - Financial tools. ledger.
   - Calendar/reminders. Org-agenda.
   - Files on disk. dired, org-mode.
   - General notes. Org-mode.
   - Literate programming. org-babel.
   - Mail. mu4e.
   - rss. elfeed, gnus, or rss2email.
   - git. magit.
   - irc. erc.
   - ...
This should be the best approach. I thumb up.
Can you set up phone alarms/reminders with org-agenda or something similar?
I can think of two easy solutions. One would be to synchronize Google Calendar and org-agenda and let Google Calendar manage the alarms/reminders in the phone. This already exists.

The other solution would be to develop an app which polls emacs/org-mode for alarms/reminders. AFAIK this doesn't exist but it would be pretty easy to make.

Can org-mode sync to CalDAV servers? Then you could use eg. Sandstorm.io's hosted Radicale app.
Org-Mode is the one thing that makes me want to move from vim to emacs.. but I just don't feel like investing the time in getting proficient with emacs at this stage.
You can try Emacs with evil (extensible vi layer). It's still a major time investment though.
Only because it's hard to learn to configure emacs. With spacemacs, it's less of a problem, since things work sanely out of the box.
With spacemacs, …, things work _insanely_ out of the box

Honestly, you don't need any extra stuff, other than `evil' if you want vim bindings. (`Viper' works fine if you grok vi.)

(comment deleted)
It's not as difficult as it seems. Actually I'd say that if you already know GNU Readline shortcuts (which are very helpful on e.g. Bash), it's an afternoon of work to learn the basics.

The problem is that most tutorials depict emacs as a bag of keyboard shortcuts. But in reality, there are some simple principles that are quite understandable.

And org-mode is so underappreciated. You can turn it into anything. I use it as a wiki and a kanban board. And it's amazing.

>The problem is that most tutorials depict emacs as a bag of keyboard shortcuts. But in reality, there are some simple principles that are quite understandable.

Do you know one that doesn't ?

Unfortunately, I don't. But I plan to write one soon.
Please do write it! What I love about vim is that it is (to me anyway) a DSL for editing a document. The language has verbs "delete, copy, change" and nouns (paragraph, document, line) etc. that you can learn and combine. So once you understand how the language works, you can do a huge amount with it.

I'd love to see a similar thing for emacs, outlining how it is supposed to be used and explored and fits together.

Do you mind telling me how you use kanban? I tried kanban.el but couldnt get it to update and populate the table, is there any way that emacs can auto-update live the kanban board based on todo states of items?? (e.g. if you hit shift + right for todo it is automatically seen in the todo column, without having to revert the file?)
Sure. I don't like kanban.el much.

After a lot of though, I had a little epiphany. Vanilla org-mode is great as a kanban as long as you don't try to visualize it as a table. One should use an outline instead. org-mode is an extension of outline-mode, so this should not be surprising.

Then everything is simple. Create custom states (I have TODO, PROG, WAIT & DONE). Agenda commands show you summarized views of the kanban. You can customize stuff with a bit of elisp, but org-mode as it is works great.

Note the outline version of a kanban is still 2D, as a table, but strictly more powerful (as you can nest tasks, add text) which would be next to impossible in a table.

evil (extensible Vi layer for Emacs) made it (switch vim -> emacs) possible for me
You don't have to switch editors. For the past few years I've moved between rubymine, pycharm, and atom for coding while still having emacs open for my org-mode notes.
OOC, why not also code in Emacs?
(comment deleted)
Evernote + Calendar + E-mail. Calendar and e-mail both provided by Fastmail.
GTF(Getting Things Done) method with OmniFocus

Boagworld has a great video on his setup and how it all works: https://boagworld.com/working-in-web/omnifocus-2/

Getting things done book: https://www.amazon.com/Getting-Things-Done-Stress-Free-Produ...

OmniFocus: https://www.omnigroup.com/omnifocus

i really like the Omnifocus app, however i dislike that the only way to sync across devices is to use their private servers, which means sending/sharing all your private stuff with them.

Technically they could easily sync via iCloud and encrypt the stored data, but of course for financial reasons companies prefer to be able to access and sell all your data...

Omnifocus does [end-to-end encryption](https://www.omnigroup.com/blog/omnifocus-now-supports-end-to...) now. They have no reason to access and sell your data because they have a healthy business model built around premium priced apps.

I think the reason a lot of companies don't use iCloud is because some of the various ways in which iCloud worked have not always been reliable or suited to the task at hand.

I didn't know they added this feature, that's actually really cool. Especially since their code is also open source.
FWIW, you can set up OmniFocus to sync using a WebDAV server you control.
In addition to the comment about private WebDAV (which took me 10 minutes to setup and as a side effect made the synchronisation much faster by using geographically closer server), OmniFocus can encrypt the data synced.
I use a txt file edited in Lighttable
With many ~/development folders on various containers organized by "tag" and a ~/projects folder, a top-level todo.txt, and a todo.txt in each ~/development/project_name, with vim and ssh. Just use ssh + vim + a "server" at home, basic files and directories and containers to organize work.

Documents, sheets and notes, are in nextcloud, as well as pictures and other things. Calendar is there as well. Its all in one server, with raid0 ssds, raid5 for massive storage, a few "service" VMs like for nextcloud, a few containers here and there, with easy access and overview of it all on the hypervisor. Android phone has notes which is nextcloud notes application for simpler reminders/buy-milk kind of things.

* Emacs org mode to prep for meetings, maintain TODO lists, and general notes

* Also keep Slack open when I am working. If someone needs something, they can ping me on Slack and I will be available within 5-30 minutes.

* Gmail stars for everything I can't address immediately. Everything else is dealt with immediately and inbox count is 0 for most of the day (both work and personal addresses).

* Iphone calendar (with alarms) for everything I am putting off for later or need to schedule/remember

* Iphone notes for everything I am putting off indefinitely (movies/books/games to consume, gifts to gift, songs to learn, general goals for the next few years, travel destinations, events)

* Occasionally I leave things out of place so that I remember what I'm doing next time I leave the house (tennis racket on the bike + tennis shoes and shorts/compression shorts in my bag)

That's pretty much it. I really don't like having to keep things in mind. And when I want to zone out on a run or a vacation, I can safely do so knowing whatever I need to accomplish is on one of those lists and will probably get an alarm from my calendar if its urgent. I used to love using Pocket for reading papers/articles when I don't have a reliable connection, but its completely broken now and doesn't save pages consistently or renders pages unreadable.

IMO maintaining a wiki is way too slow and (depending on your work place) only accessible across VPN, which is incredibly inconvenient. Paying someone to act as scrum master/maintain a kanban is generally a waste of money/time as well, unless you work at a huge company (10,000+ people across hundreds of teams). But I'm biased and have always had a strong preference for small teams.

> * Iphone calendar (with alarms) for everything I am putting off for later or need to schedule/remember

Any particular reason you mention "Iphone" calendar and notes, is there something special about them? Or will any old calendar (with alarms) and notepad app work and does the brand name not matter?

that's easy - I simply forget
I think it's fair to say that there isn't a single tool that can satisfy all your needs. I'm meticulously organized but I doubt I can get anything done with just a Wiki or any other tool. You also have to remember that in many cases, staying organized also means collaborating with other people who may not share the same opinion as you about various tools.

It took me a long time to find the right mix of tools that are ideal for my specific workflow. These are the tools I mostly rely on now:

- Outlook.com (with custom domain) for my personal email. I find Outlook's 'Sweep' feature a lot more feature rich than Gmail's filters. I have rules that automatically move newsletters over 3 days old to archive and then delete unnecessary emails over 10 days old automatically.

- I've grown to like Wunderlist a lot for personal tasks. It has its limitations but it gets the job done well. I don't bother with many folders etc. I just throw everything in the inbox and add due dates to it. Inbox is sorted by due date. Works great for me. Also, it integrates well with Outlook.

- Evernote is where is store anything that's remotely relevant to me. Again, I don't bother with multiple notebooks. I have 2 main notebooks - Inbox and Cabinet. All notes start in Inbox. When they are no longer needed for my day to day work, they are moved to Cabinet. All notes are meticulously organized with tags. I am contemplating having a 3rd notebook called Library (offline notebook) with all my favorite online articles tagged by topic. This might work better than a Wiki to be honest. I also use their browser extensions a LOT.

- I use aText snippets inside Evernote to log call notes, meeting notes etc. using a standard format. I have quite a few other snippets that I use in other applications as well.

- All my favorite articles are in Pocket

- All my personal files are on OneDrive. Work files are on Google Drive (it's easier to collaborate on Google because everyone I know uses Gmail / Google Apps).

- Work tasks are on Trello

- Photos automatically organized on Google Photos

- My code editor (Visual Studio Code) settings are automatically backed up on GitHub Gist using https://marketplace.visualstudio.com/items?itemName=Shan.cod...

I also use a few other tools occasionally but mostly rely on these for everything. Like most people, I wanted one tool to fit all my needs but I came to accept that it's just not possible considering the complex needs of each function. You might not think about it but, task management in itself is more than just a simple list of to-dos that requires a lot of programming to be done right. Stop trying to find a one-size fits all tool. Leverage multiple tools and integrate them together. For instance, Evernote talks to Gmail through a browser extension so you can always save an important email easily. You can also create more complex workflows using IFTTT. That would be the best way to organize yourself.

As a knowledge worker, I've found that there is no 'best' way of organising information. It depends on what it is that you're doing.

Having said that, the things I come back to are Trello (Kanban-style boards), Pinboard.in, a personal wiki, and Google calendar. I like everything web-based so that I can access things wherever I am and whatever device I choose to use.

Over and above that, I use a paper-based daily planner that I've iterated over time. I pull everything to do that day on to it, then bin it at the end of the day.

Version 3 is here: http://www.slideshare.net/dajbelshaw/dougs-daily-planner-v3 (CC licensed)

Out of curiosity, is anyone interested in some sort of app for this purpose? Like, organizing references to all your other data, thoughts, relationships. I'm not sure how it would work - just thinking about it right now.
Yesterday I've discovered Airtable, you might wanna check that out if you need something like this.
There are lots already, I think the problem most people have is that their data is spread out over several. Adding another is just like that XKCD about standards.
I'm very much interested in an app that:

- I can use daily for all my personal notes (ie. a digital commonplace book)

- is designed with an awareness of what keeps me from using such systems or mechanisms for uncovering those reasons (if I thought I could trust an app's being designed with that in mind, I think it'd be easier for me to stick with one)

- An example would be providing views that help me focus on the task at hand because looking at a giant task list is overwhelming for me

- was capable of providing me insights into myself through my data (eg. topic analysis of pages I visit, chat texts, etc. so I can see over time what I'm learning and how well those things align with my goals)

I'm also totally interested in helping design something like this because I've been putting together a design system I call mindful design and want to practice it with others.

A lot of the changes I've made to my personal information architecture have to do with avoiding getting stuck on the things I don't need to worry about yet. I've tried to make personal wikis many times, for example, but the friction is too high. I do want more of a "searchable database" for my stuff, still.

* Calendar as overarching to-do, covering errands, appointments, blocks of "do this work". Sometimes I follow it closely, other days it is largely a suggestion.

* Mindmapping to collect notes, outlines, brainstorm. Lately I've used Coggle [0] for its easy sharing. When I need to add details, the nodes may grow into Gdocs links.

* "Scratch" text file when I just need to jot work things down and I really don't care about the organization yet. If it becomes more lasting I tend to move towards the other tools.

* Trello[1] when I want to cut out more discrete tasks over a longer period and log their status.

* Riot.im[2] to talk to myself. This is something new I'm trying, which is that I can start a conversation without having someone in the room yet, by thinking "out loud". Then I can subsequently invite people in to continue it. A tiny nudge in context that distinguishes it from dumping a text document on someone.

[0] https://coggle.it

[1] https://trello.com

[2] https://riot.im

Brilliant use of Riot.im!!!
This is deeply uncool, but I use Excel.

Very fast to make a quick list, but of course awesome filtering/calculation/lookups etc.

I have a to-do.xlsx for work, and one for home. Different sheets for to-read, to-call etc.

Excel? Uncool indeed, I use LibreOffice Calc.

Kidding aside, I don't think it's such a bad choice. I've looked for open source calendar/todo solutions and they either suck (can't be shared with someone) or can't be self-hosted. So I ended up using a file sync system with a spreadsheet. Worked well for me and my girlfriend to have a shared calendar/todo list.

I use vimwiki for most things (in Dropbox), you can keep todo lists etc. in there. I'm moving towards a per-project wiki as well. I built a horrible hacky android app which hooks into any of the "sharing" menus which just appends a link into my wiki, so that's quite handy.

For email, I've really been enjoying Google Inbox, the ability to sleep items is nice, and the reminder function is quite a handy TODO list of sorts.

But mainly it's the vimwiki for me.

I have a 16GB iPad, about 12GB available. I keep all I really need within 6GB and remove the dead wood weekly. It is still a few thousand documents but, alas, I am fully in control.
...alas, I am fully in control.

Great existential touch there with the "alas"! I'm now picturing you as the Hamlet of personal task management.

For the bulk of my personal organization (file organization, note taking and todo lists):

https://github.com/galfarragem/hamster-gtd

My stack is becoming simpler as I declutter my life. On last update, I quitted Evernote and my notes are now managed on a text editor. I still use other tools for:

- collaboration (Trello, Slack, Gmail)

- finances (Spreadsheet: informal balance sheet updated every quarter).

- RSS (Feedly)

- image references (Pinterest: images with searchable descriptions. Unfortunately their search engine is bad, worse than Evernote's)

As simple as possible, no "dashboards" needed :)

-- Calendar for events e.g. Google Calendar

-- GoogleMail with its task function for emails

-- one file for yearly, quartely and weekly goals

-- a normal file structure, with a README in each respective project folder, all literature, documents contacts, work, etc

-- one hard copy lab book for conceptual work.

* white sheets of printer paper for drawings and sketching out ideas * evernote for storing everything long term. from photos of notes, typed notes, and drawing made with iPad. (i have about 3000 notes going back to 2008) * omnifocus for todos * google calendar for events/meetings * simplenote/notational velocity for anything which doesn't need to stick around but needs to be recorded quickly
http://freeplane.sourceforge.net/wiki/index.php/Main_Page Freeplane - mind maps

I've used txt files/folders, then excell/word, then markdown (easy to write & read), and wiki. Then I found freeplane, started to play with it and never got out, its incredible: use plain text or html (to format your notes), insert images, links to external files/folders, the visual mind-map representation gives a great overview of your notes and lets you organize in a foldable tree (branches, parents, childs) your notes (like in deeper layers of detail).

After the initial text/html nodes, branches and folds, I started to use "styles" to add icons and automatically format certain nodes with a background color (ex: TODO=yellow backgrnd, DONE=green backgrnd, PROBLEM=red backgrnd, ...)

Never really got into other freeplane features such as "notes" view or node-attributes. But it all went up to another level when I discovered that its possible to create http://www.freeplane.org/wiki/index.php/Scripting:_Other_lan... freeplane-scripts (in groovy or java or javascript, or any other JSR 223 language like Ruby). And using freeplane-script I started to make my own scripts, to automate repetitive tasks and improve its behaviour for my needs.

Why is freeplane better than the others methods I tried before? It organizes knowledge in a foldable tree, that gets bigger and bigger over time, and after a few years, its just easy to "find" in the notes. It also performs quite well (my maps are huge, huge, years old huge)

Have a look and decide for yourself.

https://www.youtube.com/results?search_query=freeplane

I've a Trello personal board with To-do, Doing, Done lists along with a 'To read/watch/listen'. I send everything interesting that I see to the To Read list and once I finish going through an item, if it's interesting enough to be saved for later, I send them to Pocket and tag it appropriately. I've a special tag called 'Toolkit' to save frameworks and other techniques that I cross-tag with other generic tags. I also add notes for videos, podcasts etc as comments.

Right now my Trello board is organised based on the work I need to do for a week. I haven't really experimented with changing this frequency though as it pretty much works for me now. And a lot of times, I need to sub divide my weekly tasks and for that I use checklists. When needed, I create separate reminders for these sub tasks using the Reminders app(primarily because it is super fast and syncs well across my devices).

When I need to look for something, I first use Spotlight to search for it as it can search across all of these apps. Works most of time, but I have to search individual apps for better results at times.

There was a great thread some time ago on HN about PKB (personal knowledge base). You might find there a lot of info. It prompted me to look for new solutions, I made a good progress but I'm still not happy. The best thing I use and can recommend is TiddlyWiki. I keep it in Dropbox so I can access it on mobile.
I would agree with tiddlywiki, I find it incredibly useful for building a personal knowledgebase.