Ask HN: How do you manage your personal software projects?

73 points by ciaoben ↗ HN
I am a solo developer who loves to work on side projects in parallel his regular job.

I am REALLY struggling to manage them, I need some system that:

- does not require more work to keep updated than the project itself - it quickly gives me the ability to understand at what point of the project I am. Often pass days or week from the last time I have been able to work on the project, and I need something that helps me having the general view of the status I left the project last time and what are the next steps. - improves my ability to track the different aspects of the project. Whatever they are different functionalities or other more broad aspects as marketing strategies....

I know are hard requirements, but I am not looking for a tool suggestion, but a method or some suggestions to find the right path.

Developing software products has a lot of facets (technical, and not!), and I am losing among them, I need someone who can show me the way :)

61 comments

[ 3.8 ms ] story [ 131 ms ] thread
I keep a journal using Google Docs. Whenever I make an entry, I have the date and a brief summary of any interesting thoughts related to the project, URL links, etc. I always keep a TODO list of what to work on next. This helps immensely when picking up the project again after a while to figure out what I was working on.
Trello works pretty well for me.

I usually create a new board with 3 columns:

Backlog - Doing - Done

You create any amount of cards in backlog and as you start to work on something you can leave comments on your status "Started working on live updates, trying to figure out if websockets are better than polling", etc.

You can easily forget to update cards so an alternative is to leave comments in your code:

```` def some_method

  User.where(email: 'aj # TODO: I WAS HERE, continue fixing the query!
end ```

By leaving this change open in git you can easily see it when you run `git diff`

For my one big side project I keep a notebook (moleskine) with thoughts and notes. I write down the current date when I make one and usually where I am.

I use Trello as well. It actually helped quite a bit launching my current endeavor, as I picked what as the MVP at the beginning. I had a "For Launch" column in addition to the ones you mentioned. It kept me focused when my brain would've had me work on other things.
I'm doing the same thing but with Asana just because it's what I use daily anyway, using their sections as the types. It's not great but like you said, laying it out in the beginning keeps the scope contained and I can list off bugs I find.
Sounds like a good idea.

The new github projects feature looks like something that would work well for this purpose, as you can mix it with milestones as well as a kanban board (Trello). GitLab also has this feature and is open-source.

I know you said you're not asking for a specific tool recommendation but it sounds like you want something like Trello (trello.com). Basically you first need to come up with the individual tasks that are needed to complete the project. At first you'll likely miss things but as you're developing you can add new tasks. For each of these tasks you'll create a card. Then on the Trello board you'll have vertical columns called lists. A simple set of lists would be:

  - Backlog: Tasks that haven't been worked on
  - In progress: Tasks you're working on
  - Complete: Tasks that are finished
You can always add more lists if your workflow requires it (e.g. adding a "In testing" list). All your tasks/cards will start in the "Backlog" list. It's also a good idea to roughly order them in priority. You can then pick the highest priority task and move it into the "In progress" list. Once you've completed the task you can move the card to the "Complete" list. Then grab the next highest priority task from the "Backlog" and repeat the process over again.

When you come back from not working on the project for a while all you have to do is look at what task(s) you have in the "In progress" list. If that information isn't enough to figure out where you left off, the task is likely to broad and should be broken down into more detailed tasks. After doing this a few times you'll get more used to creating tasks at the appropriate level of detail--it's just something you have to learn from doing.

I have a folder called Projects in my tasks app (Wunderlist), with each project listed as a to-do item, and their status listed as subtasks.

At the start of every week, I move one of these projects to my Inbox, update the subtasks to match current thoughts, work on the top few, and move project back at end of week.

It sounds like you're not really satisfied with the digital stuff.

So here's a crazy idea: you might benefit from having actual (real) folders for each project with scribbles in them.

Even though it feels like a waste of paper, for some reason it works better for me.

Maybe because I don't have to open an app for that, it's just like a second screen (without sacrificing your second screen).

This is going to sound trivial, but I've always been a big fan of writing down whatever I'm worried about on a post-it and sticking it on my monitor.

If I have too many items and the post-its fall off, I'm being too aggressive. If I have two projects, I use two colors for my post-its.

The problem with any electronic or free-form system like pencil and paper is that it is far, far too easy to type things in than it is to accomplish things. Your goal is not to create a version of your mind. Your goal is to prompt yourself enough so that you're ready to begin doing the next important thing the next time you sit down.

So set your filter to maximum and only track the bare minimum you need to pick the work back up. Trust me, once you're back in it again, the next batch of things will come to mind -- and as you work, you'll work through both the strategic and tactical items, coming up with whatever the new batch of post-its will be before you finish.

Estimate the time it will take you to complete a project, wait for a time slot like that to appear, hammer it out. Bug fixes after the initial stable release should be minimal, so should needed features.

Use your lack of time to promote good planning before a project is started as well as to prevent feature creep.

On my side, I don't manage all of them quite well. But those I have the easiest to come back to are by far those where I spent time automating things. Either setting up a dev environment with docker, having automated tests, integrating with a CI platform, having added tickets on a tracker platform, etc.

I think the more the state is explicit, the easiest it is because I'm clearly not good at documenting stuff.

Btw, I try to keep all of those things on my local server. I have JIRA, Stash and Bamboo installed, a docker registry, and a few bunch of other stuff all behind haproxy doing the SSL termination. I like using a tracker understanding code that way I can seamlessly link from/to PRs, commits and branches.

There's a few different approaches to this:

1. Set a deadline and go for it. This can take up a lot of your time and make you feel burned out. I'm assuming you want to avoid this.

2. Assume it will take longer to release than you would like, but incrementally make progress each week.

I am currently #2 after trying #1 with a completely new stack. I eventually decided that it's better to workout with my significant other a day of the week to work on the project. We decided Sunday would work best for us. If this is your case I really recommend you keep your time flexible and be willing to "swap" days or switch to 1 hr each day after work.

Now that you established a schedule, you will need to maintain two lists. A short list is what you need to work on next. A long list is what you want to accomplish and release in the big scheme of things.

With your two lists, you will prioritize. If you get hung up on something, is it really that necessary? Can you live without it until after your "release".

Keep a rough estimate going, it will give you an ok timeline and help you further prioritize.

This strategy has helped me a lot, I hope to release my side project at the end of this year. :)

In project management you balance cost, schedule, and scope.

The simplest thing is to focus on scope, and make a checklist for when the project is "ready to ship". Add things as they come up, remove them, edit them, etc.

That kind of checklist is related to the "procedural" sort of checklist which is important also.

On top of that you can add practices such as "grouping projects into milestones", "estimating the effort involved in tasks", "forming a dependency tree between the tasks", "setting delivery dates for milestones", etc.

Checklisting is the basic practice. I was on the project from hell that we rescied where a real bully of a manager got us all, developers, testers, UX, ops, on a checklist -- I think estimates are a good thing but key people on the team would not play along so we did not do them.

It was not easy but we shipped.

The best tool for this may be a paper notebook. Especially if you interleave it with something like the "pomodoro" 25 minute work units. Then spend a few minutes writing down what you did and what you expect to look at next, what unsolved problems you have, etc. before going back to the computer.

(I tend to do this at work with Notepad, but in the past I've found the medium shift to pen and paper useful in itself for changing mode of thought)

I use Microsoft OneNote for this. Searchability is fine, but if you write down the context of what you are working on, you can save minutes to hours picking up where you left off.
I used Trello for quite a while, and still use it for some things. In addition to the suggested "backlog, doing, done" columns, I have another "stopping thoughts" column, where I keep a few notes about what I was thinking when I quit working on the project for the day/week/whatever".

I've mostly switched to a self-hosted, local-to-my-dev-laptop-only jira instance, since I also use jira for regular job work. I've found that kan-ban type boards are fine for the tech side of a project, but I do like the other project management stuff that jira brings; and being the only user of it, I don't have to worry about the way I use it negatively affecting anyone else.

I use a notebook. One that I write in using a pen. When stopping for the evening/day/whatever I take 5 - 10 minutes to write in my notebook what part of the project is working, what has been changed, open questions, and a list of three things to do next. You can think of it like having a standup meeting with myself :-). When I go back to the project, I thumb back through the pages in my notebook until I find the last "log entry", read it, and them make a notation with a page number of the next open page in my notebook and an arrow to the right. Then I flip to the open page put the project name on it, and put a left arrow notation to the page where the previous update was.

By working this way my note book contains a bunch of doubly linked list of pages which are the log of a particular project, but the number of projects is not bounded. If I want to review a project from start to finish that is pretty easy to do, and the act of actually writing in my notebook affixes the information in my head more thoroughly than writing it into a computer file does.

+1 for notebooks. You can scribble notes and drawings and not worry about anything. Its simpler to manage.
I agree that pen and paper is the best way to keep notes, logs, etc., and I'd add that I think the second best way is text files. I use text files for my todo list, calendar, journal, and for other similar needs.

In my experience the simplicity of these methods dramatically lowers the cognitive barriers in motivating oneself to keep notes and logs. You'll never have to worry about migrating between platforms or TextFiles, Inc. going out of business. And you can use your own processing tools for editing, scripting, syncing and so on.

Same but onenote with a tablet and pen, because then I can access it from wherever...
With my Surface Book this became more practical. I really like the drawing experience with it. Prior to that, online solutions suffered from "hard to doodle in"
Yeah, I use a similar device and it really is an almost perfect replacement for pen and paper notes
The doubly-linked list is a really good idea! I'll have to start using it.

For no real purpose other than personal interest, I like to put a "NET" (no earlier than) date next to my page numbers in my notebook. Every time I start a new page, I write the page number and the current date (ex '#51 NET 21 Sep 2016') on the page.

As an aside, I really wish there were something that really, truly duplicated the experience of pen and paper, but were digitally indexable and searchable. There's just something really special about the tactile feedback. I also really like the way I can bounce between two arbitrary pages extremely rapidly with a physical notebook, or even look at two at once.

Plus, depending on how well you remember what, rapidly searching for a given entry is way faster if you flip to the approximate point in a physical book. That's my biggest complaint about my Kindle: random access is incredibly slow; in a physical book I've read within a year or so, I can reliably find a quote I was looking for within 30 seconds, but on my Kindle it takes forever.

While reading your comment I was reminded of this method [1] I saw awhile back of having an index in the back of a notebook and marking the edge of pages at the point of the corresponding index item. It's an interesting concept if you don't need a ton of granularity in your ability to search.

[1] http://www.highfivehq.com

That's super clever too. Man, my notebook game is getting so much better today.
I also use a set of colored posit "pointers" that I tag pages with, red for action items, blue for web site ideas, green for the current project and yellow for possibly patentable Ideas
Maybe you know about the LiveScribe SmartPen:

https://www.amazon.com/gp/product/B00524DLZ0/

This was a really nice product, until the company decided to go for the "smartphone app" experience and changed everything about it drastically. I sought out and bought the older version (the second gen) because it actually had all the features I expected.

The concept is out there, but with the following improvements you can get the experience you seek: make the pen slimmer and more natural to hold, make the special paper look more like normal paper, and have a pen dock which does all the magic of syncing and creation of digital notes etc. Bonus if it were somehow possible to replicate the pixelation of the special notebook on a carbon-paper style sheet so that you can just use whatever notebook you wanted and insert this special sheet underneath.

Also about random access: I have always wondered why the eReader software never has a mode where when you scroll on the play head, it shows the current page, plus 2 pages before and 3 after (all displayed as smaller thumbnails, sort of like Netflix). Wouldn't that be at least a little closer to the flipping experience?

Stuff like the LiveScribe is close, but not quite, what I want. I think at the moment one of the most likely candidates is actually something with AR. Plus, it would just be really stupendously cool to have a blank notebook that is "full" of AR writing.

Talk about invisible ink.

A suggestion to improve upon the user experience of navigating a notebook: http://www.highfivehq.com/#method

The doubly linked list approach might be more useful if entries aren't naturally in chronological order, or if it's more of a graph than a list :)

I also primarily use pen & paper for most things, but even with log entries (as well as simple bullet-journal structures), I struggle to effectively manage projects. I was hoping for more detail on the structures used by others.

If you're an emacs user, definitely check out org mode. Or if you've wanted an excuse to check out emacs, org mode may be a reason for you to make the move.

It's insanely powerful but not for everybody.

http://orgmode.org

I use org-mode (with vim). It's low effort and anyone can see what I plan to do and submit changes with a pull request, without needing access to something other than the repository.
I have tried a lot of things and failed at keeping them up.

The two big tips that I think have worked best for me over the years, is:

- Let your code repository do most of the talking. I don't stop a work session without committing what I have been working on, and therefore I work really hard to split any work item into small chunks. Also, given that I am sometimes interrupted in the middle of some work, it really helps to try to get the code committed after every few lines of work.

- Keep the management tooling really simple, the more complex it is the harder it will be to manage. I have evolved into keeping 2-3 types of documents (in evernote): an operations cheat sheet (how to start the code, and manually how to do things especially if I have not automated it) and 2-3 types of todo lists preferable on one page (some which I need to do in an hour, others which would be good to do, and finally some which will revolutionize the industry :-) ).

I've ended up with two different methods of similar (over?) simplicity. The fun part of side projects is that there's less communication and priorities are based on little more than our own interest.

1. A TODO text file on the project root. It's harder to get more natural that editing in vim or whatever you use. I keep different lists for features, tech debt, research, etc and slowly pluck items into the immediate todo list. What's the point of a commit? Check the line deleted from TODO

2. Google Keep is surprisingly great. Every project I work on has at least one list there so I can jot notes down where ever I am. Even offline. Some projects are entirely in Keep.

Remember why you started a project. Is it for fun? Focus on the thing that interests you most right now. Profit? Focus on proving your biggest assumption.

I'll bite, but I actually authored two tools I use to manage personal / open source projects. So while I can't fit your question exactly, I'll share what's worked well for me over the past years.

Break your project out into open source components. This gives the community a library, which makes it possible (but not guaranteed) to share responsibility to it.

It also keeps things fresh, you can keep the core of your personal project closed, so there is less moving parts to track, and hopefully less dead code on it. I’ve many “personal projects” end up getting mothballed amounting to time wasted due to me just chucking all my code in one repo and not breaking out components / pulling in the right libs when I could.

I've participated on a lot of open source projects over the years, so I’ve developed a workflow to organize the minutiae. There are times where I won’t re-open a project for a few months, so I need to be able to “snap back into” a familiar environment.

- tmuxp [1], a session manager for tmux. It allows using a .tmuxp.yaml file [2] to load a project workspace in combination with a bootstrap file [3] to automatically launch a tmux environment with tests, documentation, etc. I mimic this workflow in all my other projects.

It has worked for me across OS X, Linux, FreeBSD. On local and ssh machines. No IDE required.

Passively, another thing I do is document very well. If its python, I keep docstrings, autodoc for API generation and sphinx everywhere. Even personal projects I do at work have sphinx docs for other developers to peruse. I’m consciously doing everything possible to be helpful to other programmers on the project. Give them API docs, examples, as much as you can.

- vcspull [4] is a tool I use to declaratively manage projects in a yaml file [5]. When I want to update I just type ‘vcspull project1 project2 project3’. What’s nice is being able to pattern match projects I have, ‘vcspull django-\*’ to grab django extensions.

On projects I patch, I add my fork as a remote to keep it in sync.

So the above are my taste. YMMV. Here are some other things:

For C/C++ projects / libs, stuff like CMake [6] for build systems is great. Gets me a build that finds libs on OS X, FreeBSD and Linux. I’ve ported windows only projects in SDL2 to work across all 3 easily thanks to it.

There are other mindset things, try to strike a balance between being opinionated in your own flow (as I am), because you’re going to have another person who swears to Eclipse, then another one to Atom, you can’t make everyone happy. Invest in things universally helpful (api documentation, tests, linting), they end up helping you in the end no matter where your project ends up: Even if you wrote stuff, if you don’t document, you’ll be at a loss when you wrote stuff a year ago “Did I really write that”? With other developers, they can't read your mind, they keep into the grind faster when you explained what your project does and what your code quality / expectations are.

[1] https://github.com/tony/tmuxp [2] https://github.com/tony/tmuxp/blob/master/.tmuxp.yaml [3] https://github.com/tony/tmuxp/blob/master/bootstrap_env.py [4] https://github.com/tony/vcspull [5] https://github.com/tony/.dot-config/blob/master/.vcspull.yam...

Personally I use TDD (or more precisely BDD) for this purpose. If I leave the project overnight, or for a week or a month, I can come back to it, run the tests, and the failing and/or pending tests tell me what I was doing when I last worked on it.

So I always leave an in-process development project with failing tests. If they're all green, I don't know what to do next!

This method also works for me. Usually the failing test left behind is something in the same space of where work needs to be done, but I like to keep the failing code trivial so that I can ramp up. 'Works great. A TODO file can also be used to accomplish this if TDD isn't your thing.
I think a TODO is preferable even if you practice TDD.

Failing tests used as a "start here" are to-dos in this instance and are indistinguishable from legitimate currently-implemented feature tests if you've been away long enough.

I don't think it's good to assume it's okay to delete failing tests when returning to a project with new goals or when a bug needs fixing. For me, it's important to return to a project, run tests and have them all come back green. Then if I break something, I know it broke and isn't just an artifact from my pseudo to-dos.

Along those lines... I always try to run tests, check code coverage (including via mutation testing), and commit changes before calling it a day.

Typically the error messages that I include for my "this is where I left off" or "still to do" tests are like "page not yet implemented" or "haven't figured out the bug". No definitely don't delete failing tests! Make them pass.
I use bitbucket right now as my version control. I was using trello to help me keep track of what I want to do, but I have found a simple TODO.txt file in my repository to be much simpler.

Make sure you commit frequently, use tags, and find a good way to handle dependencies with the language you are using.

I use a Makefile to build all aspects of my side project, and deployment is very straight forward on a vps I rent.

Usually before I start any project at all, I'll write a couple pages summary in a Google doc. I'll consider implementation, but I won't try to plan out the entire implementation since it always changes.

I keep a separate Google doc(or a Trello or Kanbanflow board) with a list of specific pending tasks. This is designed to be easy to grab tasks off of without having to think much.

If it's been so long since I last did anything that I don't understand any of the tasks, I'll take 30 minutes to update the summary and task documents.

alias demo='vim "$(git rev-parse --show-toplevel)"/demo.txt'

I have a bunch of these aliases for files I use for specific purposes. "demo.txt" holds a description of my next waypoint- I call it demo because it has to be something I can show. There are aliases for bugs, howto, etc.

I don't have to use an external tool, as I'm working on some code I can just type "bug" at the command line and enter some text. Not super high tech but easy enough that I actually use it.

I recommend putting a TODO file in your projects' root folder.

You will use this file to take a _snapshot_ of your current status. Be as detailed as wanted, you need this to recover the context you were in before walking away from it.

Here's what I want in my ideal personal code environment.

1. Literate programming

I want to be able to write methods/functions in something like a Jupyter notebook, with text and section headings. This is a great way to both comment the code and to organize to-do lists. I want the text in these documents to have weak ties to the actual code (e.g. have method renaming apply to the text as well.) I should be able to toggle between "plain code" and "Jupyter style" views.

2. Ability to share modules across projects

Did I write some code to make state machines for project A? I should not have to copy paste it to get state machines into project B, I should be able to share the code between them. Something like Visual Studio's shared projects, but less wonky.

3. Audio recording

I suck at writing notes to myself, especially in exploratory code. However, I sometimes find myself thinking out loud. Why not allow recording of the thinking out loud, so I can hear what I was thinking later.

I use pivotal tracker to record tasks and then prioritise them and work through them.

It's good in that, if I don't work on it for a month or two then nothing changes.

When I start working on something... everything changes and the estimates come forward and it auto-generates the idea of a sprint, spacing work out over coming weeks based on your velocity.

It's nice.

I agree with the predominant two suggestions here: to use Trello, and to do a little more planning up front.

But it also looks like you are having some issues with getting back into the zone quickly, without too much overhead. A trick that writers use is to apparently write an incomplete line at the end of the draft. You can try something similar - depending on your project, this could be something as simple a quick note on something you can do in exactly 5 minutes the next time you set down with some detailed steps if possible (for additional context).

Its just a suggestion, which is unfortunately not backed by personal experience (I have faced other challenges with my side projects but not the one you mention).

Trello and writing a line are great; I have one extra comment about the line.

Instead of just "not finishing" something, I will do a minor brain dump of what I'm trying to accomplish into a comment or a Trello item in the details field.

Usually when I'm stopping it's because there's a thing that isn't easy to finish, and I've done a lot of thinking about how to finish, but don't have the time to do it. So I write down all of my thoughts on the topic -- sometimes it can run to several paragraphs and sometimes a table or two.

But the next time I pick it back up, I can totally get back into the right mental space quickly. The best part? After a break, even if I have a plan outlined in the comment, I often can immediately see a better solution than the one I was working toward.

On a related point I've been keeping a journal with similar notes in it; I've found this to be extremely helpful for thinking through problems. You've heard of explaining your problem to a rubber duck? Having to not only put the problem into words, but writing those words down, is a great way to get the brain churning -- and a great way to leave a map for yourself later to get back into a project.

I can second this. It's like rubber duck debugging but "past you" is explaining it to "future you" who is in a different mindset.

Most of my biggest breakthroughs have happened because of something like this.

And if you aren't afraid to check these brain dumps into your source control in some kind of notes.md, it can also give you a bit more context on what problems you were facing at that time in the repo's life, and perhaps why you thought the solution at the time was a good one.