This is exactly what I thought: Pretty much every great inventor or researcher has that one big problem, or one big question that drives them ... but I think it is a good point to say that one should not be too disappointed about not necessarily making rapid progress on "the forever project" all the time. Thanks for making me feel a bit better about seemingly (and for the duration of typing this actually) procrastinating from tackling the underlying challenges behind my PhD topic ;) ...
This is unfortunate; it would be nice if dissertations were the beginning of a "forever" project, or perhaps the continuation of one. Sadly, most of the dissertations I have seen have been a bunch of quick projects stapled together with one overarching theme, with a minority being answers to big questions (like Craig Gentry's thesis on FHE). I can think of only one, in fact, that was the beginning of someone's "forever" project, and that is Darse Billing's dissertation on computer poker players (he seems to be spending year after year working on poker AI).
My experience in grad school has been this: we are discouraged from having "forever" projects, and encouraged to have "lots of little projects that you finish and forget about" dissertations.
I very much see dissertations as the beginning of forever projects; but maybe that's because my advisor's dissertation was†, and his own advisor (Ishii, cf my first post) saw it that way as well.
Which is what led me to quit my PhD (only leaving grad school with a measly master's :) )- I felt that I was too young and inexperienced to embark on such a journey. I fully intend on going back to grad school in a couple years' time, and treat my PhD as such then.
One of the side projects I have to embark on is writing a companion to TAOCP that shows how to implement the algorithms in a functional language, probably Scheme or some subset of Common Lisp. The main complaint I have about TAOCP is Knuth's insistence on using assembly language to illustrate the algorithms -- otherwise, they give what is probably the most in-depth description of the motivation and analysis of algorithms I have seen (much more detailed than CLRS, though CLRS is more complete).
Also, it is worth pointing out that there is no "complete TAOCP" yet. It remains a work-in-progress; I am looking forward to Volume 4B (though I have yet to actually finish any of the current volumes).
Hm. I actually have four things that qualify as a "Forever Project", and I've sunk a decent amount of time and money and energy into each one. Results... nothing worth noticing.
But I like the idea of a Forever Project: just a thing that's been in your head since forever and that you can't stop thinking about. Mine have gone through many different permutations over the last 15 years, including attempts to start a company.
Forever Projects are like stars, we set our course by them but never reach them.
I am now creating my Forever Company - where I can work in a way compatible with living a life and still make a difference. Some people will join my journey some will leave as I change course. I cannot imagine finishing my Forever Company either. But I know the next steps.
I don't understand this idea of a Forever Project. A project/task is either important enough to do or it's a distraction in your life. Now if you're working towards that project because you don't have the resources, knowledge, people, at this point in time, you're making progress and that's a different story.
Take me as an example. My Forever Project is Physics. I have a degree in both CS and Physics. I made the decision years ago to work in the industry as a programmer, because it's more fun and pays better than being an academic, especially where I live. I'm not officially involved with my old University, but I still take part in their mailing lists, think about Physics in my free time, I even submit simple papers to journals [+]. I don't have delusions about being a good physicists or ever doing anything of any importance, but Physics still fascinates me, I like to ponder its problems and sometimes marvel at its beauty.
Sometimes the goal of a personal side-project is not to produce something meaningful, but just to learn and have fun while doing it.
If I take myself as an example: I have this git repository checked out as 'coding' on all my machines, which has about 15 different folders containing projects ranging from simple prototyping experiments to fully-finished projects. Only a single folder contains a project that was ever released publicly (an iOS port of a classic MS-DOS game). Everything else is half-way finished at best.
Does that mean writing all that unfinished code was merely 'a distraction'? I don't think so. All of this started out as just a way to learn about something I knew nothing about before, and most of the time I stopped working on things the moment I was satisfied with what I learned. There's an arithmetic coder in there, a marching squares implementation, an MPEG-2 decoder that fully works but is dog-slow and doesn't process the color channels, a framework for 2D pattern recognition, a component-based Python web framework, a web application that was intended as a tracker for poker scores, and some other things I can't remember.
If I had set out to complete all of these projects, I would probably have finished 1 or 2 and still know nothing about the theory that got me interested in the other 13 or so.
Can you imagine how weird I felt when I finished my "forever" project?
Ever since school I wanted to write a simulation tool similar to one I've used in school, which basically lets you write differential equations in graphical form, and integrates them for you. And back when I was in school, I didn't know a thing about differential equations, and it all looked like amazing magic to me.
And while studying physics, I learned about DGLs and Runge-Kutta integration, and then I "just" needed to manage all the data (do a topological sort on the nodes to evaluate all the formulas in correct order), stuff it into the Runge-Kutta solver, and graph the output. Yay.
I never got around to writing the graphical front end, but the all interesting (to me) parts of the program are there. Oh, and it's written in Perl 6, the programming language I help to develop in my free time. If you're interested, check out http://perlgeek.de/blog-en/perl-6/physical-modelling.html
These days Perl 6 is my main "forever"-project, and it's coming along nicely.
Modeling dynamic systems is kind of where I'm moving towards with my project at the moment. It's a visual programming language where you create "non-casual" models instead of functions. My current lead is to use a modelica-like library, Hydra, as the back-end.
Nice! I've always been fascinated by tools and techniques that reduce the burden of solving complicated PDEs. After deriving, by hand, the finite difference matrix for a two-dimensional 2nd order PDE with varied boundary conditions I thought "there has to be a better way!"
In a similar manner to you I wrote an internal DSL in Python that lets you specify your system, for example that pesky 2D 2nd order PDE
and appropriate boundary conditions in Python. My pdegen tool then works on the resultant AST to generate the primary finite difference matrix. Given some initial conditions this matrix can then be used with any linear algebra library to time-step the system.
You've given me motivation to clean up my code and release it :)
Yes, I think if you are really obsessed with something for over 10 years, it's time to make it your life's work and not a side thing. But maybe the point is that you don't want that to happen as you lose your forever project (if it gets done) or fall out of love with it.
No, but I don't think that's necessarily the point of a forever project nor of life in general. It's not a goal of mine nor of none of my friends who have one; most of the projects from people I know cannot even make money even if (partly) completed.
Edit: If you mean, you need money to live; you of course need some money, but not as much as most people think. Which makes it a question of priorities mostly. There are plenty of fantastic places where $600-700/mo will get you far, even without losing that much from your 'previous life'. You have to sacrifice things obviously, but not that much (depending on what you are used to).
My forever project is a cooking robot. I've been thinking about how I'd approach it for years, but never had the chance to make what I thought would be a worthwhile start.
Thanks! The blog is on the todo list.. I'm preparing some presentations for next month so that might be the kickoff. For now I've just put up a FB page for a first device: https://www.facebook.com/vaporware.robot
That is awesome. That's the ultimate DRY (don't repeat yourself) thing humanity needs. It will free us from all the work required to raise the quality of our food, making eating well as easy as fast food.
My personal touchstone for the beginning of the true Robot Age is a fully-automated McDonald's or equivalent being seriously deployed not as a tech demo, but because it is the best solution. Supplies in one end, someone carting the waste out the other, and everything else, including cleaning, should be done by the robot itself. (Though perhaps I'll bend on the cleaning. That does add another very significant level of complexity.)
Robots will be cheaper eventually. And very easily reprogrammed.
For me the goal is not cheap food but quality food.
And the most expensive part of going out for many people? Driving there, if you consider the cost of their time.
Ultimately, we will get robotic vehicles to do that part too. We will see some very small vehicles once robotic vehicles are commonplace, perhaps delivering only a single meal.
And we'll also see robotic cooks that can cook anything, so they may become so common you won't need to drive to them.
I would love to have a cooking robot because I don't like to spend time cooking. I worked on a related project, an automatic bartender: http://youtu.be/iqvUJ6vvzxk as a group project in one of my classes.
funfact: in the expression "reach exceeds grasp", your reach is what your fingertips can brush, your grasp is what they can wrap around.
The common-sense way of handling impossible projects is to break them down into doable sub-projects: components, layers, aspects etc. Then you can get satisfaction out of completing each of them. A psychological problem I have is accepting these as goals in themselves - if it doesn't do all of the project, it feels like it doesn't count.
However, this is silly of me, because it is still progress. A journey of a thousand miles consists of steps - not just the first one, all of them. And even for a commercial project, an inadequate beginning (that still does something) is beneficial: people love the sense of progress more than everything already done, your updates give you publicity, and give customers a reason to upgrade. Even with regard to competitors, it's good to have room to improve, because when they've copied you, you've moved ahead. If you're already perfect, once they copy you, you've nowhere to go. You're a sitting duck (wrt engineering competitive advantage).
Of course, this article has a more joyful attitude towards such projects. Maybe I should try it.
In the best case you can break it down in reusable parts. I took some of the commandline tools I created for my forever project, and turned them into a stand alone application.
my forever project, of course, is to build the metaverse. I loved this article because i can reassure myself i am not crazy after all ... or at least that I am in good company :)
Here's mine: http://www.pubsoccermanager.com/ It has gotten really slow in the last year, but I am planning to do something around it full time. Most obviously it would be a game, but I also have a couple of other ideas of how to monetize.
I actually thought this was about those projects you make for a client that will never end. Clients that keep coming back with more and more changes. Usually accompanied by a Project Manger that's afraid of the client. I ocasionally have nightmares of pressure cookers after working on a terrible website for more than a bloody year.
Interesting perspective on long-term projects. I too am working on a service I consider my forever project, but one which may actually be launchable quite soon. I can definitely relate to the dreamy state the author refers to, as I too dream of a constant barrage of features I want to build into the service. This can quickly get out of control though and become endless as a developer delves deeper, but maybe that's why it's a forever project. You want this service to exist so bad for yourself, it doesn't matter if you're the only user. It's sort of like a nagging child. It's a part of you, no matter how many diapers you got to change.
I think that part of what the author is trying to convey about such projects, rather than they being de facto 'forever projects' is that such projects are worth it by the journey alone. Even if you don't ever reach the end of it - the inspiration and learnings it gives you is enough to motivate continue working on it.
I had different forever projects at different points in time. I often come back to a planet renderer based on a certain algorithm (P-BDAM), but it lost its lure on me in recent times.
The older I grow, the more I know the way I work, my strengths and weaknesses.
I recently decided to create a behavior- / attribute-based framework for MonoGame, with functional reactive programming thrown in, and to my surprise, I currently make quick and steady progress.
Interesting rgd. the functional reactive programming. I am obsessed with this, in a networking/mobile device context instead of a gaming context - not that it should matter. And applying this to real problems beyond the toy problems that you find in tutorials on the web turns out to be real hard - this may end up becoming my Forever Project yet. Email me at hank[at]ml1.net if you want to discuss this.
My forever project is a novel. It's a great way to fall asleep. It's a project fun enough to focus on and ignore everything else, but completely stress-free so it's very relaxing.
I don't actually write my novel done or ever plan on doing so. I once tried during national novel writing month (nanowrimo), but that didn't work at all. Much better to keep it as a nice thing to think about as I'm falling asleep.
Mine is too. I'm actually on my second one. I started back in the late 1980's and spent some fifteen years planning and plotting the first one before I put that outline in a drawer and began planning and plotting the second one. I'm hopeful that someday all that writing around will morph into a real novel that other people want to read but even if not the process has forever changed me.
Release it now, it's no use just collecting dust on your hard drive. You can always improve it in the future—think of it being in constant development.
Normalizing source code, storing it as annotated ASTs. Designing a structure editor that is actually usable for real code. Applying precise version control to those ASTs' nodes.
Some features:
- Optional GC with per-type granularity
- Trying to wedge uniqueness[1] into the type system too
- Can compile statically, extremely convenient C FFI
- Codifying the environment/context that a function wants/needs. E.g. a function "void render3DScene(void)" might spec its env requirements as {OpenGLContext, NonBlocking, SceneGraphState}. Similar to state monads, but intuitive, composable, and not anal.
Well, there isn't really a body of code to version control yet, as the AST forms still aren't quite finished, and so all the ASTs will have to be generated from text code (yes, Python) until the language is properly bootstrapped.
So the versioning is still vapor. Thing is, once you've done the structures correctly (i.e. copied git) the versioning just falls right out. The whole project really started off as a logical extrapolation of git's data structures.
I don't have a dev log at all, sorry. It would be good to get my insights down on paper.
Versioning AST's has been on my mind for some time as well, I got interested it as a way to store source-code in a coding-style-neutral way (check it in & out with whatever coding style you're used to), and to be able to categorize check-ins based on the type of changes to the code (only comments, moved code, variable renames, etc).
Probably you already know about it, but have a look at papers aboht the tree edit/tree distance problem:
I´m about dreaming such a language for years. But I already have a forever project so I never really did anything about it :/ Got anything online? I´d love to watch this project.
I liked this article. I for one have been overly conscious on ends rather than means, and I forget that it was the means that got me into software development in the first place. I used to have so much fun! It never felt like work (it does now).
I haven't enjoyed it in a long while, this article makes me wonder if I should start one of my idea and develop it with no clear saleable end in mind. The thought of which takes a lot of pressure off what I got done this week, this month, yesterday.
Haha this is so true. I used my forever project (indeed, a obscure fantasy football game =) ) over the past year as a narrative to pick up the basics of programming. Although the game is still not live, I learned so much building it.
I'd add one more thing to this article: don't throw these projects away, even if they seem failed/doomed/incomplete. Store them somewhere. Back them up. Whatever. You might need them.
Back when I was a teenager, I wanted to write an art package for an obscure computer (Acorn Archimedes). I wrote the first version in BASIC, then decided to re-do it in assembler. The assembler I first used couldn't cope with the size of the project in the end, so I wrote my own assembler; at that point I discovered C, so wrote it in that. Unfortunately the C compiler I used was so full of bugs I decided to port LCC to produce ARM assembler, but then I needed a linker, so I wrote that as well. Got it all working lovely to the point where it could compile itself.
At university I figured Intel PCs were more useful than my obscure computer so... I forgot about it. Lost it.
And to this day, I can't help but wish I still had it... I had written, and lost, a complete C development system, faster than any of the other compilers/linkers I tried, small footprint (it had to fit in a machine with 4Mb RAM), and ARM chips are now everywhere. Damnit!
"... art package for an obscure computer -> wrote my own assembler... port LCC... linker..."
Sir, you have one very well-shaved yak there [1]. Excelsior!
In other news, I sometimes see people bemoaning losing the "easy programming" that the era putatively had. I sometimes wonder if time has simply faded the scars of a world in which that's the sort of thing that might realistically happen. The era made some moderately difficult things easy (people mostly seem to miss the ability to slam pixels on the screen), and everything else really, really freaking hard, including many things we'd call easy today.
130 comments
[ 4.4 ms ] story [ 211 ms ] threadThe Adams brothers, Dwarf Fortress: http://www.nytimes.com/2011/07/24/magazine/the-brilliance-of...
Elon Musk, Mars colony: http://www.wired.com/wiredscience/2012/11/elon-musk-mars-col...
Donald Knuth, The Art of Computer Programming: http://www-cs-faculty.stanford.edu/~uno/taocp.html
Hiroshi Ishii, Tangible Bits: http://web.media.mit.edu/~ishii/
(this is off the top of my head- I wish the list were longer, but it is late :) )
My experience in grad school has been this: we are discouraged from having "forever" projects, and encouraged to have "lots of little projects that you finish and forget about" dissertations.
Which is what led me to quit my PhD (only leaving grad school with a measly master's :) )- I felt that I was too young and inexperienced to embark on such a journey. I fully intend on going back to grad school in a couple years' time, and treat my PhD as such then.
†: http://dspace.mit.edu/handle/1721.1/29264
Also, it is worth pointing out that there is no "complete TAOCP" yet. It remains a work-in-progress; I am looking forward to Volume 4B (though I have yet to actually finish any of the current volumes).
But I like the idea of a Forever Project: just a thing that's been in your head since forever and that you can't stop thinking about. Mine have gone through many different permutations over the last 15 years, including attempts to start a company.
I am now creating my Forever Company - where I can work in a way compatible with living a life and still make a difference. Some people will join my journey some will leave as I change course. I cannot imagine finishing my Forever Company either. But I know the next steps.
Thank you OP - some good inspiration there
In a way the forever company is you, 'Me Inc' as it were. Hopefully forever building and growing.
I don't understand this idea of a Forever Project. A project/task is either important enough to do or it's a distraction in your life. Now if you're working towards that project because you don't have the resources, knowledge, people, at this point in time, you're making progress and that's a different story.
[+] http://arxiv.org/abs/1208.4990
Edit: Btw. you are right in saying that these things are a "distraction in life", but I think it's a good distraction, like a hobby.
Physics for you is an interest and a hobby. Those are great to keep you informed.
I feel sorry for you if you don't have space in your life to enjoy some unimportant things just for the sake of it.
Since I do not have to reach a target I can try solving old problems in new ways and see how it works out in the long run.
My forever project spawned some nice sub modules which I use in productiv code.
In some branches of my forever project I tryed things generally known to be bad style to see some of the problems myself.
If I take myself as an example: I have this git repository checked out as 'coding' on all my machines, which has about 15 different folders containing projects ranging from simple prototyping experiments to fully-finished projects. Only a single folder contains a project that was ever released publicly (an iOS port of a classic MS-DOS game). Everything else is half-way finished at best.
Does that mean writing all that unfinished code was merely 'a distraction'? I don't think so. All of this started out as just a way to learn about something I knew nothing about before, and most of the time I stopped working on things the moment I was satisfied with what I learned. There's an arithmetic coder in there, a marching squares implementation, an MPEG-2 decoder that fully works but is dog-slow and doesn't process the color channels, a framework for 2D pattern recognition, a component-based Python web framework, a web application that was intended as a tracker for poker scores, and some other things I can't remember.
If I had set out to complete all of these projects, I would probably have finished 1 or 2 and still know nothing about the theory that got me interested in the other 13 or so.
Ever since school I wanted to write a simulation tool similar to one I've used in school, which basically lets you write differential equations in graphical form, and integrates them for you. And back when I was in school, I didn't know a thing about differential equations, and it all looked like amazing magic to me.
And while studying physics, I learned about DGLs and Runge-Kutta integration, and then I "just" needed to manage all the data (do a topological sort on the nodes to evaluate all the formulas in correct order), stuff it into the Runge-Kutta solver, and graph the output. Yay.
I never got around to writing the graphical front end, but the all interesting (to me) parts of the program are there. Oh, and it's written in Perl 6, the programming language I help to develop in my free time. If you're interested, check out http://perlgeek.de/blog-en/perl-6/physical-modelling.html
These days Perl 6 is my main "forever"-project, and it's coming along nicely.
A somewhat outdated and presentation of the idea (it's a bit buzz-wordy): https://patch-tag.com/r/worldsayshi/nodespace-staging/wiki/
It is very much a project of taking on more than I currently grasp. But I have already learned a lot by working on and thinking about it. :)
In a similar manner to you I wrote an internal DSL in Python that lets you specify your system, for example that pesky 2D 2nd order PDE
and appropriate boundary conditions in Python. My pdegen tool then works on the resultant AST to generate the primary finite difference matrix. Given some initial conditions this matrix can then be used with any linear algebra library to time-step the system.You've given me motivation to clean up my code and release it :)
* for various definitions of the word released
(Great book! http://en.wikipedia.org/wiki/The_Forever_War )
http://statspotting.com/2013/01/that-one-idea-that-does-not-...
Edit: If you mean, you need money to live; you of course need some money, but not as much as most people think. Which makes it a question of priorities mostly. There are plenty of fantastic places where $600-700/mo will get you far, even without losing that much from your 'previous life'. You have to sacrifice things obviously, but not that much (depending on what you are used to).
http://www.youtube.com/watch?v=0sHCQWjTrJ8
He clearly articulates the emotional weight of these forever projects though.
The stars aligned this year, and I'm actually able to work on it now. This is the current temperature of my stove: http://dave-stuff.s3-website-us-east-1.amazonaws.com/
For me the goal is not cheap food but quality food.
And the most expensive part of going out for many people? Driving there, if you consider the cost of their time.
Ultimately, we will get robotic vehicles to do that part too. We will see some very small vehicles once robotic vehicles are commonplace, perhaps delivering only a single meal.
And we'll also see robotic cooks that can cook anything, so they may become so common you won't need to drive to them.
The common-sense way of handling impossible projects is to break them down into doable sub-projects: components, layers, aspects etc. Then you can get satisfaction out of completing each of them. A psychological problem I have is accepting these as goals in themselves - if it doesn't do all of the project, it feels like it doesn't count.
However, this is silly of me, because it is still progress. A journey of a thousand miles consists of steps - not just the first one, all of them. And even for a commercial project, an inadequate beginning (that still does something) is beneficial: people love the sense of progress more than everything already done, your updates give you publicity, and give customers a reason to upgrade. Even with regard to competitors, it's good to have room to improve, because when they've copied you, you've moved ahead. If you're already perfect, once they copy you, you've nowhere to go. You're a sitting duck (wrt engineering competitive advantage).
Of course, this article has a more joyful attitude towards such projects. Maybe I should try it.
And big globs of functionality keep getting grafted on until the point where you're terrified to touch anything in case it all falls apart.
The older I grow, the more I know the way I work, my strengths and weaknesses.
I recently decided to create a behavior- / attribute-based framework for MonoGame, with functional reactive programming thrown in, and to my surprise, I currently make quick and steady progress.
http://khitchdee.forumatic.com/viewtopic.php?f=2&t=33
I don't actually write my novel done or ever plan on doing so. I once tried during national novel writing month (nanowrimo), but that didn't work at all. Much better to keep it as a nice thing to think about as I'm falling asleep.
I intend to release it someday hopefully soon, but it will never truly be finished.
"Real artists ship."
But until I write the editor for it, there's really nothing to look at. Editor is coming Real Soon Now.
Some features:
- Optional GC with per-type granularity
- Trying to wedge uniqueness[1] into the type system too
- Can compile statically, extremely convenient C FFI
- Codifying the environment/context that a function wants/needs. E.g. a function "void render3DScene(void)" might spec its env requirements as {OpenGLContext, NonBlocking, SceneGraphState}. Similar to state monads, but intuitive, composable, and not anal.
[1] http://en.wikipedia.org/wiki/Uniqueness_type
I've been quite fascinated by ASTs (and verification thereof) ever since poking around Python's approach to it.
So the versioning is still vapor. Thing is, once you've done the structures correctly (i.e. copied git) the versioning just falls right out. The whole project really started off as a logical extrapolation of git's data structures.
I don't have a dev log at all, sorry. It would be good to get my insights down on paper.
Probably you already know about it, but have a look at papers aboht the tree edit/tree distance problem:
http://scholar.google.nl/scholar?q=tree+edit+problem&hl=...
It's exactly what you would need to store versioned trees such as AST's.
I haven't enjoyed it in a long while, this article makes me wonder if I should start one of my idea and develop it with no clear saleable end in mind. The thought of which takes a lot of pressure off what I got done this week, this month, yesterday.
Great inspiring read. Thanks!
Back when I was a teenager, I wanted to write an art package for an obscure computer (Acorn Archimedes). I wrote the first version in BASIC, then decided to re-do it in assembler. The assembler I first used couldn't cope with the size of the project in the end, so I wrote my own assembler; at that point I discovered C, so wrote it in that. Unfortunately the C compiler I used was so full of bugs I decided to port LCC to produce ARM assembler, but then I needed a linker, so I wrote that as well. Got it all working lovely to the point where it could compile itself.
At university I figured Intel PCs were more useful than my obscure computer so... I forgot about it. Lost it.
And to this day, I can't help but wish I still had it... I had written, and lost, a complete C development system, faster than any of the other compilers/linkers I tried, small footprint (it had to fit in a machine with 4Mb RAM), and ARM chips are now everywhere. Damnit!
So please... save your work.
Sir, you have one very well-shaved yak there [1]. Excelsior!
In other news, I sometimes see people bemoaning losing the "easy programming" that the era putatively had. I sometimes wonder if time has simply faded the scars of a world in which that's the sort of thing that might realistically happen. The era made some moderately difficult things easy (people mostly seem to miss the ability to slam pixels on the screen), and everything else really, really freaking hard, including many things we'd call easy today.
[1]: http://www.catb.org/jargon/html/Y/yak-shaving.html