Incidentally, this describes what I believe to be the great difficulty of PhD research. You have to take a topic you find interesting and read all possible related work in it, which tends to result in significant scope creep as you realize just how much there is that already does you want to do. Having exhausted your initial energy and excitement for the project, you have to force yourself the remaining 20-30% of he way to the finish line to get that work to a publishable state.
This is why so many research papers have "Towards" as the first word in the title. I.e. we didn't actually achieve our original goal, but here's a writeup of what progress we made.
I heard academic research described like this: Imagine yourself inside a balloon. The body of research in your field is represented by the current size of the balloon. Now put your fingertip on the wall of the balloon and push. That little indentation is your Ph.D.
Another analogy is a lake: The lake is the current body of research, you walk up and dump in an additional bucket of water. That's your Ph.D.
I feel one also has to have a drive for “marketing” their idea and that is 50% of the PhD. Or at least a blind enthusiasm that they are doing something significantly better than previous work.
How else does one stand up and proclaim that the fruits of years of labor and expense of a small fortune is an incremental improvement ?
While working on a project for a class, I once found a solution to a particular problem that hadn’t been really solved. Got to do a conference poster. Someone asked if it was my PhD topic (it wasn’t). I felt it was too silly to even count toward such. The problem likely had not been unsolved because very few ever cared.
(I eventually dropped out)
Instead I write software mostly with no UI that nobody ever sees. But I enjoy the work and can convince myself that someone finds it useful.
I think the author is really just getting at the fact that humans are by nature intelligent and by nature tend to think of similar ideas. So you can either unknowingly complete a project which is inevitably in some sense a replication of another project, or you can do the research first and realize it's partially a replication which is a bit disheartening. I think the solution might lie in realizing that completing a project for the sake of your own learning might be the most important factor. (This is easier said than done is when you are trying to complete novel academic research or when you are trying to make a profit off of your unique project.) But those, too, are more than forgiving to research that seem only to slightly tweak something that already exists.
> Perhaps there’s some kind of conservation law here: Any increases in programming speed will be offset by a corresponding increase in unnecessary features, rabbit holes, and diversions.
This resonates hard. LLMs enable true perfectionism, the ability to completely fulfil your vision for a project. This lets you add many features without burning out due to fatigue or boredom. However (as the author points out), most projects' original goal does not require these complementary features.
The flip side is that "fulfil your vision" requires actually having a vision before each session. Without explicit scope set upfront, the LLM defaults to filling time with adjacent improvements that feel like progress.
I've started writing a one-line success criterion before opening the editor, the way @sambaumann's parent comment hints at — "the failure-mode-resistant version of this task is X." Not because the LLM can't figure out what I want, but because if I can't compress the goal to one
sentence, I'm not actually ready to build, I'm ready to wander.
The scope creep cure pre-LLM was fatigue. Now fatigue takes much longer to kick in, so the cure has to be cognitive instead.
Over planning and scope creep are a problem, but let's not swing the pendulum to far the other way. Some of my most successful projects were projects where I planned out and worked through most of the features ahead of time through the process of modeling my data without any working software to try out. When I'm in that phase, I often don't really know what is too much. If I leave out features I think I or the users will probably want, I spend a lot of time with significant redesign of core aspects of the code. If I'm wrong, the project gets too big and we chalk it up to scope creep.
My ability to get this right is often a matter of how well I know the domain. If I don't know the domain as well I think I do, I fall into a lot of rework. If I know the domain more than I imagine then I waste my time with a baby step process when I could have run. All of this is a big judgement call, and I have "regrets" in both directions.
1. Insist on doing everything through “channels.” Never permit short-cuts to be taken in order to expedite decisions.
2. Make “speeches,” Talk as frequently as possible and at great length. Illustrate your “points” by long anecdotes and accounts of personal experiences.
3. When possible refer all matters to committees, for “further study and consideration”. Attempt to make the committees as large as possible – never less than five.
4. Bring up irrelevant issues as frequently as possible.
5. Haggle over precise wordings of communications, minutes, resolutions.
6. Refer back to matters decided upon at the last meeting and attempt to re-open the question of the advisability of that decision.
7. Advocate “caution.” Be “reasonable” and urge your fellow-conferees to be “reasonable” and avoid haste which might result in embarrassments or difficulties later on.
8. Be worried about the propriety of any decision – raise the question of whether such action as is contemplated lies within the jurisdiction of the group or whether it might conflict with the policy of some higher echelon.
Managers and Supervisors:
1. Demand written orders.
2. “Misunderstand” orders. Ask endless questions or engage in long correspondence about such orders. Quibble over them when you can.
3. Do everything possible to delay the delivery of orders. Even though parts of the order may be ready beforehand, don’t deliver it until its completely ready.
4. Don’t order new working materials until your current stocks have been virtually exhausted, so that the slightest delay in filling your order will mean a shutdown.
5. Order high-quality materials which are hard to get. If you don’t get them argue about it. Warn that inferior materials will mean inferior work.
6. In making work assignments, always sing out the unimportant jobs first. See that important jobs are assigned to inefficient workers with poor equipment.
7. Insist on perfect work in relatively unimportant products send back for refinishing those which have the least flaws. Approve other defective parts whose flaws are not visible to the naked eye.
8. Make mistakes in routing so that parts
and materials will be sent to the wrong place in
the plant.
9. When training new workers, give incomplete or misleading instructions.
10. To lower moral and with it production, be pleasant to inefficient workers; give them undeserved promotions. Discriminate against efficient workers; complain unjustly about their work.
11. Hold meetings when there is critical work to be done.
12. Multiply paperwork in plausible ways. Start duplicating files.
13. Multiply the procedures and clearances involved in issuing instructions, making payments, and so on. See that three people have to approve everything where one would do.
Definitely have found myself in a similar situation in fact most of the times option 2 happens. I too have caught myself just thinking rather than building and glad I am not the only one who repeatedly tells himself I should just build it rather than enter the rabbit hole of what is out there.
i feel a lot are missing the point here of identifying the "why" in why you want to build a project.
do you want to learn a new skill? do you want to scratch a very specific personal itch for just yourself? do you want to solve problems for others as well? do you want to build a startup/business around the idea?
all of these necessitate different approaches and strategies to research and coding. scratching an itch? maybe fully vibe coding is fine. want to learn? ditch the vibes and write by hand and ignore prior art. want to build a business? do some actual market research first and decide if this is something you actually want to pursue.
this post was a good reminder for me to identify the why as early on as possible and to be ok with just building something for myself without always having to monetize a side project which, for me, just zaps all joy from it.
Our CEO at Rec Room put this a way I really like, "Teams are always telling me they wish they did shorter projects. I've almost never heard a team say, 'we wish we delayed launch, did something more complex, polished more'"
I don't think it holds in 100% of situations but I do think if you're going to make an error one way or the other, I'd rather do something smaller and release too early than do something bigger and waste time.
Yes, why would teams of a company prefer shorter less risky projects to look better in front of the person that controls their livelihoods on whether they become homeless or die due to lack of health insurance next sprint?
Totally realistic and honest outlooks that are replicable throughout all swaths of industry really.
Yes, why would teams want to build a complex thing where they can claim tribal knowledge while building up their fiefs? Truly there is no junk incentive here, people just WANT complexity. No ulterior motives to be found!
also if youre in a large organization, this is a great way to sabotage other peoples projects while elevating your stature. Require that they go evaluate alternatives and prior art, and write a slew of analysis and decision documentation
In one of his speeches, Obama said "Better is good". I think about this a lot. It feels like better compounds over time, too. Small improvements add up. From experience, nothing new is perfect the first go round, so sitting around trying to come up with a perfect design is counterproductive because there's no such thing.
"impediment to action advances action. what stands in the way, becomes the way".
> "impediment to action advances action. what stands in the way, becomes the way".
Don’t know where this is from but it’s like saying “when you stop work in progress, progress is stopped”, I mean, sure. That’s how things go, but why even say such an obvious fact of life?
before, things that would block me were acts of an unkind universe. I'm exaggerating for effect here. These days, I look at obstacles more like an opportunity. Does this make sense?
Firstly - Greetings! It’s so rare to see a Clojure person in the wild! and secondly, I really resonated with this! it feels like we, computer programmer, typically overthink too much to begin with, and then LLMs come along and actually help us overthink even more!
Sabotage is intentional, but the problem is unintended excursions, which is endemic to any scouting.
The real problem is avoidance, when cuts are warranted and you don't want them, so you ... hide, often by working hard on something else.
The solution is to value your time. Most don't, so (self-) managers instead need to dangle other opportunities: finish this so you can do that. You can't take candy from a baby without trouble; instead, you trade for something else.
Scope creep is scary when you have the wrong pretext: to "just" implement a small feature or a project, when in reality the prerequisites to do so are enormous.
I'm _exactly_ in this situation right now with a side project.
It's in a field that I have little experience with (Information Retrieval). So there is obviously prior art that I could learn from or even integrate with.
This article motivates me further to learn things by focusing on building my own and peek into prior art as I go, when I'm stuck or need ideas.
Recently a Clojure documentary came out and the approach of Rich Hickey was seemingly the opposite: Deep research of prior art, papers, other languages over a long period of time.
However, he also mentioned that he made other languages before. So the larger story starts earlier, by making things and learning from practice.
Maybe that's also the bigger lesson: Don't overthink, start by making the thing. But later when you learned a bunch of practical lessons and maybe hit a wall or two, then you might need that deeper research to push further.
Just code using c++ (or in a language with a similar syntax complexity or a massive runtime,
java, microsoft rust, etc).
It gets even better with ISO regular feature creep: you'll find always a dev
to manage to make hard dependent on the latest "standard".
Basically, you will end up dependent on the massive complexity of a compiler
due to the syntax complexity, and the cherry on top, thanks to ISO,
you'll get feature creep creating a cycle of planned obsolescence around 5 to 10 years.
I found that setting deadlines solves most scope creep problems. Anecdotally, I am more likely to complete a project for a game jam or programming contest (which come with hard deadlines) than finishing an open-ended project.
See also "Why does the C++ standard ship every three years" (as opposed to ship when the desired features are ready):
62 comments
[ 4.2 ms ] story [ 77.5 ms ] threadI heard academic research described like this: Imagine yourself inside a balloon. The body of research in your field is represented by the current size of the balloon. Now put your fingertip on the wall of the balloon and push. That little indentation is your Ph.D.
Another analogy is a lake: The lake is the current body of research, you walk up and dump in an additional bucket of water. That's your Ph.D.
I'm sure there are many others.
How else does one stand up and proclaim that the fruits of years of labor and expense of a small fortune is an incremental improvement ?
While working on a project for a class, I once found a solution to a particular problem that hadn’t been really solved. Got to do a conference poster. Someone asked if it was my PhD topic (it wasn’t). I felt it was too silly to even count toward such. The problem likely had not been unsolved because very few ever cared.
(I eventually dropped out)
Instead I write software mostly with no UI that nobody ever sees. But I enjoy the work and can convince myself that someone finds it useful.
Maybe I lack imagination or curiosity, but it makes it difficult to come up with an idea and follow it through.
You start with a simple goal → then research → then keep expanding scope → and never ship.
The people who actually finish things do the opposite: lock scope early, ignore “better ideas”, ship v1.
Most projects don’t fail due to lack of ideas, they fail because they never converge.
This resonates hard. LLMs enable true perfectionism, the ability to completely fulfil your vision for a project. This lets you add many features without burning out due to fatigue or boredom. However (as the author points out), most projects' original goal does not require these complementary features.
My ability to get this right is often a matter of how well I know the domain. If I don't know the domain as well I think I do, I fall into a lot of rework. If I know the domain more than I imagine then I waste my time with a baby step process when I could have run. All of this is a big judgement call, and I have "regrets" in both directions.
Organizations and Conferences:
1. Insist on doing everything through “channels.” Never permit short-cuts to be taken in order to expedite decisions.
2. Make “speeches,” Talk as frequently as possible and at great length. Illustrate your “points” by long anecdotes and accounts of personal experiences.
3. When possible refer all matters to committees, for “further study and consideration”. Attempt to make the committees as large as possible – never less than five.
4. Bring up irrelevant issues as frequently as possible.
5. Haggle over precise wordings of communications, minutes, resolutions.
6. Refer back to matters decided upon at the last meeting and attempt to re-open the question of the advisability of that decision.
7. Advocate “caution.” Be “reasonable” and urge your fellow-conferees to be “reasonable” and avoid haste which might result in embarrassments or difficulties later on.
8. Be worried about the propriety of any decision – raise the question of whether such action as is contemplated lies within the jurisdiction of the group or whether it might conflict with the policy of some higher echelon.
Managers and Supervisors:
1. Demand written orders.
2. “Misunderstand” orders. Ask endless questions or engage in long correspondence about such orders. Quibble over them when you can.
3. Do everything possible to delay the delivery of orders. Even though parts of the order may be ready beforehand, don’t deliver it until its completely ready.
4. Don’t order new working materials until your current stocks have been virtually exhausted, so that the slightest delay in filling your order will mean a shutdown.
5. Order high-quality materials which are hard to get. If you don’t get them argue about it. Warn that inferior materials will mean inferior work.
6. In making work assignments, always sing out the unimportant jobs first. See that important jobs are assigned to inefficient workers with poor equipment.
7. Insist on perfect work in relatively unimportant products send back for refinishing those which have the least flaws. Approve other defective parts whose flaws are not visible to the naked eye.
8. Make mistakes in routing so that parts and materials will be sent to the wrong place in the plant.
9. When training new workers, give incomplete or misleading instructions.
10. To lower moral and with it production, be pleasant to inefficient workers; give them undeserved promotions. Discriminate against efficient workers; complain unjustly about their work.
11. Hold meetings when there is critical work to be done.
12. Multiply paperwork in plausible ways. Start duplicating files.
13. Multiply the procedures and clearances involved in issuing instructions, making payments, and so on. See that three people have to approve everything where one would do.
14. Apply all regulations to the last letter.
do you want to learn a new skill? do you want to scratch a very specific personal itch for just yourself? do you want to solve problems for others as well? do you want to build a startup/business around the idea?
all of these necessitate different approaches and strategies to research and coding. scratching an itch? maybe fully vibe coding is fine. want to learn? ditch the vibes and write by hand and ignore prior art. want to build a business? do some actual market research first and decide if this is something you actually want to pursue.
this post was a good reminder for me to identify the why as early on as possible and to be ok with just building something for myself without always having to monetize a side project which, for me, just zaps all joy from it.
I don't think it holds in 100% of situations but I do think if you're going to make an error one way or the other, I'd rather do something smaller and release too early than do something bigger and waste time.
We must run in very different circles, but IME this just strains credulity.
Totally realistic and honest outlooks that are replicable throughout all swaths of industry really.
I think calculating this sort of tradeoff is part of the skillset engineers tend to lack.
"impediment to action advances action. what stands in the way, becomes the way".
Don’t know where this is from but it’s like saying “when you stop work in progress, progress is stopped”, I mean, sure. That’s how things go, but why even say such an obvious fact of life?
before, things that would block me were acts of an unkind universe. I'm exaggerating for effect here. These days, I look at obstacles more like an opportunity. Does this make sense?
The real problem is avoidance, when cuts are warranted and you don't want them, so you ... hide, often by working hard on something else.
The solution is to value your time. Most don't, so (self-) managers instead need to dangle other opportunities: finish this so you can do that. You can't take candy from a baby without trouble; instead, you trade for something else.
It's in a field that I have little experience with (Information Retrieval). So there is obviously prior art that I could learn from or even integrate with.
This article motivates me further to learn things by focusing on building my own and peek into prior art as I go, when I'm stuck or need ideas.
Recently a Clojure documentary came out and the approach of Rich Hickey was seemingly the opposite: Deep research of prior art, papers, other languages over a long period of time.
However, he also mentioned that he made other languages before. So the larger story starts earlier, by making things and learning from practice.
Maybe that's also the bigger lesson: Don't overthink, start by making the thing. But later when you learned a bunch of practical lessons and maybe hit a wall or two, then you might need that deeper research to push further.
Basically, you will end up dependent on the massive complexity of a compiler due to the syntax complexity, and the cherry on top, thanks to ISO, you'll get feature creep creating a cycle of planned obsolescence around 5 to 10 years.
Oh, sorry, "they" called that "innovation".
See also "Why does the C++ standard ship every three years" (as opposed to ship when the desired features are ready):
https://news.ycombinator.com/item?id=20428703 (2019-07-13, 220 comments)
Prototype a minority of the time. Research a majority of the time. At some point the ratio flips as research fades out and producing increases.