I don't understand why people criticize this post. When you run a homepage or a blog, it's unavoidable to write script style code. Even if the quality is a bit low, that's the limit within a tutorial. Because if you go into actual design, things like boundaries, policies, error handling, and so on require a lot of prior knowledge. So when certain knowledge is needed, you can only post something as a simple runnable script.
For example, if I were building real software, I would design everything from policy to error logging policies and so on. But when writing a blog post, it's just simplified into a short runnable script.
I've tried most form of planning - from the basic AGENTS.md guide to keeping ./dev/ plan files, todo list tools, sqlite db with both minimal and extensive tracking, etc.
None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI paradigm.
My current scheme is basically this - in order of the task's complexity:
- Tell an agent to do something
- Tell an agent to make a plan then tell it to execute on it.
- Tell an agent to make a plan, write to a file, have a subagent review it, then execute it.
- Do the above, but instead tell the agent they're in a supervise mode and to have subagents implement as many phases and rollover with a handoff.md while they, as the supervisor agent, keeps driving the task to completion.
The latter two i have under a sigil so they're prepared prompts i can inject with a few keystrokes.
If i feel very fancy i'll tell them to update the plan with a checklist and add checkboxes, but it just doesn't pay enough to have 'init-prompt' level planning feature or tools if in the same context you already have files/read/write.
A while back I created a tool called Jobs [0] to help with this workflow. The pattern is:
1. Have a conversation with a smart model (Opus/Fable) about what you're building. Go back & forth until you've ironed out the important architectural choices (deciding what to build).
2. Ask the model to write up its plan in a Markdown doc, including a structured plan in YAML format (telling it to consult `job schema`).
3. Clear the context and tell a leaner model (Sonnet/Opus) to read the plan doc and then pick up the task via `job status`.
From there, the CLI helps the agent take the next step. I designed the `job` CLI through extensive iteration with agents, conducting user-centered design with the agents to make it as smooth and intuitive to them as possible.
When context gets full, you can pause, clear, and pick right back up. Using Jobs (or other tools like it), you can take on large, ambitious plans and keep the agents on-task the entire time.
What's the point of the scratch pad? Isn't the same data already in the context? Or does it help because contexts are lossy and bias towards the start and end?
Similar question with the to-do list. Do they actually help task completion? Is there any research on that? I think they're less helpful with more recent models, but maybe they still help with smaller ones?
The system prompt asking it to make a plan before starting work does sound helpful though. (Of course it would also be great to see numbers there :)
15 comments
[ 5.1 ms ] story [ 35.2 ms ] threadPlease stop posting.
For example, if I were building real software, I would design everything from policy to error logging policies and so on. But when writing a blog post, it's just simplified into a short runnable script.
None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI paradigm.
My current scheme is basically this - in order of the task's complexity:
- Tell an agent to do something
- Tell an agent to make a plan then tell it to execute on it.
- Tell an agent to make a plan, write to a file, have a subagent review it, then execute it.
- Do the above, but instead tell the agent they're in a supervise mode and to have subagents implement as many phases and rollover with a handoff.md while they, as the supervisor agent, keeps driving the task to completion.
The latter two i have under a sigil so they're prepared prompts i can inject with a few keystrokes.
If i feel very fancy i'll tell them to update the plan with a checklist and add checkboxes, but it just doesn't pay enough to have 'init-prompt' level planning feature or tools if in the same context you already have files/read/write.
1. Have a conversation with a smart model (Opus/Fable) about what you're building. Go back & forth until you've ironed out the important architectural choices (deciding what to build).
2. Ask the model to write up its plan in a Markdown doc, including a structured plan in YAML format (telling it to consult `job schema`).
3. Clear the context and tell a leaner model (Sonnet/Opus) to read the plan doc and then pick up the task via `job status`.
From there, the CLI helps the agent take the next step. I designed the `job` CLI through extensive iteration with agents, conducting user-centered design with the agents to make it as smooth and intuitive to them as possible.
When context gets full, you can pause, clear, and pick right back up. Using Jobs (or other tools like it), you can take on large, ambitious plans and keep the agents on-task the entire time.
This blog is on medium so I guess the migration went sideways!
Joke aside, nice series of tutorials, don't let the haters get to you. I think with the current token panic it might get handy soon
Similar question with the to-do list. Do they actually help task completion? Is there any research on that? I think they're less helpful with more recent models, but maybe they still help with smaller ones?
The system prompt asking it to make a plan before starting work does sound helpful though. (Of course it would also be great to see numbers there :)