I wouldn't be surprised if agents start getting managed by a distributed agentic system - think about it. Right now you get codex/claude/etc... and it's system prompt and various other internally managed prompts are locked down to the version you downloaded. What if a distributed system ran experimental prompts and monitored the success rate (what code makes it into a commit) and provides feedback to the agent manager. That could help automatically fine tune it's own prompts.
Edit: to say more about my opinions, "agentic loop" could mean a few things -- it could mean the thing you say, or it could mean calling multiple individual agents in a loop ... whereas "agentic harness" evokes a sort of interface between the LLM and the digital outside world which mediates how the LLM embodies itself in that world. That latter thing is exactly what you're describing, as far as I can tell.
I recently built my own coding agent, due to dissatisfaction with the ones that are out there (though the Claude Code UI is very nice). It works as suggested in the article. It starts a custom Docker container and asks the model, GPT-5 in this case, to send shell scripts down the wire which are then run in the container. The container is augmented with some extra CLI tools to make the agent's life easier.
My agent has a few other tricks up its sleeve and it's very new, so I'm still experimenting with lots of different ideas, but there are a few things I noticed.
One is that GPT-5 is extremely willing to speculate. This is partly because of how I prompt it, but it's willing to write scripts that try five or six things at once in a single script, including things like reading files that might not exist. This level of speculative execution speeds things up dramatically especially as GPT-5 is otherwise a very slow model that likes to think about things a lot.
Another is that you can give it very complex "missions" and it will drive things to completion using tactics that I've not seen from other agents. For example, if it needs to check something that's buried in a library dependency, it'll just clone the upstream repository into its home directory and explore that to find what it needs before going back to working on the user's project.
None of this triggers any user interaction due to running in the container. In fact, no user interaction is possible. You set it going and do something else until it finishes. The model is very much to queue up "missions" that can then run in parallel and you merge them together at the end. The agent also has a mode where it takes the mission, writes a spec, reviews the spec, updates the spec given the review, codes, reviews the code, etc.
Even though it's early days I've set this agent missions that it spent 20 minutes of continuous uninterrupted inferencing time on, and succeeded excellently. I think this UI paradigm is the way to go. You can't scale up AI assisted coding if you're constantly needing to interact with the agent. Getting the most out of models requires maximally exploiting parallelism, so sandboxing is a must.
Can you share a link to your repo? I am curious to understand the architecture here, and what the prompts you're using are like that enable GPT-5 to have that kind of emergent behavior.
What kind of tasks were successful? What kinds of tasks failed? Have you tried deeply "get me the data" tasks say for example "assemble me the time series of cash flows associated with the Japanese Soverieng Debt".
Something I have been struggling with is that for "go and look and find this and join it and then see" kind of exercises really seem to go way way off the rails and Claude in particular likes to "mock" anything that gets a little bit hard even if I scream "NEVER MOCK DATA" in my agents file.
I would be curious if there are tricks in these long running loops like constantly injecting new "DO NOT MOCK" shouts at it every n cycles or something.
For lightweight sandboxing on Linux you can use bubblewrap or firejail instead of Docker. They are faster and _simpler_. Here is a bwrap script I wrote to run Claude in a minimal sandbox an hour back:
This is gold. Thanks so much for sharing. Being able to put claude in handcuffs and use --dangerously-skip-permissions without concern is a game changer!
On one hand, folks are complaining that even basic tasks are impossible with LLMs.
On the other hand we have cursed language [0] which was fully driven by AI and seems to be functional. Btw, how much did that cost?
I feel like I've been hugely successful tools like Claude Code, aider, open code. Especially when I can define custom tools. "You" have to be a part of the loop, in some capacity. To provide guidance and/or direction. I'm puzzled by the fact that people are surprised by this. When I'm working with other entities (people) who are not artificially intelligent, the majority of the time is spent clarifying requirements and aligning on goals. Why would it be different with LLMs?
This is great. I feel like most of the oxygen is going to go to the sandboxing question (fair enough). But I'm kind of obsessed with what agent loops for engineering tasks that aren't coding look like, and also the tweaks you need for agent loops that handle large amounts of anything (source code lines, raw metrics or oTel span data, whatever).
There was an interval where the notion of "context engineering" came into fashion, and we quickly dunked all over it (I don't blame anybody for that; "prompt engineering" seemed pretty cringe-y to me), but there's definitely something to the engineering problems of managing a fixed-size context window while iterating indefinitely through a complex problem, and there's all sorts of tricks for handling it.
I use agentic loops for document and spreadsheet (aka bullshit to interface with non engineers in the company) since my model of choice is adept at markdown and csv, and the office software can import and export those.
I can imagine an agentic loop that updates dependencies à la Dependabot/Renovate-style by going through the changelog of a new version, reviewing new code changes, and evaluating whether it's worth it to upgrade (or even dangerous to do so, either from stability or security point of view). Too often these tools are used to blindly respin builds with the latest and greatest versions, which is what gets most people in trouble when their NPM deps become malicious.
We have a GH workflow that spins up Claude with a prompt like this and it works quite well:
—
You are going to create a summary of the important changes for a PR created by
dependabot:
1. Read the body of PR $ARGUMENTS with the `gh` cmdline tool.
2. Create a todo list for yourself for each package in PLAN.md.
3. For each todo, read the releases page(s) (add /releases to the github repo
url) to cover the update range.
4. Summarize breaking changes and CVE updates, see template at the end. Grouped
into Dev Dependencies and Production Dependencies.
5. For production dependencies, find code usages (with `rg`/`grep`) and
determine if breaking changes will affect us. Add a list of all found usages,
linking to the direct line on github (like
https://github.com/solarmonkey/app/blob/master/frontend/some...).
Finish each with a verdict: unsure/likely OK/*may break*/*SURELY
BREAKING*!
6. Fill the table up top with the verdicts
7. Write the filled out template to `deps_update_analysis.md`. Don't add a final
summary or conclusive thoughts. The template is all we need.
Huggingface has a series on Tiny Agents[1] where they go into a minimal version of the loop with tools.
I’ve been working on a weekend project building one of these and the tools you provide are what makes the agent suitable for the task.
Even small models and non tool models can use MCP with correct prompting, but you have to edit what tools they can use.
Context is the big bottleneck to manage. Programming agents benefit from knowing shell commands, but generic agents you have to teach/trick into the right interactions, which burns context. Then out of the box an agent will want to read everything, so I have to comment which commands are expensive or cheap context wise.
My project is almost ready to explore context management methods: things like removing old files from memory, warning the agent about limits, using a second agent that does compaction.
It would be cool to eventually RLHF the agents too, but that’s unlikely with my weekend project budget.
One important issue with agentic loops is that agents are lazy, so you need some sort of retrigger mechanism. Claude code supports hooks, you can wire your agent stop hook to a local LLM, feed the context in and ask the model to prompt Claude to continue if needed. It works pretty well, Claude can override retriggers if it's REALLY sure it's done.
Regarding sandboxing, VMs are the way. Prompt injected agents WILL be able to escape containers 100%.
Wrong takeaway, but claude code was released February of this year??? I swear people have been glazing it for way longer... my memory isn't that bad right?
I'm surprised to see so many people using containers when setting up a KVM is so easy, gives the most robust environment possible, and to my knowledge much has better isolation. A vanilla build of Linux plus your IDE of choice and you're off to the races.
> If anything goes wrong it’s a Microsoft Azure machine somewhere that’s burning CPU and the worst that can happen is code you checked out into the environment might be exfiltrated by an attacker, or bad code might be pushed to the attached GitHub repository.
Isn't that risking getting banned from Azure? The compromised agent might not accomplish anything useful, but its attempts might get (correctly!) flagged by the cloud provider.
A little off-topic but I’ve been waiting for a post by someone who’s built something like Claude Code or Codex that can explain how the state machine works for execution.
Clearly it’s not just a good system prompt and hoping for the best. Obviously they’ve both got fairly good algorithms for working toward whatever the user asked for incrementally, handling expectable errors, changing course if necessary, etc.
I mean ultimately it’s gotta be a really big flow chart that results in the LLM getting prompted various ways in all kinds of directions before ultimately converging on the final response, right? Or am I way off-base?
Has anyone seen such a post explaining this architecture / state machine / algorithm?
My preferred ergonomics for agents are like using a washing machine: choose what clothes you want to have cleaned (whites or colors..) and press “go”.
Claude Code plan mode is the washing machine of agentic loops. It lets you see a detailed plan and if you approve you can let it rip with trust that it’ll do what it says, and go do something else while it’s at work.
Adding tests and other ways to validate its work help it go further & run longer off a single plan.
> Update: It turns out Anthropic have their own documentation on Safe YOLO mode for Claude Code which says:
> Letting Claude run arbitrary commands is risky and can result in data loss, system corruption, or even data exfiltration (e.g., via prompt injection attacks). To minimize these risks, use --dangerously-skip-permissions in a container without internet access. You can follow this reference implementation using Docker Dev Containers. [https://github.com/anthropics/claude-code/tree/main/.devcont...]
And… that link goes to a devcontainer that firewalls itself from inside using, effectively, sudo iptables. If Claude Code can’t break that all by itself in a single try, I’d be a bit surprised.
Resonating with the statement "Designing agentic loops is a very new skill—Claude Code was first released in just February 2025!" when I was investigating Codex[0].
In case anyone else is confused: I saw the word "loop" and wondered what the point is to run an LLM in a loop? Like, what's the goal here? You want to write N different React todo web apps, or something?
After reading the article a bit, I realized this isn't about "loops" at all. What it is about is "bypassing the yes/no human check on each step the LLM wants to take in completing some multi-step task". You can already to this with, for example, claude code. However, one of the points of having a human approve LLM actions is to prevent it doing something stupid or destructive. So now you have a second problem: how to prevent the LLM from poking holes in spacetime?
37 comments
[ 2.1 ms ] story [ 53.8 ms ] threadAny reason why you want to rename it?
Edit: to say more about my opinions, "agentic loop" could mean a few things -- it could mean the thing you say, or it could mean calling multiple individual agents in a loop ... whereas "agentic harness" evokes a sort of interface between the LLM and the digital outside world which mediates how the LLM embodies itself in that world. That latter thing is exactly what you're describing, as far as I can tell.
My agent has a few other tricks up its sleeve and it's very new, so I'm still experimenting with lots of different ideas, but there are a few things I noticed.
One is that GPT-5 is extremely willing to speculate. This is partly because of how I prompt it, but it's willing to write scripts that try five or six things at once in a single script, including things like reading files that might not exist. This level of speculative execution speeds things up dramatically especially as GPT-5 is otherwise a very slow model that likes to think about things a lot.
Another is that you can give it very complex "missions" and it will drive things to completion using tactics that I've not seen from other agents. For example, if it needs to check something that's buried in a library dependency, it'll just clone the upstream repository into its home directory and explore that to find what it needs before going back to working on the user's project.
None of this triggers any user interaction due to running in the container. In fact, no user interaction is possible. You set it going and do something else until it finishes. The model is very much to queue up "missions" that can then run in parallel and you merge them together at the end. The agent also has a mode where it takes the mission, writes a spec, reviews the spec, updates the spec given the review, codes, reviews the code, etc.
Even though it's early days I've set this agent missions that it spent 20 minutes of continuous uninterrupted inferencing time on, and succeeded excellently. I think this UI paradigm is the way to go. You can't scale up AI assisted coding if you're constantly needing to interact with the agent. Getting the most out of models requires maximally exploiting parallelism, so sandboxing is a must.
Something I have been struggling with is that for "go and look and find this and join it and then see" kind of exercises really seem to go way way off the rails and Claude in particular likes to "mock" anything that gets a little bit hard even if I scream "NEVER MOCK DATA" in my agents file.
I would be curious if there are tricks in these long running loops like constantly injecting new "DO NOT MOCK" shouts at it every n cycles or something.
On the other hand we have cursed language [0] which was fully driven by AI and seems to be functional. Btw, how much did that cost?
I feel like I've been hugely successful tools like Claude Code, aider, open code. Especially when I can define custom tools. "You" have to be a part of the loop, in some capacity. To provide guidance and/or direction. I'm puzzled by the fact that people are surprised by this. When I'm working with other entities (people) who are not artificially intelligent, the majority of the time is spent clarifying requirements and aligning on goals. Why would it be different with LLMs?
0: https://ghuntley.com/cursed/
There was an interval where the notion of "context engineering" came into fashion, and we quickly dunked all over it (I don't blame anybody for that; "prompt engineering" seemed pretty cringe-y to me), but there's definitely something to the engineering problems of managing a fixed-size context window while iterating indefinitely through a complex problem, and there's all sorts of tricks for handling it.
—
You are going to create a summary of the important changes for a PR created by dependabot:
1. Read the body of PR $ARGUMENTS with the `gh` cmdline tool. 2. Create a todo list for yourself for each package in PLAN.md. 3. For each todo, read the releases page(s) (add /releases to the github repo url) to cover the update range. 4. Summarize breaking changes and CVE updates, see template at the end. Grouped into Dev Dependencies and Production Dependencies. 5. For production dependencies, find code usages (with `rg`/`grep`) and determine if breaking changes will affect us. Add a list of all found usages, linking to the direct line on github (like https://github.com/solarmonkey/app/blob/master/frontend/some...). Finish each with a verdict: unsure/likely OK/*may break*/*SURELY BREAKING*! 6. Fill the table up top with the verdicts 7. Write the filled out template to `deps_update_analysis.md`. Don't add a final summary or conclusive thoughts. The template is all we need.
[snip template to fill]
I’ve been working on a weekend project building one of these and the tools you provide are what makes the agent suitable for the task.
Even small models and non tool models can use MCP with correct prompting, but you have to edit what tools they can use.
Context is the big bottleneck to manage. Programming agents benefit from knowing shell commands, but generic agents you have to teach/trick into the right interactions, which burns context. Then out of the box an agent will want to read everything, so I have to comment which commands are expensive or cheap context wise.
My project is almost ready to explore context management methods: things like removing old files from memory, warning the agent about limits, using a second agent that does compaction.
It would be cool to eventually RLHF the agents too, but that’s unlikely with my weekend project budget.
[1] https://huggingface.co/learn/mcp-course/en/unit2/tiny-agents
Regarding sandboxing, VMs are the way. Prompt injected agents WILL be able to escape containers 100%.
And for more context on Efrit this is a fun watch: "When Steve Gives Claude Full Access To 50 Years of Emacs Capabilities" https://www.youtube.com/watch?v=ZJUyVVFOXOc
Isn't that risking getting banned from Azure? The compromised agent might not accomplish anything useful, but its attempts might get (correctly!) flagged by the cloud provider.
Is it just me ?
Clearly it’s not just a good system prompt and hoping for the best. Obviously they’ve both got fairly good algorithms for working toward whatever the user asked for incrementally, handling expectable errors, changing course if necessary, etc.
I mean ultimately it’s gotta be a really big flow chart that results in the LLM getting prompted various ways in all kinds of directions before ultimately converging on the final response, right? Or am I way off-base?
Has anyone seen such a post explaining this architecture / state machine / algorithm?
https://github.com/codazoda/llm-jail
Claude Code plan mode is the washing machine of agentic loops. It lets you see a detailed plan and if you approve you can let it rip with trust that it’ll do what it says, and go do something else while it’s at work.
Adding tests and other ways to validate its work help it go further & run longer off a single plan.
> Update: It turns out Anthropic have their own documentation on Safe YOLO mode for Claude Code which says:
> Letting Claude run arbitrary commands is risky and can result in data loss, system corruption, or even data exfiltration (e.g., via prompt injection attacks). To minimize these risks, use --dangerously-skip-permissions in a container without internet access. You can follow this reference implementation using Docker Dev Containers. [https://github.com/anthropics/claude-code/tree/main/.devcont...]
And… that link goes to a devcontainer that firewalls itself from inside using, effectively, sudo iptables. If Claude Code can’t break that all by itself in a single try, I’d be a bit surprised.
[0]: https://blog.toolkami.com/openai-codex-tools/#coding-agents
Create a group for you and the `claude` user, and have it do `umask 002` on login.
The shellagent use case has much more parallels with 1980s TTY+mainframes dealing with multiple users than it has with containers.
I.e. Docker's USP was reproducibility; not access controls.
where do i sign up?
After reading the article a bit, I realized this isn't about "loops" at all. What it is about is "bypassing the yes/no human check on each step the LLM wants to take in completing some multi-step task". You can already to this with, for example, claude code. However, one of the points of having a human approve LLM actions is to prevent it doing something stupid or destructive. So now you have a second problem: how to prevent the LLM from poking holes in spacetime?
Answer: sandbox it.
And that's what the article is about.