It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
I told DeepSeek to read the article and investigate links and terms in the article.
The gist of it is that uses a bunch of skills from a solution called ironbee in a loop iterate with LLMs to help them solve the tasks better by improving the feedback loop via browser-testing, debugging, visual-testing, etc..
From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..
If the agent never sees the hidden tests, what is the verifier actually checking against? Trying to understand what "verification" means here concretely.
7 comments
[ 5.8 ms ] story [ 35.7 ms ] threadThe gist of it is that uses a bunch of skills from a solution called ironbee in a loop iterate with LLMs to help them solve the tasks better by improving the feedback loop via browser-testing, debugging, visual-testing, etc..
The skills can be found here:
https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...
Question that was left unanswered:
How much longer does a 4x DeepSkeep loop took compared to Opus?
The skills can be found here:
https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...
Question that was left unanswered:
How much longer does a 4x DeepSkeep loop took compared to Opus?