[dead]
Exactly! Number of turns, average tokens to achieve a task using your CLI, as well as average number of characters being returned per CLI command alongside other metrics: all important to both users and agents! I am…
Working on testing and monitoring agent-readiness of CLIs with www.cliwatch.com, some interesting challenges in building test suites and analyzing data :)
Also worth pushing for a more standardized skills command for CLIs, similar to —help, but for (agent/human) workflows, https://cliwatch.com/blog/designing-a-cli-skills-protocol (if you ship these with your CLI, you also…
Resource allocation based on your hackernews upvotes? Thanks in advance folks ;)
See also https://cliwatch.com/blog/designing-a-cli-skills-protocol
We are working on supporting agent harnesses @ www.cliwatch.com, so both 1. LLM model as well 2. LLM model + harness performance can be evaluated against your software/CLI. We also support building evals against your…
Building www.cliwatch.com, so you can keep an eye on how agent-friendly your CLI is ;) feel free to request a benchmark against your CLI docs. Cheers
cliwatch.com, creating some benchmarks, reach out if you are interested :)
In a similar fashion it appears that article was automated - did the author read every word in their own article?
Not sure about the end of thinking, would say that this is the start of managing ever more stochastic systems
[dead]
Exactly! Number of turns, average tokens to achieve a task using your CLI, as well as average number of characters being returned per CLI command alongside other metrics: all important to both users and agents! I am…
Working on testing and monitoring agent-readiness of CLIs with www.cliwatch.com, some interesting challenges in building test suites and analyzing data :)
Also worth pushing for a more standardized skills command for CLIs, similar to —help, but for (agent/human) workflows, https://cliwatch.com/blog/designing-a-cli-skills-protocol (if you ship these with your CLI, you also…
Resource allocation based on your hackernews upvotes? Thanks in advance folks ;)
See also https://cliwatch.com/blog/designing-a-cli-skills-protocol
We are working on supporting agent harnesses @ www.cliwatch.com, so both 1. LLM model as well 2. LLM model + harness performance can be evaluated against your software/CLI. We also support building evals against your…
Building www.cliwatch.com, so you can keep an eye on how agent-friendly your CLI is ;) feel free to request a benchmark against your CLI docs. Cheers
cliwatch.com, creating some benchmarks, reach out if you are interested :)
In a similar fashion it appears that article was automated - did the author read every word in their own article?
Not sure about the end of thinking, would say that this is the start of managing ever more stochastic systems
[dead]
[dead]
[dead]