dacharyc
No user record in our sample, but dacharyc has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but dacharyc has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
I suspect this is a result of relevance-based retrieval. In my colleague's testing, they found that sometimes the content comes back out of order or not at all, depending on the implementation's interpretation of which…
Ah, good point - this was intended to be a bonus point for agents that do not use a working browser, to evaluate whether they understood and communicated that the content was missing. But it should be an either/or - not…
Yeah, my colleague and I have been seeing in testing how much this is actually a problem in practice. It has been - surprising, and a little dismaying - how much this negatively impacts content retrieval and results in…
Hah, I actually originally had some stuff in the site that Claude Code's summarization agent (presumably Haiku) thought was prompt injection, and refused to give content to the foreground agent I was working with. I had…
Hah, that's actually what drove me to try to create this to begin with. I've been writing a lot about these issues, and someone said to me: > It'd be nice to have a test harness: "Test my agent," to score them and give…
Yeah, good call, we're on the same page about that. I designed this tool (agentreadingtest.com) to raise awareness of these issues in a more general way, so people can point agents at it and see how it performs for…
Hey there - I'm the test author, and you've hit on one of the main points. For the summarization/relevance-based content return, this is a consideration for some of the agent platforms (although I've found others…
Actually, according to my count, there are 9 other companies in this space that offer deploy environments as a service, in addition to 6 platforms that also offer this functionality as part of the platform. I've been…
There are a bunch of services that already do this in some permutation or another: Tugboat: https://tugboat.qa/ Squash: https://www.squash.io/ Dockup: https://getdockup.com/ Release: https://www.releaseapp.io/…