Does anyone know what the impact on prompt caching is?
I like the idea of dynamic tool/skill sets, but wonder if it needs to be part of the new session workflow where you would choose a group to activate rather than this dynamic.
I'm pretty sure it's a bad idea because each time you connect and disconnect tools, you are hitting a full cache miss on the full context, that will be probably more expensive than having these tools in place from the very beginning.
9 comments
[ 4.8 ms ] story [ 30.3 ms ] threadI like the idea of dynamic tool/skill sets, but wonder if it needs to be part of the new session workflow where you would choose a group to activate rather than this dynamic.
I also really don't see how the hacky command interceptor is that much better than a dedicated tool. You're one which away from a very confused LLM.