Show HN: Agent Caching in Fiddler (telerik.com)

2 points by zlatkov ↗ HN
Hi HN!

We’re the team behind Fiddler, and we recently built an Agent Cache feature.

It works by capturing an LLM request/response pair and, once caching is enabled, serving subsequent matching calls locally so duplicate requests never hit the provider.

It’s meant to reduce three pains that show up together in agent development: token cost, feedback-loop latency, and output non-determinism.

We’d love to hear your thoughts.

1 comment

[ 4.0 ms ] story [ 11.5 ms ] thread
Cool idea. I have had rather a bad experience with semantic caching. Do you have benchmarks that demonstrate the effectiveness?