Strongly agree with this approach. Real world task specific LLM query workloads tend to be long lived workflows. They should be brokered and handled rather than wait on.
This is perfect for my use case! I'm building a MCP tool that can take 4~10 minutes to complete, and I'm using exactly what you described in README (having another MCP tool for models to poll results) to solve the async problem, which is not ideal. Hope this gets adopted more widely!
2 comments
[ 0.34 ms ] story [ 16.6 ms ] thread