Show HN: Streaming Structured Output Is Still Broken (schnabl.cx)

1 points by chrissdot ↗ HN
Structured output has become a de facto expectation for LLM APIs, but streaming that structured output is still a mess. A big miss, streaming partial results in parallel can nearly double agent throughput.

OpenAI, Anthropic, and Gemini handle streaming structured responses, but each breaks developer expectations differently: missing parsed results, losing final token stats, or adding unnecessary boilerplate.

0 comments

[ 4.5 ms ] story [ 14.1 ms ] thread

No comments yet.