Hi Satoshi
What was the point of Claude code or Gemini banning the OP? Why would they care about how IDEs use the underlying API?
That snippet is trying to say that you can calculate KV for all the input tokens at once, and you don't need to loop over them since you have them all available. Instead for decode, you need to sequentially generate…
Prefill is part of Inference. It's the first major step where you calculate all the keys and values for the input tokens. Decode is the next major step where you start generating output tokens one at a time. Both run on…
One of us, one of us...
Hi Satoshi
What was the point of Claude code or Gemini banning the OP? Why would they care about how IDEs use the underlying API?
That snippet is trying to say that you can calculate KV for all the input tokens at once, and you don't need to loop over them since you have them all available. Instead for decode, you need to sequentially generate…
Prefill is part of Inference. It's the first major step where you calculate all the keys and values for the input tokens. Decode is the next major step where you start generating output tokens one at a time. Both run on…
One of us, one of us...