Ask HN: How to store KV cache efficiently on disk?

1 points by ban-lan-gen ↗ HN
If I have a lot of precomputed KV cache values (basically just tensors), how do I store them efficiently on disk? Ideally, I would want to lookup their values by prefix quickly as well.

0 comments

[ 3.7 ms ] story [ 8.2 ms ] thread

No comments yet.