Ask HN: How to store KV cache efficiently on disk? 1 points by ban-lan-gen 1y ago ↗ HN If I have a lot of precomputed KV cache values (basically just tensors), how do I store them efficiently on disk? Ideally, I would want to lookup their values by prefix quickly as well.
0 comments
[ 4.9 ms ] story [ 12.8 ms ] threadNo comments yet.