mhutchw

I’m trying to build a side project in the inference space. I’ve been talking to a few inference engineers and startups and I’ve been hearing how annoying it is to not have manual control over the KV cache at times and…