Trained KV cache bank turns any LLM into Jev like Model (gambler-relay-us-west1.leo-fish.ts.net)

Here you can see a 26B model hacked using a trained KV cache bank, responding like Gemma. The latency I am seeing is <127 ms.

That's awesome I came from the jev in python thread. is the model shareable? do you have any more writing on this? would love to read more about it! cool demo anyways.