In-browser LLM inference engine with WebGPU and OpenAI API (blog.mlc.ai)

Comes with ability to do full structured generation with json schema

also a in-browser demo https://chat.webllm.ai/

It's good to see that it contains the standard OpenAI API interface in JavaScript which looks very convenient.

AMAZING!

So cool. No more privacy concerns!