Ask HN: What's the current state of in-browser GPU accelerated LLM inference? 6 points by ramesh31 2y ago ↗ HN Curious if anything is approaching usability at GPT3+ levels, even with a large binary download.
[–] rustystump 2y ago ↗ I don't think there has been much change since this. https://news.ycombinator.com/item?id=35583349WebGPU is getting close to general support which will make things a bit faster but compute isn't as much of the issue as ram.
1 comment
[ 4.1 ms ] story [ 14.8 ms ] threadWebGPU is getting close to general support which will make things a bit faster but compute isn't as much of the issue as ram.