4 comments

[ 3.0 ms ] story [ 21.9 ms ] thread
It is very fast, but the rate of hallucinations is also very high.
It’s a small model. A 70B or larger model on a chip would be much more impressive and hopefully would have the same speed