Honestly, I was very impressed with this. I did find the latency was quite high, so I tried to tune it to use MLX on Mac GPUs, and it's so much faster. Very cool either way, we're getting close to a full AI interactive tool that's offline and private!
1 comment
[ 0.25 ms ] story [ 1.9 ms ] threadhttps://github.com/skorotkiewicz/lfm3-chat/pull/1