Bonsai: A 27B reasoning model on a 16 GB M2 Mac, with Ferrox (medium.com) 1 points by antonellof 5d ago ↗ HN
[–] antonellof 5d ago ↗ Run a 27B reasoning model locally on a 16GB M2 Mac. Ferrox + ternary quantization delivers 5.9GB models without sacrificing much quality.
1 comment
[ 2.8 ms ] story [ 66.7 ms ] thread