stream from the SSD, perform the calculation, discard, repeat
TLDR I took a stab at leveraging Dan's work and making it more practical: https://github.com/matt-k-wong/mlx-flash 2 bit quantization lobotomizes the model but is impressive nonetheless! Maybe one day we'll be able to…
Very cool!
stream from the SSD, perform the calculation, discard, repeat
TLDR I took a stab at leveraging Dan's work and making it more practical: https://github.com/matt-k-wong/mlx-flash 2 bit quantization lobotomizes the model but is impressive nonetheless! Maybe one day we'll be able to…
Very cool!