I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.
I have the Air too. It's thermal throttled. Try someone's M1 Pro, or the Mac Mini. Also make sure you're using the GPU (I'm not sure but AFAIK these models can run on CPU too?).
People at my employer use maxxed out Macbook Pro M2 for their ML research needs and say you can't get better performance for that price (in a laptop).
5 comments
[ 3.6 ms ] story [ 20.6 ms ] threadPeople at my employer use maxxed out Macbook Pro M2 for their ML research needs and say you can't get better performance for that price (in a laptop).