I do not host LLMs specifically, but almost everything I use is hosted on my own hardware. A Dell R720 costs maybe $500 fully kitted out give or take a few hundred. Plenty of my friends are hosting LLMs on their laptops.
I don't put my stuff on other people's computers unless there's no alternative.
1 comment
[ 2.7 ms ] story [ 9.6 ms ] threadI don't put my stuff on other people's computers unless there's no alternative.