Ask HN: How to Fine Tune LLMs? 2 points by krishadi 3y ago ↗ HN I wanted to play around with fine tuning LLMs on my Mac. Any recommendations on how (or) where to start? Is there a blog you'd suggest?
[–] PaulHoule 3y ago ↗ These instructions got me startedhttps://huggingface.co/docs/transformers/trainingwith the caveat that the default training rate is too high and you get better results with 2e-5 [–] krishadi 3y ago ↗ Great! Thanks...
[–] verdverm 3y ago ↗ There are a number of ways, with increasing methods and capabilities. It's going to depend on what you are trying to do and the constraints you haveGoogle put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663
3 comments
[ 4.7 ms ] story [ 14.9 ms ] threadhttps://huggingface.co/docs/transformers/training
with the caveat that the default training rate is too high and you get better results with 2e-5
Google put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663