HN Explorer

Ask HN: Who is doing distillation or fine tuning?

2 points by mrburton ↗ HN
If you're doing fine tuning, how are you curating or creating your training data? If you're doing distillation what tools? Share you're experience.

That includes you PewDiePie; Felix Kjellberg.

1 comment of 2

[ 0.29 ms ] story [ 6.9 ms ] thread
svcrunch
For me, training data comes from agent runs.

I do fine tuning with pytorch, and RL-forms of distillation with Tinker! I hope that helps. I work with a full stack of models that enable agentic AI: embedders, rerankers, LLMs, decision models.

Curious, why are you asking?